February 17, 2022 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by ME&A, Inc. consultants Elisabeth Hayek-Weinmann, Team Leader, Selma Tijerino, Civil Society Specialist, and Juan Pablo Fontán, Public Administration Specialist. FINAL PERFORMANCE EVALUATION OF USAID/EL SALVADOR GOVERNMENT INTEGRITY PROJECT FINAL PERFORMANCE EVALUATION OF USAID/EL SALVADOR GOVERNMENT INTEGRITY PROJECT Submitted on: February 17, 2022 Contract Number: AID-OAA-I-15-00024/AID-519-TO-16-00002 Activity Start Date and End Date: August 1, 2016 – March 31, 2022 Contracting Officer’s Representative (COR): Annie de Valencia Submitted to: USAID/El Salvador Annie de Valencia, Contracting Officer’s Representative (COR) avalencia@usaid.gov Submitted by: USAID/El Salvador Monitoring, Evaluation, and Learning Initiative Contact: Randal Thompson, Chief of Party rthompson@engl.com Photo Caption: Young volunteer participating in the campaign to collect signatures from citizens in support of the National anti-corruption Agreement in 2018. Photo Credit: Taken by GIP Project DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ABSTRACT The Final Performance Evaluation of the United States Agency for International Development (USAID) Government Integrity Project (GIP) addressed the following six main questions: 1. To what extent has GIP contributed to strengthening oversight institutions’ capacity to monitor, disclose, highlight, discipline, investigate, communicate, or bring attention to public resource use?; 2. What have been the main accomplishments and challenges of the anti-corruption Interinstitutional Working Group (GTIAC) in terms of key actors/champions, coordination of the participant institutions, and sustainability?; 3. To what extent has the activity contributed to better respond to access to public information requirements, implemented more expansive open government and accountability practices, and assisted compliance of ethics standards in selected municipalities, and Government of El Salvador (GOES) agencies (Ministry of Public Works [MOP], Ministry of Health of El Salvador [MINSAL], Ministry of Education, Science, and Technology [MINEDUCYT], Court of Accounts of the Republic [CCR], and Public Defender's Office [PGR])?; 4. To what extent has the assistance provided to civil society organizations (CSOs) contributed to institutionalizing and expanding transparency and citizen oversight?; 5. How do recipients and decision￾makers of the assisted CSOs, municipalities, GOES institutions, and independent regulatory entities (Government Ethics Tribunal [TEG], PGR, and CCR) value the technical assistance provided?; 6. What interventions promoted gender equality and social inclusion during the implementation of GIP? What factors enabled or inhibited the integration of these topics?. Recommendations derived from the evaluation should serve as inputs for learning and continuous improvements during GIP’s remaining implementation period and design of future activities. The evaluation team employed a mixed-methods approach, including document review, key informant interviews, online surveys, focus group discussions, and site visits. The evaluation found that GIP has contributed to the institutional strengthening of targeted government agencies, independent regulatory agencies, and municipalities through specialized trainings, technical support for institutional policy design, process streamlining, online platform development, and promotion of the Institutional Integrity Model. GIP support was crucial for the creation of an anti￾corruption interinstitutional mechanism. With GIP’s support, civil society organizations (CSOs) increased their engagement on transparency and social oversight issues, enhanced inter-organization strategic alliances, and established collaborative relationships with public institutions. Advances regarding the strengthening of public institutions have suffered setbacks because of the political context hostile to transparency and accountability issues. The sustainability of CSOs’ expanded engagement also is at risk in this context. TABLE OF CONTENTS EXECUTIVE SUMMARY................................................................................................................................. 1 INTRODUCTION ........................................................................................................................................... 1 METHODOLOGY ........................................................................................................................................... 2 MAIN FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS........................................................................... 2 1. EVALUATION BACKGROUND, PURPOSE AND AUDIENCE .................................................... 1 1.1 EVALUATION BACKGROUND AND ACTIVITY OVERVIEW .......................................................... 1 1.2 EVALUATION PURPOSE AND AUDIENCE ..................................................................................... 3 2. EVALUATION QUESTIONS ................................................................................................................... 4 3. METHODOLOGY...................................................................................................................................... 5 3.1 GENERAL EVALUATION APPROACH ............................................................................................. 5 3.2 DATA COLLECTION METHODS ....................................................................................................... 5 3.2.1 Document Review................................................................................................................................ 5 3.2.2 Key Informant Interviews................................................................................................................... 5 3.2.3 Online Surveys....................................................................................................................................... 5 3.2.4 Focus Group Discussions.................................................................................................................... 6 3.2.5 Direct Observation and Site Visits.................................................................................................... 6 3.2.6 Co-Creation Workshops..................................................................................................................... 7 3.3 SAMPLING METHODS ........................................................................................................................... 7 3.3.1 Key Informant Interviews................................................................................................................... 7 3.3.2 Online Surveys....................................................................................................................................... 7 3.3.3 Focus Group Discussions.................................................................................................................... 7 3.3.4 Site Visits................................................................................................................................................ 7 3.4 GENDER AND SOCIAL INCLUSION ANALYSIS ........................................................................... 7 3.5 DATA ANALYSIS METHODOLOGY ................................................................................................. 8 3.6 METHODOLOGICAL LIMITATIONS ................................................................................................ 8 4. FINDINGS & CONCLUSIONS ............................................................................................................... 9 4.1 EQ1: TO WHAT EXTENT HAS GIP CONTRIBUTED TO STRENGTHENING OVERSIGHT INSTITUTIONS’ CAPACITY TO MONITOR, DISCLOSE, HIGHLIGHT, DISCIPLINE, INVESTIGATE, COMMUNICATE, OR BRING ATTENTION TO PUBLIC RESOURCE USE? ........................................................................................ 9 4.1.1 Government’s Ethics Tribunal Findings........................................................................................... 9 4.1.2 Government’s Ethics Tribunal Conclusions.................................................................................10 4.1.3 Institute for Access of Public Information Findings...................................................................10 4.1.4 Institute for Access to Public Information Conclusions.............................................................11 4.1.5 Public Defender’s Office Findings..................................................................................................11 4.1.6 Public Defender’s Office Conclusions...........................................................................................11 4.1.7 Court of Accounts of the Republic Findings (CCR) ...................................................................12 4.1.8 Court of Accounts of the Republic Conclusions.........................................................................12 4.1.9 Attorney General’s Office Findings................................................................................................12 4.1.10 Supreme Court of Justice Findings ..........................................................................................13 4.2 EQ2: WHAT HAVE BEEN THE MAIN ACCOMPLISHMENTS AND CHALLENGES OF THE ANTI￾CORRUPTION INTERINSTITUTIONAL WORKING GROUP (GTIAC) IN TERMS OF KEY ACTORS/CHAMPIONS, COORDINATION OF THE PARTICIPANT INSTITUTIONS, AND SUSTAINABILITY? ................................................................................................................................................................... 13 4.2.1 Findings................................................................................................................................................13 4.2.2 Conclusions.........................................................................................................................................15 4.3 EQ3: TO WHAT EXTENT HAS THE ACTIVITY CONTRIBUTED TO BETTER RESPOND TO ACCESS TO PUBLIC INFORMATION REQUIREMENTS, IMPLEMENTED MORE EXPANSIVE OPEN GOVERNMENT AND ACCOUNTABILITY PRACTICES, AND ASSISTED COMPLIANCE OF ETHICS STANDARDS IN: ................... 15 4.3.1 Municipalities Findings.....................................................................................................................15 4.3.2 Municipalities Conclusions..............................................................................................................16 4.3.3 Ministry of Public Works Findings................................................................................................. 17 4.3.4 Ministry of Public Works Conclusions..........................................................................................18 4.3.5 Ministry of Education and Ministry of Health Findings...........................................................18 4.3.6 Ministry of Education and Ministry of Health Conclusions....................................................19 4.4 EQ4: TO WHAT EXTENT HAS THE ASSISTANCE PROVIDED TO CIVIL SOCIETY ORGANIZATIONS CONTRIBUTED TO INSTITUTIONALIZING AND EXPANDING TRANSPARENCY AND CITIZEN OVERSIGHT?............................................................................................................................................... 19 4.4.1 Findings................................................................................................................................................19 Academic Institutions and Think Tanks 19 Advocacy Organizations 21 Grassroots Organizations 21 4.4.2 Conclusions.........................................................................................................................................22 Academic Institutions, Think Tanks, and Advocacy Organizations 22 Grassroots Organizations 23 4.5 EQ4.1: WHAT HAVE BEEN THE KEY CHALLENGES FACED BY CSOS TO OVERSIGHT THE USE OF PUBLIC FUNDS? .......................................................................................................................................... 23 4.5.1 Findings................................................................................................................................................23 4.5.2 Conclusions.........................................................................................................................................24 4.6 EQ4.2: TO WHAT EXTENT IS CSOS’ ENGAGEMENT LIKELY TO CONTINUE AND BE SCALED UP AFTER THE GIP ENDS? ......................................................................................................................................... 24 4.6.1 Findings................................................................................................................................................24 4.6.2 Conclusions.........................................................................................................................................25 4.7 EQ4.3: TO WHAT EXTENT HAS THE ASSISTANCE PROVIDED BY GIP CONTRIBUTED TO IMPROVING THE ORGANIZATIONAL CAPACITIES OF BENEFICIARY CSOS? ............................................................... 25 4.7.1 Findings................................................................................................................................................25 4.7.2 Conclusions.........................................................................................................................................25 4.8 EQ5: HOW DO RECIPIENTS AND DECISION-MAKERS OF THE ASSISTED CSOS, MUNICIPALITIES, GOES INSTITUTIONS, AND INDEPENDENT REGULATORY ENTITIES (TEG, PGR, AND CCR) VALUE THE TECHNICAL ASSISTANCE PROVIDED? ............................................................................................... 26 4.8.1 Municipalities Findings.....................................................................................................................26 4.8.2 Municipalities Conclusions..............................................................................................................29 4.8.3 Central Government Agencies Findings.......................................................................................29 4.8.4 Central Government Agencies Conclusions.................................................................................31 4.8.5 Independent Regulatory Institutions Findings.............................................................................31 4.8.6 Independent Regulatory Institutions Conclusions......................................................................34 4.8.7 Civil Society Organizations Findings..............................................................................................34 4.8.8 CSOS Conclusions.............................................................................................................................36 4.9 EQ5.1: WHAT ARE THE HINDRANCES AND BARRIERS THAT RECIPIENTS FACE IN APPLYING THE NEW KNOWLEDGE GAINED THROUGH GIP? .................................................................................................. 36 4.9.1 Findings................................................................................................................................................36 4.9.2 Conclusions.........................................................................................................................................37 4.10EQ6: WHAT INTERVENTIONS PROMOTED GENDER EQUALITY AND SOCIAL INCLUSION DURING THE IMPLEMENTATION OF GIP? WHAT FACTORS ENABLED OR INHIBITED THE INTEGRATION OF THESE TOPICS? ...................................................................................................................................................... 38 4.10.1 Findings..........................................................................................................................................38 4.10.2 Conclusions....................................................................................................................................39 5. RECOMMENDATIONS ........................................................................................................................ 39 5.1 GOVERNMENT & INDEPENDENT REGULATORY INSTITUTIONS............................................................ 39 5.2 CIVIL SOCIETY ORGANIZATIONS............................................................................................................ 40 CROSS-CUTTING RECOMMENDATION............................................................................................................ 40 ANNEXES ....................................................................................................................................................... 43 ANNEX 1: EVALUATION STATEMENT OF WORK............................................................................. 44 ANNEX 2: BIBLIOGRAPHY .......................................................................................................................... 60 ANNEX 3: LIST OF KII PARTICIPANTS .......................................................................................................... 65 ANNEX 4: LIST OF FGD PARTICIPANTS.................................................................................................... 69 ANNEX 5: LIST OF CO-CREATION WORKSHOP PARTICIPANTS ............................................................ 71 ANNEX 6: KII AND FGD GUIDES .................................................................................................................. 72 ANNEX 7: OLS QUESTIONNAIRES ...................................................................................................... 75 LIST OF TABLES Table A1: Summary of Findings, Conclusions, and Recommendations iii Table 1: GIP’s Objectives, Components, and Expected Results (ERs) 2 Table 2: FGDs by Organization and Number of Participants 6 Table 3: Site Visits 6 Table 4: Perceived Quality of Training Received by Municipals PIOs (N=30) 26 Table 5: Perception of Municipal PIO on the Application of New Knowledge (N=30) 27 Table 6: Perceived Quality of Training Received by PIOs of Central Government Agencies (N=16) 28 Table 7: Perceived Quality of Training Received by ARMOs of Central Government Agencies (N=18) 29 Table 8: Perception of PIO of Central Government Agencies on the Application of New Knowledge (N=16) 29 Table 9: Perception of ARMO of Central Government Agencies on the Application of New Knowledge (N=18) 30 Table 10: Perceived Quality of Training Received by Members of Governmental Ethics Commissions (N=15) 31 Table 11: Perception of the Members of the Ethics Commissions on the Application of New Knowledge (N=15) 31 Table 12: Perceived Quality of Training Received by Young Women Beneficiaries on DAIP and Citizen Oversight (N=9) 33 LIST OF FIGURES Figure 1: Perception of Municipal PIO on the Application of New Knowledge (N=30) 27 Figure 2: Perception of by Members of Governmental Ethics Commissions on the Application of New Knowledge (N=15) 32 Figure 3: Perceived Applicability of the Training Received by Young Women Beneficiaries on DAIP and Citizen Oversight (N=9) 34 ACRONYMS ALAC Spanish acronym for Centro de Asesoría Legal Anticorrupción (Legal Aid and anti-corruption Office) ARMO Archive and Records Management Officer CAPRES Spanish acronym for Casa Presidencial (Office of the Presidency of the Republic) CCNIS Spanish acronym for Consejo Coordinador Nacional Indígena Salvadoreño (Salvadorean National Council of Indigenous Population) CCR Spanish acronym for Corte de Cuentas de la República (Court of Accounts of the Republic) CDCS Country Development Cooperation Strategy CICIES Spanish acronym for Comisión Internacional contra la Impunidad en El Salvador (International Commission against Impunity in El Salvador – Organization of American States) CLIN Contract Line Number CONEXION Spanish acronym for Asociación Conexión al Desarrollo de El Salvador (Association for the Connection to Development of El Salvador) COR Contracting Officer’s Representative COVID-19 Coronavirus Disease 2019 CSJ Spanish acronym for Corte Suprema de Justicia (Supreme Court of Justice) CSO Civil Society Organization CUM Spanish acronym for Centro Universitario de Monitoreo (University Monitoring Center) DAIP Spanish acronym for Derecho al Acceso a la Información Pública (Right of Access to Public Information) DGS Democratic Governance and Security Office DTJ Spanish acronym for Fundación Democracia, Transparencia y Justicia (Foundation for Democracy, Transparency, and Justicia) EQ Evaluation Question ER Expected Result ET Evaluation Team FGD Focus Group Discussion FGR Spanish Acronym for Fiscalía General de la República (Attorney General’s Office) FODES Spanish acronym for Fondo para el Desarrollo Económico y Social de los Municipios (Economic and Social Development for Municipalities Fund) FUNDE Spanish acronym for Fundación Nacional para el Desarrollo (National Foundation for Development) FUSADES Spanish acronym for Fundación Salvadoreña para el Desarrollo Económico y Social (Salvadoran Foundation for the Economic and Social Development) GIP USAID/El Salvador Government Integrity Project GMIES Spanish acronym for Grupo de Monitoreo Independiente de El Salvador (Independent Monitoring Group of El Salvador) GOES Spanish acronym for Gobierno de El Salvador (Government of El Salvador) GTIAC Spanish acronym for Grupo de Trabajo Interinstitucional Anticorrupción de El Salvador (Anti-corruption Interinstitutional Working Group of El Salvador) IAIP Spanish acronym for Instituto de Acceso a la Información Pública (Institute of Access to Public Information) IIM Institutional Integrity Model IP Implementing Partner ISAS Institutional Self-Assessment Scorecard ISD Spanish acronym for Iniciativa Social para la Democracia (Social Initiative for Democracy) ISDEM Salvadoran Institute for Municipal Development IT Information Technology KII Key Informant Interview LAIP Spanish acronym for Ley de Acceso a la Información Pública (Access to Public Information Law) LGBTQI+ Lesbian, Gay, Bisexual, Transgender, Queer, Intersex, and Plus M&E Monitoring and Evaluation MEL Monitoring, Evaluation, and Learning MINEDUCYT Spanish acronym for Ministerio de Educación, Ciencia y Tecnología de El Salvador (Ministry of Education, Science, and Technology) MINSAL Spanish acronym for Ministerio de Salud de El Salvador (Ministry of Health of El Salvador) MOP Spanish acronym for Ministerio de Obras Públicas (Ministry of Public Works) MOU Memorandum of Understanding OCCCR Spanish acronym for Observatorio Ciudadano de la Corta de Cuentas de la República (Citizen Observatory of the Court of Accounts of the Republic) OLS Online Survey PGR Spanish acronym for Procuraduría General de la República (Public Defenders’ Office) PIO Access to Public Information Officer SOW Statement of Work TEG Spanish acronym for Tribunal de Ética Gubernamental (Government Ethics Tribunal) TRACODA Spanish acronym for Transparencia, Contraloría Social y Datos Abiertos (Transparency, Social Controller, and Open Data) UAIP Spanish acronym for Unidad de Acceso a la Información Pública (Access to Public Information Unit) UFG Spanish acronym for Universidad Francisco Gavidia (Francisco Gavidia University) UGDA Spanish acronym for Unidad de Gestión Documental y Archivos (Archive and Records Management Unit) UJMD Spanish acronym for Universidad Dr. José Matías Delgado (Dr. José Matías Delgado University) USAID United States Agency for International Development 1 EXECUTIVE SUMMARY INTRODUCTION This Executive Summary presents an overview of the primary findings, conclusions, and recommendations from the Final Performance Evaluation of the United States Agency for International Development (USAID) Government Integrity Project (GIP) (the activity). The purpose of this evaluation is to inform USAID/El Salvador regarding the activity’s achievements and challenges and determine if the services were implemented as planned. It sought to identify a) GIP’s contributions to the strengthening of public institutions in charge of implementing regulations regarding access to public information, accountability of public resources, and anti￾corruption efforts, and b) its contributions to increase citizens’ oversight capacity, participation and awareness of anti-corruption reforms. USAID/El Salvador will use the evaluation’s findings and recommendations as inputs for learning and continuous improvements during GIP’s remaining implementation period and for the design of future activities. The evaluation answered the following six evaluation questions (EQs) and four sub-EQs: 1. To what extent has GIP contributed to strengthening oversight institutions’ capacity to monitor, disclose, highlight, discipline, investigate, communicate, or bring attention to public resource use? 2. What have been the main accomplishments and challenges of the anti-corruption Interinstitutional Working Group (GTIAC) in terms of key actors/champions, coordination of the participant institutions, and sustainability? 3. To what extent has the activity contributed to better respond to access to public information requirements, implemented more expansive open government and accountability practices, and assisted compliance of ethics standards in: ● Selected municipalities, and ● Government of El Salvador (GOES) agencies (Ministry of Public Works [MOP], Ministry of Health of El Salvador [MINSAL], Ministry of Education, Science, and Technology [MINEDUCYT], Court of Accounts of the Republic [CCR], and Public Defender's Office [PGR])? 4. To what extent has the assistance provided to civil society organizations (CSOs) contributed to institutionalizing and expanding transparency and citizen oversight? 4.1 What have been the key challenges faced by CSOs to oversight of the use of public funds? 4.2 To what extent is CSOs’ engagement likely to continue and be scaled up after GIP ends? 4.3 To what extent has the assistance provided by GIP contributed to improving the organizational capacities of beneficiary CSOs? 5. How do recipients and decision-makers of the assisted CSOs, municipalities, GOES institutions, and independent regulatory entities (Government Ethics Tribunal [TEG], PGR, and CCR) value the technical assistance provided? 2 5.1 What are the hindrances and barriers that recipients face in applying the new knowledge gained through GIP? 6. What interventions promoted gender equality and social inclusion during the implementation of GIP? What factors enabled or inhibited the integration of these topics? METHODOLOGY The evaluation team (ET) employed a mixed-methods data collection design that included document review, key informant interviews (KIIs), online surveys (OLSs), focus group discussions (FGDs), and one co-creation workshop with CSO representatives to discuss and validate the recommendations. The ET reviewed 96 documents, including GIP reports, training materials, and studies commissioned, as well as other relevant documents. It conducted 37 semi-structured individual KIIs and eight KIIs with small groups, covering five key stakeholder groups, four FGDs, and five OLSs. MAIN FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS A summary of the main findings, conclusions, and recommendations from the GIP Final Performance Evaluation are presented in Table A1 below. 3 Table A1: Summary of Findings, Conclusions, and Recommendations Findings Conclusions EQ1: To what extent has GIP contributed to strengthening communicate, or bring attention to public resource use? oversight institutions’ capacity to monitor, disclose, highlight, discipline, investigate, Government’s Ethical Tribunal: Interviewees agreed that the different trainings provided by the project helped to improve investigation techniques for the identification of suitable evidence within an administrative process. A Directory of Ethics Commissions was created to support TEG’s monitoring role regarding the installation and training of the Public Ethics Commissions. One of TEG’s interviewed representatives and three of the six participants in the TEG focus group pointed out that a challenge to increasing institutional effectiveness is related to the collegial nature of the Tribunal. GIP contributions have been focused on strengthening procedures, drafting manuals, policies, and technological tools. This is supported by the progress of the project’s output indicators and the KIIs and FGD conducted by the ET. There are no indicators measuring the extent to which the Tribunal has improved its investigative and sentencing effectiveness. This does not necessarily mean that there are no outcome effects, but that they have not been systematically measured. Institute for Access to Public Information (IAIP): GIP staff reported that IAIP’s institutional portal, transparency portal, and case management system, among others, were developed and enhanced. At least four CSO members interviewed agreed that the push that IAIP made in its first years of operation was significant in these areas. Changes in the leadership of IAIP meant a marked setback in the institute’s performance at the central level over the last two years. There has been a firm commitment and advances made to strengthen electronic platforms, both related to the publication of public information and for the training of public officials at the national level. The main challenge for sustainability of the technical advances made is related to the national political context which limits the institutional commitment to comply with its legal mandate and is not conducive for officials and institutions to operate without external pressure and restrictions. Public Defender’s Office: The PGR official interviewed stated that the mechanisms implemented by GIP have helped to improve the internal administrative processes. The improvement of an information technology (IT) system for the management of the Family Unit cases and the creation of a web portal for the publication of open data, were mentioned. According to the PGR official, the Institutional Integrity Model (IIM) has been a fundamental contribution to the PGR’s understanding that public integrity must be part of the work culture and the basis for quality management services. The PGR is an example of the IIM’s impact on the organizational structure of a public institution. It also exemplifies how the IIM has served as a conceptual framework to give coherence and direction to broader institutional initiatives. Knowing that changes in organizational culture are medium-term processes, the changes made in the PGR's organizational structure could be an excellent basis for promoting a second generation of efforts that emphasize outcome measurements related to changes perceived from the perspective of the end users (citizens). Court of Accounts of the Republic: A Citizen Observatory of the CCR (OCCCR) was established. The ET was able to corroborate through GIP reports and supporting documents that a consortium of There is little evidence of the continuity of the OCCCR. The lack of institutional commitment is attributable to the national political context 4 Findings Conclusions three CSOs produced four studies related to the profiles of the judges of accounts, and rate of judgments in relation to the number of audit findings. A law reform was presented to the Legislative to mandate the CCR to undertake “concurrent audits.” The CCR official interviewed by the ET reported that there were no major advances in relation to the recommendations of the studies that the OCCCR produced. of confrontation oversight issues. between the government and CSOs over citizen Attorney General’s Office (FGR): The ET was able to corroborate, through the review of GIP reports, that improvements were made to the online systems for complaints management, public information requests, the web portal, and the FGR Public Information index. Improvements were made to the staff performance evaluation process, the application of a Code of Ethics, and reforms to the Organic Law of the FGR to give a legal mandate to these elements. Given the restrictions to access FGR and CSJ representatives during this evaluation, the ET was not able to obtain internal opinions. The team did not record any opinions from other GTIAC members that would have highlighted any points about the FGR and CSJ role in the coordination mechanism. Supreme Court of Justice (CSJ): The ET was able to corroborate, through the review of GIP reports, that the CSJ, as a member institution of the GTIAC, participated in elaborating the Protocol for the Handling of Corruption Complaints and Whistleblower Protection and was in charge of one of the training modules for GTIAC members. EQ2: What have been the main accomplishments and challenges of the anti-corruption key actors/champions, coordination of the participant institutions, and sustainability? Interinstitutional Working Group (GTIAC) in terms of The GIP played a key supporting role in the GTIAC’s creation. As documented by meeting minutes attached to the implementing partner’s (IP) reports, the signing of the Memorandum of Understanding (MOU) formalizing the creation of the mechanism, was preceded by a high-level dialogue between the leadership of the six signatory institutions and IP representatives. A protocol for interinstitutional communications and coordination was signed and a joint training program was implemented. The GTIAC’s work entered a phase of stagnation because of the sudden leadership changes at the CSJ and FGR and as a result of IAIP’s weakening role. The ET conducted KIIs with representatives of two of the six institutions participating in the coordination mechanism. The interviewees stated that the joint training program opened the opportunity for a peer-to￾peer dialogue and interchange of technical criteria. The TEG’s The formation of the GTIAC in itself represented an important first step towards an effective national anti-corruption policy. It was made possible by the high level of dialogue and trust built between Tetra Tech and the leadership and technical staff of the institutions participating in the coordination mechanism. The continuity and effectiveness of the GTIAC’s efforts are highly dependent on the personal good will of its constituent members. Continued leadership and technical resources are needed to sustain the operation of the mechanism. In practice, Tetra Tech played the role of technical secretariat to the GTIAC and provided the support needed for its operation. 5 Findings Conclusions President played a key role within GTIAC and is committed to giving continuity to the joint training program. EQ3: To what extent has the activity contributed to better response to access to public information requirements, implemented more expansive open government and accountability practices, and assisted compliance of ethics standards in selected municipalities, GOES Agencies (MOP, MINSAL, MINEDUCYT, CCR, and PGR)? Municipalities: Municipal Access to Public Information Officers (PIOs) and Records Management and Archiving Officers (ARMOs) have been professionalized through the GIP’s training. Of the PIOs and ARMOs interviewed, 80 percent and 100 percent respectively, stated that the trainings have empowered them to comply with citizen information requests. The volume of information requests to municipalities does not seem to have increased significantly in recent years. The GIP has succeeded in strengthening the Access to Public Information Unit (UAIP) and Archive and Records Management Unit (UGDA) through the professionalization of their officials and a certain degree of institutionalization of the procedures associated with the Access to Public Information Law (LAIP). The strengthening of the UAIP and UGDA does not seem to have increased the volume of information requests received by the municipalities. The effort faces sustainability risks due to the tendency to centralize the local development policies, including the reduction in the allocation of public funds from the central level to the municipalities Ministry of Public Works: The previous administration (2015-2019) adopted the IIM to channel its anti-corruption efforts and drafted an institutional corruption risk map, the procedures to manage asset disclosure reports, and the system to manage complaints and the institutional Code of Ethics. The interviewees emphasized that the main contribution of the IIM was to bring a conceptual framework and greater coherence to the measures that the MOP authorities wanted to promote in terms of transparency and the fight against corruption. With the new administration, the effort remains at a formal level but there is no indication of implementation and progress. The MOP experience is another example of how the IIM has served to give coherence and structure to the initiatives on institutional integrity. Ministry of Education, Science, and Technology and Ministry of Health: A clear achievement of the GIP in terms of the strengthening of MINEDUCYT and MINSAL has been the design of the online system for managing citizen complaints. They stated that the ministries have benefited from having carried out a process mapping and emphasized that the system will allow for more agile case management. They indicated that the system is ready to be implemented but has not yet gone live. The management of the system The online platform will be a significant aid for a more systematic and more agile management of citizen complaints in MINEDUCYT/MINSAL given the manual processes and the varied number of citizen attention entry points and units that intervene in each ministry. Centralized management from CAPRES may represent an obstacle to the operational efficiency of the system by setting the way in which complaints will be processed and the results communicated. 6 Findings Conclusions and the dissemination Office (CAPRES). of results will be centralized at the President’s EQ4: To what extent has the assistance provided to CSOs contributed to institutionalizing and expanding transparency and citizen oversight? With GIP support, academic institutions have enhanced their role in the promotion of a nationwide anti-corruption agenda. Two leading universities created a dialogue platform and facilitated the drafting of a National anti-corruption Agreement which was signed by 80 CSO representatives and a public campaign with the participation of young men and women was launched to disseminate its content and to promote signatures of support from the general public. CSO representative interviews recorded a unanimous positive assessment of the dialogue platform. An anti-corruption Specialized Monitoring Center was created within Francisco Gavidia University (UFG) to follow up on implementation of the anti-corruption policies and regulations included in the Agreement. Advocacy organizations have developed new skills and advocacy strategies with development of apps and the promotion of open data. “Hackathons contests” were used to promote youth participation and awareness of transparency and accountability issues. CSO key informants also reported the development of new competencies for monitoring political parties’ finances and the expertise to promote strategic litigation processes. The GIP’s partnership with the Social Initiative for Democracy (ISD) was a key factor to reach vulnerable groups with awareness campaigns through community radio broadcasts and training programs. The ET registered a positive assessment from the beneficiaries regarding the content of the courses. GIP also met requests to adapt the methodology for the online format. The ET found that the cost and accessibility to digital connection are obstacles that limit the participation of beneficiaries in rural areas. New specialized knowledge, stronger skills, and opportunities for dialogue and forging consensus are clearly identifiable contributions that the GIP has made to the strengthening of the CSOs. As a result, they have expanded their areas of influence (i.e., strategic litigation, financial oversight of political parties, promotion of open data policies and practices, adoption of new digital tools). Strengthened technical competencies and skills to forge strategic alliances have been institutionalized, as shown by the sustained publication of reports, the permanence of the networks built (as expressed by the joint communications issued), and their continuous engagement in working together on a common agenda. The GIP’s partnership with the ISD allowed for the effective inclusion of vulnerable populations through their own organizations. Even though the opinions gathered during the FGDs and OLS are positive in terms of the value assigned to the knowledge acquired, the need to adapt the training to the e-learning format and the cost and accessibility to digital connection should be taken into consideration in future training programs. EQ4.1: What have been the key challenges faced by CSOs to oversight the use of public funds? 7 Findings Conclusions The key obstacles identified by the CSO representatives interviewed are: a. The limited culture of questioning political power as well as citizens’ poor perception of the benefits that their engagement with transparency and accountability activities could bring to their daily life. b. The weakening of the regulatory institutions responsible for compliance with the transparency and accountability legal framework (specifically, IAIP, CCR, and FGR). c. The rising political and security risks they are facing. They highlighted the language used by government’s representatives to dismiss their work, the harassment in the official media, and the threats of politically motivated judicial persecution. In a context of long-term cultural barriers to the questioning of public authorities, less responsive public institutions can jeopardize the progress made regarding citizens’ awareness and willingness to engage with transparency and accountability issues. With the current deterioration of the political context, the main challenges faced by CSOs are the legal threats and the blocking of much-needed financial support from international donors. EQ4.2: To what extent is CSOs’ engagement likely to continue and be scaled up after the GIP ends? All CSO key informants expressed in general terms their commitment to continue to develop their respective areas of work. A solid ground for the continuity of their work can be found in the institutionalization of the newly developed expertise and in the consolidation of strategic alliances built around a common agenda of transparency, accountability, and anti-corruption objectives. The CSOs have indicated a strong commitment to continue their efforts. In assessing the likelihood of a scaled-up engagement, two limiting factors must be considered: CSOs’ dependency on financial support from international donors and the increasingly hostile political environment in which they operate. EQ4.3: To what extent has the assistance provided by GIP contributed to improving the organizational capacities of beneficiary CSOs? As documented in the IP’s reports, one workshop was held with the participation of GIP’s CSO partners to analyze the current context in which they operate and identify the challenges in advancing the transparency and anti-corruption agenda. There is no indication that other areas of organizational strategic planning have been addressed. In one specific case, support was given for the organization’s legal registration, as stated by its director during the interview with the ET. Strengthening CSOs’ organizational capacities for strategic planning and monitoring is a pending task. EQ5: How do recipients and decision-makers of the assisted (TEG, PGR, and CCR) value the technical assistance provided? CSOs, municipalities, GOES institutions, and independent regulatory entities Municipalities. All PIOs and ARMOs interviewed highly valued the training and technical assistance received. This is confirmed by the municipal PIOs who responded to the online survey: 90 percent of The trainings are well appreciated by the PIOs and ARMOs in the municipalities in terms of learning the essential aspects of their functions and the content of LAIP, especially when they are starting 8 Findings Conclusions them either “agreed” or “strongly agreed” that the topics of the trainings received were relevant, and 87 percent of them “agreed” or “strongly agreed” that training were of good quality. their functions. The PIOs and ARMOs seem to be more confident in applying the new knowledge at the individual level in their day-to-day work. In the future they could benefit from receiving further training on how to manage even greater institutional support. Central Government Agencies: MOP interviewees positively valued the IIM as a guiding model for measuring and structuring their efforts related to the fight against corruption. MINEDUCYT and MINSAL value the online platform that will enable them to manage the citizen complaints in a more efficient way. The PIOs and ARMOs of the central government agencies highly value the content of the trainings. In total, 87 percent of the PIOs of the central government agencies who responded to the online survey either “agreed” or “strongly agreed” that the topics of the trainings received were relevant and of good quality. Further, 83 percent the ARMOs from central government agencies who responded to the online survey either “agreed” or “strongly agreed” that the training topics received were relevant, and 72 percent that it was of good quality. The training and technical assistance provided to central government institutions seem to have helped them to learn more about institutional integrity, as well as about implementation processes and mechanisms. There appears to be no substantive difference between the level of PIO and ARMO’s satisfaction between municipalities and central government agencies. Independent Regulatory Entities (TEG, PGR, CCR): TEG’s legal staff show an intermediate level of satisfaction and point out challenges in the applicability and specialization of the training topics. All the investigators and legal collaborators that participated in the FGD stated that the training received was helpful and that they have acquired new investigation techniques. Most members of the Ethics Commissions positively value the training received. Overall, 87 percent of the Ethics Commission members who responded to the online survey either “agreed” or “strongly agreed” that the topics of the trainings received were relevant and 87 percent that they were of good quality. The PGR representative valued the quality of the training and the potential for cultural change that the IIM model represents and stated that the training methodologies were appropriate and the contents presented met high quality standards. The level of satisfaction with the contents of the training received and with the scope of their applicability is high. There tends to be a little more satisfaction with the contents and formats of the training and a little less satisfaction with the institutional support to apply the new knowledge acquired. Civil Society Organizations. Academic Institutions consider that they have a greater capacity to build networks than before. All (100 percent) of those interviewed considered that the support received for building alliances and working with other CSOs to The organizations feel greater capacity and potential for working in networks and coordinating among themselves. This creates a future basis for fine-tuning and specializing the capacity to influence the public agenda. 9 Findings Conclusions promote diversification and complementarity in capacity-building efforts had been valuable. Advocacy organizations value the knowledge acquired and the technical assistance received as timely, pertinent, and of high added value. All (100 percent) of those interviewed evaluated the Tetra Tech team as highly professional, collaborative, and responsive to the needs of the CSOs. They highlighted the flexibility that Tetra Tech had in adapting to changes in the national situation. Grassroots organizations are satisfied with the competency development approach of the trainings. All (100 percent) of the people participating in the FGDs conducted with vulnerable populations and young women agreed that they now feel more empowered as they have acquired new practical knowledge about their rights and tools to access information. The participants also considered the efforts to adapt the training materials to the culture and needs of their respective groups to be very positive. The organizations recognize that the national situation has been, and possibly will continue to be, highly changeable. This represents a challenge in adapting to changes without losing the scope and effectiveness of interventions in the long term. EQ5.1: What are the hindrances and barriers that recipients face in applying the new knowledge gained through GIP? Municipalities: Municipal PIOs and ARMOs interviewed pointed out that many new municipal council members, and recently appointed unit heads are unaware of the LAIP and its processes. PIOs and ARMOs are now more vulnerable to arbitrary dismissals or transfers if, for example, they should happen to incommode members of municipal councils when complying with sensitive public information requests. After the change of administrations on May 1, 2021, there were massive municipal dismissals, in clear violation of the Municipal Administrative Career Law approved in 2007 and with little possibility for those affected to appeal to an independent judicial system. A common thread of the main barriers identified is the lack of continuity of public policies and priorities from one government administration to the next, as well as the high rotation of middle management and technical staff that affects government institutions. The second thread stems from the risks of political persecution impacting the CSOs working on transparency and anti-corruption issues. Central government agencies: The uneven literacy level regarding the use of online platforms among potential users of the MINEDUCYT/MINSAL system, will require not just a campaign to promote its usage but also a clear step-by-step set of instructions explaining how to access the platform. The lack of a modern civil service career law that includes civil servant performance assessment and protection against unfair dismissal limits any ongoing commitment and willingness on the part of the technical staff to apply the new competences developed with GIP’s support. 10 Findings Conclusions Independent Regulatory Entities. The achievements observed regarding the enhanced use of IT-supported processes and online platforms require an increased allocation of institutional resources in order to cope with technical demands and the maintenance of these systems. The immediate future of the national political context is uncertain and there is great caution on the part of the regulatory institution’s authorities, whose independence is being curtailed. Their commitment and future actions will clearly depend on the evolution of this context. Civil Society Organizations: The main barrier voiced by the CSOs is the increased perception of political risks, resulting from increased harassment and politicized judicial persecution. Such a scenario drastically reduces the possibilities for collaboration and response by public institutions, increases uncertainty about their financial sustainability, and limits their capacity to provide technical assistance to citizens in their demands for access to information. EQ6: What interventions promoted gender equality the integration of these topics? and social inclusion during the implementation of GIP? What factors enabled or inhibited According to the MINEDUCYT and MINSAL representatives interviewed, gender and vulnerable groups data were taken into consideration in the design of the complaints management system. TEG’s President valued the support received from the GIP to disseminate the Tribunal’s gender policy in close coordination as well as the support received for the publication of the government’s Ethics Law in Braille format. Training programs on access to public information rights and social oversight were specifically designed to meet the needs of young women and vulnerable groups. The ISD director highlighted the development of a communication campaign to raise awareness and promote citizen participation in social oversight activities among vulnerable groups. Partnerships were built with the network of community radios and the municipalities. A dialogue platform organized by GIP’s CSOs partners allowed for discussion of the impact of corruption on women’s rights. Working in partnership with the central government institutions, regulatory agencies. and municipalities targeted by the project, GIP has systematically implemented a strategy to promote gender equality and the inclusion of vulnerable groups. Through its alliance with the ISD, and by facilitating the coordination between CSOs with different expertise, the GIP was able to effectively reach the final beneficiaries and their organizations. Despite the technical assistance provided by GIP to public institutions’ staff members on gender equality and social inclusion, and the support given for the development of policies and guidelines, there is no clear evidence of how they have been translated into concrete measures and how they have affected the delivery of public services. 11 Findings Conclusions Enabling factors: GIP’s mapping of key stakeholders and the partnership with ISD (and its partners Independent Monitoring Group of El Salvador [GMIES], Salvadorean National Council of Indigenous Population [CCNIS], Futuro Abierto) played a central role in facilitating the implementation of gender equality and social inclusion interventions. This partnership allowed the inclusion of young women and vulnerable groups through their own organizations and, in so doing, empowered them individually and as organized groups. Limiting factors: As reported during the interviews with MINEDUCYT, MINSAL, and MOP representatives, there is a lack of sensitivity among government staff members regarding gender equality, which is perceived as something that is not relevant to the public servant’s functions. A similar appreciation was made by the municipal PIOs and ARMOs interviewed. A specific limiting factor was raised by one of the government representatives, who referred to a CAPRES communication guideline that restricts the use of gender equality language. Recommendations Government Agencies and Independent Regulatory Institutions ● GIP should maintain an agenda focused on building capacities on the IIM at technical level and maintaining contacts and networks within the TEG, PGR, and CCR, to the extent that institutional relations with them are possible. Cultural changes at the institutional level require long-term effort and maintaining training initiatives for the technical teams of these institutions will allow them to consolidate and sustain the GIP’s achievements. ● In the case of the IAIP and TEG, it is recommended that the training work be carried out with PIOs and members of the Ethics Commissions, who are officials that do not belong to the control institutions but are spread across all ministries, municipalities, and other public institutions in a decentralized manner. ● GIP should consider the introduction of a self-assessment scorecard, like those employed by the IIM, to keep track of the implementation of the specific content of policies, guidelines an action plans on gender equality and inclusion of vulnerable groups that have been designed with the GIP support. ● In terms of outcome-oriented indicators, GIP should keep track of the financial and technical institutional resources specifically allocated for the implementation of policies and guidelines on gender and inclusion of vulnerable groups designed with GIP technical assistance. Civil Society Organizations 12 Findings Conclusions ● ● ● ● ● ● ● GIP needs to develop protection mechanisms and support measures against political persecution, which will be high on the agenda of the CSOs in the next 24 months. CSOs will benefit from receiving training on issues of legal and digital protection. CSOs will benefit from learning about public opinion formation processes, such as survey reading, perception management, semiotic interpretation, social communication tools, and traditional and digital media management, as well as training to enhance the professionalization of their communications teams. To further enhance their capacity to create an informed public opinion, CSOs would benefit from learning about international experiences of interorganizational coordination and investigative journalism as a means of enhancing the effectiveness of their own communication strategies. To improve the scope of their awareness campaigns, and engage with citizens nationwide, CSOs would also benefit from learning about experiences using new social media channels. CSOs would benefit from receiving training based on international lobbying expertise to improve their advocacy strategies, such as stakeholder analysis, risk mapping, and designing of fundraising strategies. The change in the working context for CSOs to one of political persecution will require the young leaders (35-45 years old) to change their paradigms and understand the need to develop a more sophisticated and mature advocacy capacity. The last year of GIP implementation could serve to promote discussion with young leaders and prepare the groundwork for a new cycle and type of work for the medium term. CSOs will need to improve their capacity to work collaboratively with new stakeholders, in particular with political actors, the private sector and the international community. Organizations will continue to benefit from having structured spaces for the exchange of views and reflection on the potential for collaboration. As a cross cutting recommendation, improvements on the monitoring and evaluation (M&E) system of the IP and of its partners, can be introduced. Currently, GIP’s M&E system records data for 29 indicators with measurable targets assigned to them (four additional standard indicators have also been recorded). The introduction of outcome-oriented indicators would enable prioritization of the most effective interventions and enhance sustainability. Specific examples are listed in the main text of this section. 1 1. EVALUATION BACKGROUND, PURPOSE AND AUDIENCE 1.1 EVALUATION BACKGROUND AND ACTIVITY OVERVIEW This report presents the primary findings, conclusions, and recommendations from the Final Performance Evaluation of the United States Agency for International Development (USAID) Government Integrity Project (GIP), and answers to the requirements of the Statement of Work (SOW) prepared by USAID/El Salvador (Annex 1, Statement of Work revised version 09.29.21). The evaluation covers the GIP’s implementing period between March 3, 2016, and June 30, 2021. The GIP is being implemented by Tetra Tech DPK under the Award AID-519-C-16-00001 (March 3, 2016-March 2, 2021) with an allocated funding of $20,291,708.00. It has covered two implementing phases, a three-year base period with and an optional two-year period, which has been extended to March 2023 with a $4.5M TEC increase. Modifications to the original Contract Line Number (CLIN) were introduced to increase the number of institutions given technical assistance and develop new activities associated to unanticipated opportunities to enhance the GIP’s achievements (CLIN 002, Task Orders No. 1 and No. 2). The GIP was designed to enhance the institutionalization and implementation of transparency regulations established by the Access to Public Information Law (LAIP).1 The underlying assumption was that the strengthened legal framework and advances made through the newly established independent oversight institution—the Institute of Access to Public Information (IAIP)—will provide incentives for citizens to demand public information access and to participate in monitoring activities over the use of public funds. The GIP was structured around two objectives and six components designed to a) strengthen the capacity of government institutions and municipalities to effectively comply with transparency and accountability regulations, and b) increase citizen capacity to oversee the use of public funds (see Table 1). It targets oversight institutions, government agencies, and selected municipalities. The GIP also partners with civil society organizations (CSOs) in performing advocacy and oversight roles and in promoting universal application of transparency tools by a wide spectrum of population groups, especially those considered vulnerable. These efforts were designed to contribute to institutionalizing and expanding transparency and citizen oversight, thereby increasing accountability, and reducing opportunities for corruption. 1 Asamblea Legislativa, Ley de Acceso a la Información Pública, Decreto Legislativo No.534, San Salvador 8 de abril 2011 2 Table 1: GIP’s Objectives, Components, and Expected Results (ERs) Objectives Components ERs ● 1. Strengthened Government of El Salvador (GOES) transparency, accountability, and professional standards2 1.A Capacities of regulatory agencies that oversee transparency and ethics regulations increased ● ● ● The role of the IAIP as the key institution of the access to public information law consolidated. The links between IAIP and civil society strengthened. Government Ethics Tribunal (TEG) investigation capacity improved, and its role as the implementing institution of the ethics law consolidated. ● At least three transparency and accountability mechanisms implemented by the TEG—within the base period—to allow citizens to monitor the implementation of the government ethics law. 1.B Central government’s transparency increased ● ● Response to access to information law requests improved. Transparency, accountability, and/or anti￾corruption mechanisms implemented. 1.C Professionalization of central GOES public servants enhanced ● ● Civil service reform promoted. Civil service best practices (including merit￾based recruitment, evaluation, and promotion) advanced by GOES. ● Procedures to receive and submit complaints of violations to the Ethics Law to the TEG improved in at least three government institutions during the first three years of implementation. ● Procedure to submit, receive, and respond to complaints of violations to the Ethics Law improved in at least one additional government institution yearly (during the option period). 1.D Transparency and professionalism at the municipal level ● ● Implemented transparency and accountability mechanisms in selected municipalities. Improved response to access to information 2 Addresses the need for key oversight institutions to effectively promote implementation of transparency and professionalism regulations and select GOES institutions at the national and municipal levels to better respond to access to public information requirements, as well as to implement more expansive open government and accountability practices. Under this objective, the GIP also assists the GOES in complying with ethics standards and professionalizing the public service workforce through the promotion of civil service reform and optimization of select human resource management processes. 3 Objectives Components ERs increased requests and complaints of violations Ethics Law in selected municipalities. to the 2. Citizen oversight, participation, and awareness of anti￾corruption reforms increased3 2.A Civil society’s capacities to monitor public resources and combat corruption strengthened ● ● Contribute to a more accountable government through the development of CSO-led oversight/advocacy actions Support GOES-CSOs joint actions to combat corruption through at least three GOES￾CSO dialogue platforms 2.B Transparency rights ▪ Increased awareness of transparency rights and in vulnerable citizen participation and in targeted vulnerable populations expanded ▪ populations through the development of transparency outreach initiatives. Increased use of transparency mechanisms by targeted vulnerable populations through the implementation of oversight actions. Since its inception in 2016, the GIP worked in close partnership with public institutions, municipalities, and CSOs in a favorable environment created by a political consensus in support of the transparency and anti-corruption agenda. However, increasing obstacles and setbacks have been experienced by GIP’s stakeholders since the political environment started to deteriorate in May 2021, when the newly elected Legislative Assembly, dominated by the President’s New Ideas party, removed all five magistrates of the Supreme Court of Justice (CSJ) and the Attorney General, a move that was denounced as illegal by several CSOs. In the following weeks, the Attorney General renounced the agreement signed with the Organization of American States (OAS) that created the International Commission Against Impunity in El Salvador (CICIES), marking a clear departure from previous institutional policies. 1.2 EVALUATION PURPOSE AND AUDIENCE The purposes of the present evaluation are to a) inform USAID/El Salvador regarding GIP’s achievements and challenges, b) determine whether the intended outcomes were achieved, c) provide empirical evidence regarding management issues, and d) document learning and continuous improvement. The evaluation’s findings will be used as an input for making changes to the GIP during its remaining implementation period and for the design of future activities. The main audience for the evaluation is USAID/El Salvador, specifically the Mission Management, the Democratic Governance and Security Office (DGS), the Regional Office of Acquisition and Assistance, the Regional Program Office (RPO), and Tetra Tech DPK. Secondary audiences within USAID include the Bureau for Latin America and the Caribbean; the Bureau for Policy, Planning, and Learning (PPL); and the Bureau for Development, Democracy, and Innovation (DDI). The Mission intends to disseminate the lessons learned and best practices identified in this evaluation to ongoing and potential stakeholders for the GIP’s future actions, and the evaluators have sought 3 Focuses on the demand side by improving CSOs’ capacities to develop and implement monitoring and oversight tools, increase citizen participation, and expand awareness on transparency regulations among the general population, with emphasis in rural areas and among people with disabilities. 4 to ensure that all recommendations made are realistic, applicable, and useful to USAID, implementing partners (IPs), and the beneficiaries as deemed relevant. 2. EVALUATION QUESTIONS USAID/El Salvador identified the following six evaluation questions (EQs), in order of priority: 1. To what extent has GIP contributed to strengthening oversight institutions’ capacity to monitor, disclose, highlight, discipline, investigate, communicate, or bring attention to public resource use? 2. What have been the main accomplishments and challenges of the anti-corruption Interinstitutional Working Group (GTIAC) in terms of key actors/champions, coordination of the participant institutions, and sustainability? 3. To what extent has the activity contributed to better respond to access to public information requirements, implemented more expansive open government and accountability practices, and assisted compliance of ethics standards in: ● Selected municipalities, and ● GOES agencies (Ministry of Public Works [MOP], Ministry of Health [MINSAL], Ministry of Education, Science, and Technology [MINEDUCYT], Court of Accounts [CCR], and Public Defender’s Office [PGR])? 4. To what extent has the assistance provided to CSOs contributed to institutionalizing and expanding transparency and citizen oversight? 4.1 What have been the key challenges faced by CSOs to oversight the use of public funds? 4.2 To what extent is CSOs’ engagement likely to continue and be scaled up after the GIP ends? 4.3 To what extent has the assistance provided by GIP contributed to improving the organizational capacities of beneficiary CSOs? 5. How do recipients and decision-makers of the assisted CSOs, municipalities, GOES institutions, and independent regulatory entities (TEG, PGR, and CCR) value the technical assistance provided? 5.1 What are the hindrances and barriers that recipients face in applying the new knowledge gained through GIP? 6. What interventions promoted gender equality and social inclusion during the implementation of GIP? What factors enabled or inhibited the integration of these topics? The EQs were identified by USAID/El Salvador in line with the USAID Democracy, Human Rights, and Governance (DRG) Learning Agenda and, specifically, with the Country Development Cooperation Strategy (CDCS) 2020-2025 Learning Plan. 5 3. METHODOLOGY 3.1 GENERAL EVALUATION APPROACH This evaluation falls into the category of a performance evaluation and analyzes the contributions made by the activity to the GIP’s intended outcomes and implementation strategies and processes. The evaluation employed a non-experimental mixed-methods approach that included a combination of document review with qualitative and quantitative data collection methods to ensure the relevance and validity of the fact-finding process and recommendations: key informant interviews (KIIs), focus group discussions (FGDs), online surveys (OLSs), direct observation and site visits were conducted. In line with a participatory approach, GIP also planned co-creation workshops to identify possible recommendations. The evaluation was conducted in close dialogue with Tetra Tech DPK, GIP’s IP. 3.2 DATA COLLECTION METHODS 3.2.1 Document Review The evaluation team (ET) reviewed all the documents pertaining to the activity provided by Tetra Tech DPK (technical papers, progress reports, annual reports, commissioned systematizations and studies, training materials, and monitoring, evaluation, and learning [MEL] plans), as well as documents found by the team during its own literature search. A bibliography of documents reviewed for the evaluation is presented in Annex 2. 3.2.2 Key Informant Interviews The ET conducted 37 semi-structured KIIs with individuals and eight KIIs with small groups (two people), which covered all GIP stakeholder groups. The key informants interviewed also included Tetra Tech staff—the GIP’s Director, four Area Coordinators, anti-corruption Specialist, and Monitoring and Evaluation (M&E) Officer—and the current GIP USAID Contracting Officer’s Representative (COR) as well as the two previous ones. Of the total number of people interviewed, 43.3 percent were women and 56.7 percent were men. Its distribution in terms of GIP’s stakeholder category presents as follows: 22.6 percent of informants were central government officials; 26.4 percent municipal officials; 32.1 percent CSO representatives, and 18.9 percent Tetra Tech and USAID managers. The list of key informants interviewed is presented in Annex 3. 3.2.3 Online Surveys The ET administered a total of five OLSs to identify the extent to which respondents’ expectations were met regarding the GIPs training programs, as well as how they perceived the resulting changes in the performance of their organizations. Four of the OLSs targeted Public Information Officers (PIOs), Data and Archive Managers, Ethics Commissioners, and Public Defenders’ Office technical staff members and covered the following topics: a) relevance of the training and technical tools; b) quality and timeliness; c) applicability and contribution to improve the fulfillment of the institutional mandate; d) scope and challenges in the adoption and use of new knowledge and instruments; e) institutional resources allocated for their adoption and other questions related to sustainability; and f) inclusion of gender and 6 social inclusion perspectives. They consisted of close-ended questions using yes/no, categorical, or scaled responses. One OLS aimed at young women combined close-ended and scaled responses with a limited number of open-ended questions. The OLSs’ distribution of respondents, according to the targeted groups, was as follows: central government officials 32 percent, municipal officials 60 percent, and young women 9 percent. Distribution according to sex was: 60 percent women and 40 percent men. 3.2.4 Focus Group Discussions The ET conducted four FGDs to gather information about GIP’s implementing strategies regarding the inclusion of women and vulnerable populations, and to cover training-related issues with the technical staff of the TEG. Two of the three FGDs targeting vulnerable groups were held in person and one was conducted online. The FGD with the TEG official was held online. Table 2: FGDs by Organization and Number of Participants FGD Organization Number of Participants Indigenous Population Social Initiative for Democracy (ISD) – Salvadorian National Council of Indigenous Population (CCNIS) 8 (38% women, 62% men) Young Rural Women ISD – Open Future Association (Asociación Futuro Abierto) 5 (100% women) Lesbian, Gay, Bisexual, Transgender, Queer ISD – Independent Monitoring Group of El 3 Plus (LGBTQ+) Salvador (GMIES) (100% women) Community Technical staff at the Investigation Unit TEG 8 (50% women, 50% men) Total 24 (63% women, 37% men) 3.2.5 Direct Observation and Site Visits The ET conducted direct observation and KIIs during site visits to six of the municipalities selected. Of the remaining two, one had to be canceled (Santa Ana) due to the lack of response to the requests made and, in the second case (San Martin), the team held KIIs with municipal officials online due to security concerns. The municipalities visited are presented in the Table 3. Table 3: Site Visits Municipality Department Armenia Department of Sonsonate Ilobasco Department of Cabañas Mejicanos Department San Salvador La Libertad Department of La Libertad Santa Tecla Department of La Libertad San Pedro Perulapán Department of Cuscatlán 7 3.2.6 Co-Creation Workshops The ET designed two co-creation workshops to receive input from stakeholders to draft the evaluation’s recommendations, with emphasis on how the Mission can improve future activities on transparency and anti-corruption. The team held one online workshop with the participation of CSO representatives—Acción Ciudadana, the Legal Aid and Anticorrupción Office (ALAC), the National Foundation for Development (FUNDE), and the Salvadoran Foundation for the Economic and Social Development (FUSADES)—and Tetra Tech personnel. The workshop planned to be held with the participation of public institution representatives was canceled due to the lack of confirmed participants. 3.3 SAMPLING METHODS 3.3.1 Key Informant Interviews For the KIIs, the ET used a purposive, non-probability, sampling method. Participants were selected based on the evaluation objectives and chosen from a list of key informants provided by the IP who, due to their position or participation in the GIP, had first-hand knowledge relevant to the EQs. The ET identified six additional KIIs with informed observers. 3.3.2 Online Surveys The ET sent invitations to participate to the entire database provided by Tetra Tech by selected stakeholders categories, except for those who had not registered an e-mail address. 3.3.3 Focus Group Discussions FGDs targeted individuals with shared experience in interacting with GIP’s work. To gather attitudes, feelings, beliefs, and reactions, priority was given to three specific groups: young rural women, indigenous populations and members of the LGBTQI+ community. The participants were identified with the support of Tetra Tech and ISD, an umbrella organization that brings together grassroots associations. The ET conducted an additional FGD with members of the legal team in charge of investigating the cases re-referred to the TEG, extending an invitation to participate to the legal staff at the Investigative Unit. 3.3.4 Site Visits The ET selected eight municipalities for site visits that were identified with the support of Tetra Tech, using the following sampling criteria: a) an equal number of municipalities assisted during the GIPs two implementing stages (four from the base period and four from the extension period); b) an equal number of municipalities within each stage, where the outcomes could be considered successful and municipalities where the outcomes were limited; and c) municipalities where Citizen Oversight Committees were established. 3.4 GENDER AND SOCIAL INCLUSION ANALYSIS The evaluation analyzed the measures and strategies adopted by the IP to promote gender equality and the inclusion of vulnerable groups across different GIP activities. The ET analyzed the IP’s reports to determine participation trends, including disaggregation by sex, age, and geography across different beneficiary groups. The team also explored data trends through KIIs, FGDs and OLSs by asking the participants to answer a set of specific questions regarding their 8 gender. The team designed three FGDs to gather information from vulnerable groups and administered one OLS specifically to a group of young women. 3.5 DATA ANALYSIS METHODOLOGY The ET conducted qualitative analysis of KIIs, FGDs, technical documents, and IP reports. It also conducted quantitative data analysis of online survey results and IP performance data. The data analysis used the following qualitative approaches: a before-and-after perspective, applicability of the new knowledge, and a gender/inclusion analysis approach to draw conclusions on the implications of the results, related to inclusion variables for target groups, for each EQ. 3.6 METHODOLOGICAL LIMITATIONS As foreseen in the evaluation plan, the ET faced limitations due to the high rate of personnel rotation linked to changes in national and local government administrations. With the inauguration of a new government administration in June 2019, the leadership of three government agencies assisted by GIP (the MOP, MINEDUCYT, and MINSAL) changed. Twenty￾nine (29) out of the 30 targeted municipalities changed administration on May 1, 2021. As a proxy indicator of the personnel rotation, the ET found that 21 percent of the e-mails were inactive, with the highest rate of undelivered e-mails (25 percent) corresponding to Ethics Commissioners. Restrictions on the direct communication with officials at the Attorney General’s Office (FGR), CSJ, and IAIP limited the ET’s ability to gather information regarding GIP’s activities in support of these institutions. As a result, the analysis relied on the review of documents available and on verifying the functioning of the online platforms open to the public. For the IAIP, it was possible to gather additional information from KIIs and OLSs addressed to PIOs in government institutions and municipalities to ensure the validity of the findings and conclusions. The ET encountered an additional limitation: the relatively low response rate (14 percent) to the OLSs, which registered 112 responses out of 791 active e-mails. The highest response rates corresponded to the surveys aimed at PIOs and Archive and Record Management Officers (ARMOs), with 19 percent and 21 percent respectively. The survey aimed at the technical staff of the PGR registered the lowest response rate with 3 percent. The time lapsed, and a lack of communications between the project and former participants, reduced respondents’ interest in responding to a cold request for the survey. To counter the anticipated low response rate, the ET distributed OLSs to the complete list of e-mail addresses for each target group of the online surveys. The ET also encountered a certain degree of reluctance to participate in the interviews that could be associated with the political context. Four requests for interviews were unsuccessful (11 percent of the total), two of which explicitly cited concerns related to the political environment. 9 4. FINDINGS & CONCLUSIONS 4.1 EQ1: TO WHAT EXTENT HAS GIP CONTRIBUTED TO STRENGTHENING OVERSIGHT INSTITUTIONS’ CAPACITY TO MONITOR, DISCLOSE, HIGHLIGHT, DISCIPLINE, INVESTIGATE, COMMUNICATE, OR BRING ATTENTION TO PUBLIC RESOURCE USE? 4.1.1 Government’s Ethics Tribunal Findings GIP has strengthened the TEG in its capacity to investigate and manage administrative sanctioning processes, which is one of the main legal mandates of the Tribunal's Plenary.4 The TEG’s representatives interviewed agreed that the different trainings provided by the project helped to improve investigation techniques, such as the identification of suitable evidence within an administrative process. They mentioned that many of the investigators have a background in criminal law issues and that they benefited from learning about the particularities of administrative processes. The FGD with TEG’s investigators offered a more nuanced assessment on this point, expressing an intermediate level of satisfaction with the training program and pointing out the need for follow-up sessions to develop an even more specialized knowledge. TEG’s officials interviewed also agreed that the implementation of the “Manual on Administrative Sanctioning Processes” that GIP developed has helped to organize and structure the Tribunal's investigation and sanctioning processes. They also welcomed the creation of the case management system, which makes it possible to analyze the different stages of the processes, identify bottlenecks, strengths, and weaknesses to provide a better service to the public. However, this appraisal was not unanimously shared by the FGD participants. The ET had no access to internal records and was not able to measure the improvements. GIP has strengthened the TEG in its function of monitoring the installation and training of the Public Ethics Commissions. According to one interviewee, the operation of the “Directory of Ethics Commissions” allows the TEG to have better control over the established Commissions and to monitor which government institutions have yet to install theirs. It also will help the Tribunal to keep better track of the Commissions members’ training, which is one of the main legal mandates of the Tribunal.5 The main challenges in translating the new investigative capacities into a more effective sentencing process are related to the collegial nature of the Tribunal's sentencing body and the national political context. One of the individuals interviewed, and three of the six FGD participants, pointed out that the cases to be investigated are delegated to the legal teams by the Tribunal’s plenary but that the deliberation and final decision remain a legal attribution of the plenary, which is a collegial body and whose members make their own assessments. Such assessments are often influenced by the heated national political environment surrounding cases of corruption and the misappropriation of public funds. 4 Asamblea Legislativa de El Salvador. 2011. Ley de Ética Gubernamental, Decreto Legislativo No. 873. San Salvador 13 de octubre 2011. Governmental Ethics Law, article 20, section 1. 5 Ibid. Government Ethics Law, Article 19, section b and Article 20, section d. 10 4.1.2 Government’s Ethics Tribunal Conclusions ● GIP’s contributions have been focused on strengthening procedures, developing manuals, developing policies, and providing technological tools. ● There are no indicators measuring to what extent the Tribunal has improved its effectiveness, nor how the capacity building is translating into improvements in citizens’ perception of TEG’s performance. During the interviews conducted by the ET, it was also not possible to identify clear answers in this regard. This does not necessarily mean that there are no outcome effects, but that they have not been systematically measured (i.e., percentage of cases successfully investigated applying the new techniques and presented to the Tribunal for sentencing). ● The main challenge for sustainability is related to the national political context, which is not conducive to independent action by officials and institutions. 4.1.3 Institute for Access of Public Information Findings GIP main contributions to the strengthening of the IAIP are linked to the technical support provided for the development of: a technological platform to train IAIP’s counterparts in public institutions, a platform to disseminate public information, and a system to manage the appeal requests in cases of denial of information. The project’s staff explained that IAIP’s institutional portal, transparency portal, and case management system, among others were developed and enhanced to respond to requests for public information in a timelier manner, with greater transparency and ease for citizens. At least four CSO members interviewed agreed that the push that IAIP made the first years of the project was significant in these areas. There is a consensus that there is a marked setback in the IAIP's performance at the central level in the last two years. Of the CSOs representatives interviewed,100 percent agreed that the number of resolutions of cases unfavorable to citizens has decreased, as well as the proportion of public information that is classified as “classified.” They also agreed that the amount of public information published by IAIP and public institutions has decreased. The interviewees agreed that a set of reforms to LAIP presented by the Minister of the Interior to the Legislative Assembly in June 2021 is equivalent in practice to the repeal of the Law.6 By February 1st, 2022 he Legislative Assembly has not approved the reforms, but it represents a latent threat to the work of the IAIP if they approve them. Municipalities are receiving less guidance and support from IAIP since the recent change of authorities. Of the municipal officials interviewed, 100 percent stated that their UAIPs received less support from IAIP’s staff in 2020 and 2021 and that the training received on how to handle and publish public information have also been significantly reduced. In four of the seven municipalities visited, the officials interviewed noted that they have not received IAIP’s annual evaluation report on the information published by each municipality despite their requests on this respect. The interviewees mentioned that this instrument, applied since 2015, enables them to know their strengths and the aspects that need to be improved to respond more 6 Fusades (2021), Boletín No 148. 6 razones por las cuales las propuestas de reformas a la LAIP equivalen a su derogatoria 11 effectively to citizen requests for information, such as construction project contracts and city council agreements. It should be noted that it was not possible to obtain opinions from the Institute’s officials due to the restrictions on contacting them during this evaluation. 4.1.4 Institute for Access to Public Information Conclusions ● There has been a firm commitment and advances made to strengthen electronic platforms, related to both the publication of public information and training public officials at the national level. ● The main challenge for sustainability relates to the national political context, which is not conducive for officials and institutions to operate without external pressure. 4.1.5 Public Defender’s Office Findings PGR modernized elements of its quality management system. The interviewee affirmed that several GIP-implemented mechanisms have helped improve the internal management of transparency processes, as well as the services provided by the institution. Some of the main efforts highlighted included the: elaboration of the risk map for the hiring of personnel and public procurement, improvement of computer systems for the management of the Family Unit cases, and web portal for the publication of open data. The key informant stated that the Institutional Integrity Model’s (IIM) fundamental contribution to the PGR was the understanding that ethics must be part of the work culture as a basis for quality management of services. PGR made changes in its organizational structure to promote institutional integrity. The interviewee noted that the PGR restructured a system of new units to address ethics, transparency, and corruption control in a comprehensive manner and that a new Integrity Department (“Dirección de Integridad”) was created to coordinate the new measures implemented. The key informant also highlighted GIP’s support for the drafting of the new Organic Law of the PGR, which includes organizational restructuring as well as mechanisms and administrative sanctions for unethical behavior, the design of a code of ethics, and parameters for personnel selection based on merit, among others. The new law was approved by the Legislative Assembly in March 2021 and became effective in September 2021. The challenge remains as to how to translate the new procedures and organizational structure improvements into changes perceived by the citizens during the provision of services. The interviewee stated that most progress has been made in changes to the organizational structure and improvements to technological systems, and estimated that the implementation of all new elements of institutional integrity are at an early stage (3-5 percent implementation). As an example, the key informant noted that PGR established a citizen complaints channel, which generates a high volume of information, and highlighted that the pending challenge is to decide what to do with all this information. The ET did not find systematized measurements for how these improvements to the processes and organizational structure are being implemented and turned into efficiencies and quality changes for the citizens. 4.1.6 Public Defender’s Office Conclusions 12 ● The PGR is an example of the IIM’s impact on the organizational structure of a public institution, which can serve as a benchmark for other implementations. It is also an example of how the IIM has served as a conceptual framework that gives coherence and direction to the institutional integrity initiatives. ● Regardless of the national political context, and knowing that changes in organizational culture are medium-term processes, the changes made in the PGR’s organizational structure could be an excellent basis for promoting a second generation of efforts that emphasize outcome measurements, which considers change results perceived from the perspective of the end users (citizens). 4.1.7 Court of Accounts of the Republic Findings (CCR) A collaborative relationship was created between CCR and academic institutions and other CSOs. The ET was able to corroborate in the reports produced by the project that a consortium of three OSCs produced four studies related to the profiles of the judges of accounts, and rate of judgments in relation to the number of audit findings. Based on the studies’ recommendations, the project supported a proposal that required, among other issues, the Legislative Assembly to reform the law governing the Court of Accounts to mandate “concurrent audits.” A Citizen Observatory of the CCR was established (OCCCR) and training for young citizens on how to exercise citizen oversight of CCR was conducted with the support of GIP. The ET found no evidence of the continuity of the OCCCR or indications of any other CCR engagement with citizens’ oversight activities. The CCR official interviewed expressed that there were no major oversight exercises after the trainings provided to the youth, and no major advances in relation to the recommendations of the studies that the Observatory produced. The interviewee started working in 2020 at CCR, which may have an impact on his actual knowledge of the project. 4.1.8 Court of Accounts of the Republic Conclusions ● There is little evidence of the continuity of citizen oversight initiated by the Observatory. Nor is there any institutional commitment from the CCR to deepen this effort in the short term. ● The lack of institutional commitment is possibly attributable to the national political context of confrontation between the government and CSOs over citizen oversight issues. In November 2020, an unconstitutionality lawsuit was filed against the election of the current CCR magistrates, which was accepted by the Constitutional Chamber of the CSJ in November 2021. The Chamber’s resolution is still pending. 4.1.9 Attorney General’s Office Findings With GIP’ support FGR developed new web systems. The ET was able to corroborate, through the review of GIP reports, that FGR designed online systems for complaints management and public information requests, the institutional web portal, and the FGR Public Information index. FGR strengthened its personnel performance management system. Based on the same source, the ET was able to determine that FGR improved its staff performance evaluation process, 13 the application of a Code of Ethics and reforms were introduced to the Organic Law of the FGR to give a legal mandate to these elements. Given the restriction of access to members of the FGR during this evaluation, the ET could not gather internal opinions and assess the current status of implementation of the newly designed systems. 4.1.10 Supreme Court of Justice Findings The CSJ has been one of the six member institutions of the GTIAC). The ET was able to corroborate in the reports produced by the project that the CSJ participated in the elaboration of the Protocol for the Handling of Corruption Complaints and Whistleblower Protection and led one of the training modules for GTIAC members. Given the restricted access to members of the CSJ during this evaluation, the ET was not able to obtain internal opinions. The ET did not record any opinions from other GTIAC members that would have highlighted any points about the CSJ role in the coordination mechanism. 4.2 EQ2: WHAT HAVE BEEN THE MAIN ACCOMPLISHMENTS AND CHALLENGES OF THE ANTI-CORRUPTION INTERINSTITUTIONAL WORKING GROUP (GTIAC) IN TERMS OF KEY ACTORS/CHAMPIONS, COORDINATION OF THE PARTICIPANT INSTITUTIONS, AND SUSTAINABILITY? The GTIAC’s work entered a phase of stagnation as a consequence of the sudden leadership changes at the CSJ and FGR, which were made by the Legislative Assembly during its inaugural session in May 2021. GTIAC’s work also lost traction as a result of the weakening IAIP role, as previously discussed. The findings on the GTIAC’s accomplishments and challenges are mainly based on the review of the documents supporting the IP’s monthly reports. The ET complemented and cross-checked this information with the opinions and assessments from the KIIs conducted with representatives of two of the six institutions participating in the coordination mechanism. 4.2.1 Findings The GIP played a key supporting role in the creation of GTIAC. As documented by meeting minutes attached to GIP reports, a high-level dialogue between the leadership of the six signatory institutions and IP’s representatives and project-supported meetings with technical staff from these institutions preceded the signing of the Memorandum of Understanding (MOU) formalizing the creation of the mechanism. The MOU was signed by the president of the CSJ, the Attorney General, the Public Defender, the president of the CCR, the president of the TEG, and the president of the IAIP in June 2018. The memorandum establishes four objectives: 1) coordinate and develop activities within the framework of the commitments acquired by signing the MOU, in accordance with the competences of each of the signatory institutions; 2) promote response mechanisms to the phenomenon of corruption; 3) provide interinstitutional collaboration, when required by any of the signatory institutions; and 4) unify criteria.7 7 GTIAC, Carta de Entendimiento para Fortalecer la Cooperación Interinstitucional en la Lucha contra la Corrupción. San Salvador, El Salvador, 4 de junio 2018. 14 Common Agenda and Action Plan proposed by GTIAC’s Technical Committee. With GIP support, GTIAC’s technical working group identified priority areas to work on, including joint training, the improvement of interinstitutional communications and forensic auditing, and drafting a work plan.8 Advances were made with the signing of the interinstitutional communication protocol and the implementation of a joint training program (described below). As corroborated by reviewing the supporting documents of GIP reports, the technical committee prepared an end-to-end mapping of the anti-corruption processes among the member institutions to detect gaps, duplication, and opportunities to improve regulations or mandates. Agreement reached on GTIAC’s Communications and Coordination Protocol. The heads of the six institutions participating in the GTIAC signed a protocol outlining the role of the liaison officers, the SOW, and the constitution and methodology of the Technical Committee. It also identified the interinstitutional communication channels.9 Learning from Latin American experiences and joint training implemented. With GIP’s support, GTIAC had access to lessons learned from the experience of similar coordination mechanism in Latin America10 and conducted study visits to Peru and Uruguay. GTIAC implemented a joint training program,11 covering five modules, with the participation of 35 technical staff members from GTIAC institutions. The training covered the following topics: 1) culture, detection, and systemic anti-corruption measures; 2) intra- and interinstitutional anti￾corruption processes and procedures; 3) evidence-gathering strategies in anti-corruption investigations; 4) government audits and specialized practices in the prevention, detection, and systemic fight against corruption; 5) specialized strategies to prevent, detect, and fight corruption using a systemic approach. An additional module was designed for the training of trainers. During the KIIs conducted with the President and Director of the Investigation Unit of TEG, as well as with the Director of the Integrity Unit at the PGR, the interviewees emphasized that the joint training program opened the opportunity for a peer-to-peer dialogue and interchange of technical criteria, and that the personal contacts and discussion were conducive to a more efficient communication and the harmonization of legal criteria and approaches. Both TEG representatives also consider this experience to have improved the effectiveness of the cases being investigated. The TEG’s President played a key role within GTIAC and is committed to give continuity to the joint training program. Minutes from the GTIAC working session shows the active role played by the TEG’s President; during his KII, he expressed his commitment to sponsor a second edition of the training program.12 8 GTIAC’s Technical Working Table, Propuestas para una Agenda Compartida and Plan de Trabajo Conjunto abril 2018 – marzo 2019. 9 GTIAC, Protocolo para la Comunicación, Coordinación y Colaboración Interinstitucional en el Combate Sistémico de la Corrupción, San Salvador, El Salvador 25 de agosto 2022. 10 Instituto International de Estudios Anticorrupción, Lecciones de algunos esfuerzos de coordinación interinstitucional en la lucha contra la corrupción en Latinoamericana, ppt, San Salvador El Salvador, 15 de febrero 2018. 11 GTIAC Diplomado en Prevención y Combate Sistémico de la corrupción en El Salvador. 12 During a follow-up interview with TEG’s Investigative Unit Director, the ET was informed that the second edition was planned to take place in January 2022. 15 4.2.2 Conclusions ● The formation of the GTIAC in itself represented an important first step towards an effective national anti-corruption policy. It was made possible by the high level of dialogue and trust built between Tetra Tech and the leadership and technical staff of the institutions participating in the coordination mechanism. The GIP worked closely with the technical committee specifically designated to draft the memorandum. ● The opportunity to learn from Latin American anti-corruption coordination mechanisms, and the study visits, enabled the creation of a shared understanding among GTIAC members around the scope and challenges of their work. ● The signing of the communication and coordination protocol and implementation of the joint training program created formal and informal peer-to-peer channels of dialogue and coordination which had a positive impact on the handling of the cases being investigated. ● The continuity and effectiveness of the GTIAC’s efforts are highly dependent on the personal good will of its constituent members. Continued leadership and technical resources are needed to sustain the operation of the mechanism. In practice, Tetra Tech played the role of technical secretariat to the GTIAC and provided the support needed for its operation. 4.3 EQ3: TO WHAT EXTENT HAS THE ACTIVITY CONTRIBUTED TO BETTER RESPOND TO ACCESS TO PUBLIC INFORMATION REQUIREMENTS, IMPLEMENTED MORE EXPANSIVE OPEN GOVERNMENT AND ACCOUNTABILITY PRACTICES, AND ASSISTED COMPLIANCE OF ETHICS STANDARDS IN: 4.3.1 Municipalities Findings Municipal PIOs and ARMOs have been professionalized through GIP training. Of the PIOs and ARMOs interviewed, 80 percent and 100 percent respectively, stated that the trainings have empowered them to comply with citizen information requests to their municipality. They emphasized improved knowledge of the contents of the LAIP, as well as how to explain more clearly to council members and the heads of the mayors’ office units what their obligations are when faced with requests for information. They also agreed that the designation of physical spaces for the Access to Public Information Unit (UAIP) and Archive and Records Management Unit (UGDA) offices, as well as the equipment and furnishings, has helped the rest of the mayors’ office units to recognize the relevance of their functions. The project’s system of indicators shows that 151 PIOs and ARMOs have been trained, representing 58 percent of the total number of municipalities (262). The communities of practice promoted by the project have created useful support networks among the different municipalities. Of the PIOs and ARMOs interviewed, 90 percent stated that these communities have been useful in keeping them informed about training dates and locations, information related to IAIP guidelines, and contact information for other PIOs and ARMOs to resolve doubts about how to proceed with the requests for information they receive. The main tool of communication used has been WhatsApp groups on officer cell phones. All of the interviewees stated that this support network has been particularly useful since the flow of communication from the IAIP to the municipalities has been reduced in 16 the last two years. They noted that the networks were still active without IAIP or project intervention at the time of this evaluation. Municipal employees have a greater understanding of the role of the PIO and ARMO and their obligations regarding the right of access to public information (DAIP). All of the interviewees indicated that their work has been facilitated since the mayors, municipal councilors, and unit heads have come to understand the legal mandate that the LAIP establishes for their municipality. They added that this understanding has increased thanks to the project’s training, and that they feel empowered by knowing that their information requests to the unit heads are backed by a legal mandate. The interviewees indicated that, despite the advances made there is still considerable resistance from some heads of unit and municipal council members, to provide public information and that the work of sensitization needs to continue. The volume of information requests to municipalities does not seem to have increased significantly in recent years. The PIOs interviewed mentioned that the volume of information requests received by their municipality is around 10 to 20 per month, which increases in the months before election campaigns. All of the interviewees reported that the monthly volume has remained stable over the last three or four years, with some relative decline during the quarantine period in 2020 due to the coronavirus disease 2019 (COVID-19) pandemic. The reversal of the local development model based on the decentralization and strengthening of municipal management capacities, represents a risk to access to public information. The PIOs and ARMOs interviewed expressed that the training and support received from the Salvadorean Institute for Municipal Development (ISDEM) has been key guidance in the work of their units. All of those interviewed stated that this orientation will be needed in the future as ISDEM was dissolved in November 2021. In relation to available resources, two of the six ARMOs interviewed noted that during the current year, their allocation of archive space or personnel was reduced and/or allocated to other units, in accordance with priorities established by the new members of the municipal council. Both ARMOs anticipate that their units could be subject to additional future cuts in space or staffing. With the change of municipal authorities in 2021, the continuity of the work of the UAIP and UGDA is at risk. Fifty percent of those interviewed expressed that the new municipal authorities do not know the content of the LAIP and that this could make it difficult for them to receive support for information requests made to other units of the mayor's office. GIP has conducted initial awareness-raising activities on access to public information aimed at the new municipal authorities but there is still a considerable LAIP knowledge gap to fill. 4.3.2 Municipalities Conclusions ● The project has succeeded in strengthening the UAIP and UGDA through the professionalization of their officials and a certain degree of institutionalization of the procedures associated with the LAIP. This has helped to normalize the capacity and fluidity with which the municipalities supported by the project respond to citizen requests for public information. ● This effort has empowered actors in other municipalities who are interested in promoting the access to public information to strengthen and expand their units as well as groups sensitized to the relevance of the issue (i.e., some mayors, counselors, and unit chiefs). 17 ● The strengthening of the UAIP and UGDA does not seem to have increased the volume of information requests received by the municipalities, at least neither the project nor the municipalities have systematically recorded this type of information. ● The effort faces sustainability risks due to the tendency to centralize local development policies, including the reduction in the allocation of public funds from the central level to the municipalities. In this sense, it is foreseeable that the reduction of the Economic and Social Development Fund (FODES) from 10 to 1.5 percent of the National General Budget as of 2022 will generate pressure on the municipalities to redirect UAIP and UGDA resources to other units that the municipal authorities will consider a priority. 4.3.3 Ministry of Public Works Findings The previous administration of the MOP (2014-2019) adopted the IIM to channel its anti-corruption efforts. The three people interviewed from the ministry (two of them former employees at the time of the evaluation) agreed that MOP incorporated IIM into the strategic plans for transparency and corruption control within the institution. This incorporation included the implementation of the institutional self-assessment scorecard (ISAS) in 2018 and 2019, the drafting of an Integrity Improvement Plan to improve results, as well as the elaboration of a institutional corruption risk map, the establishment of an anti-corruption committee within the MOP, a draft of the procedure to manage asset disclosure reports, the standard operating procedures for a complaints management system, and the institutional Code of Ethics. All three interviewees emphasized that the main contribution of the IIM was to bring a conceptual framework and greater coherence to the measures that the MOP authorities wanted to promote in terms of transparency and the fight against corruption. The Minister’s leadership was the critical factor in the MOP’s firm commitment to implement the integrity model. All three interviewees emphasized that, from the beginning, the Minister showed an outstanding personal conviction to promote the fight against corruption from. They affirmed that this political will to promote the issue has been palpable in the Minister since he served as member of the Legislative Assembly. The interviewees stated that the coordination of the efforts from the high hierarchical level made the processes move forward relatively quickly. Two of the three people interviewed also highlighted the leadership role played by the person appointed by the project to work at the MOP’s facilities to coordinate the efforts between the Ministry and the project. The work at the MOP placed more emphasis on strengthening and measuring improvements in internal processes than on measuring the final results available to external stakeholders. During the interviews, it was not possible to identify measurements that reveal how capacity building translates into service improvements perceived by stakeholders. One of the three people interviewed pointed out that elaboration of the risk map during the contract allocation procedures and execution of public works were the most delicate parts of the process, as they identified the sensitive areas most vulnerable to corruption, including the involvement of contractors. He pointed out that one success associated with the GIP’s support was to have completed and installed such a map, and that a practical improvement would have been to establish ways of measuring progress in the management of such a map. 18 The effort remains at a formal level but there is no indication of implementation and progress in the new administration (2019-2024). The current MOP employee interviewed agreed that no progress has been perceived in the capacity-building processes since the change of authorities in mid-2019. The anti-corruption Committee that was created and the functions assigned to the officials are being maintained but there is no activity on their part. Nor has any new ISAS measurement been implemented since the last in 2019. 4.3.4 Ministry of Public Works Conclusions ● The MOP is an example of how the IIM has served as a reference to give coherence and structure to the initiatives on institutional integrity that the authorities sought to promote on their own initiative. ● The ET found visible progress in diagnosing and strengthening processes and policies within the MOP. The team found less progress in the production of systematic measurements of comptrollership that are available to external stakeholders and users. ● The generation and dissemination of this type of information tends to help inform external and internal stakeholders who are involved in the details of the processes to be made transparent or monitored. This type of action tends to empower them vis-à￾vis authorities who are less committed to maintaining transparency policies and it tends to underpin the sustainability of organizational changes. 4.3.5 Ministry of Education and Ministry of Health Findings The design of the online system for managing citizen complaints is a clear achievement in terms of strengthening MINEDUCYT and MINSAL. The study on daily corruption,13 commissioned with GIP’s support, provided useful inputs for the design of the new system. The four people consulted from both ministries highlighted that the ministries currently have several citizens’ “attention windows”—or entry points—where complaints, information requests, and grievances are received, and that the management of these processes is done manually with no standardized registers. The four representatives of MINEDUCYT and MINSAL interviewed stated that the ministries have benefited from having carried out, with GIP’s help, a process mapping to organize the workflow of the different actors and units involved. They also emphasized that the main added value of the system will be the automation of the process, which will allow for more agile case management, as well as the generation of monitoring and performance indicators for better analysis of the processes. They indicated that the system is ready to be implemented but has not yet gone live. The results in terms of dissemination campaigns to encourage reporting are limited. All key informants from both ministries explained that the progress in this type of campaign includes the design of some digital graphic pieces as well as the production of some multimedia content. Yet, this volume of pieces does not exceed ten items. All agreed that the development of the COVID-19 pandemic affected the level of progress on this issue, mainly in MINSAL, where 13 USAID, UFG, UJMD. 2018. Estudio sobre el Fenómeno de la Corrupción en El Salvador y sus Efectos en la Ciudadanía. Dec 2018. 19 the ministry dramatically refocused institutional priorities in 2020 and 2021 on managing the pandemic. Management of the system and dissemination of results will be centralized at the President’s Office (CAPRES). Those interviewed explained that the decision as to when the system will be launched is now up to the CAPRES’s Communications Secretariat, which oversees the official communication agenda in a centralized manner. This secretariat will determine the way in which complaints will be processed through the system, as well as the management of communicating the results. 4.3.6 Ministry of Education and Ministry of Health Conclusions ● The online platform will be a significant aid for a more systematic and more agile management of citizen complaints in MINEDUCYT/MINSAL given the manual processes and the varied number of citizen attention entry points and units that intervene in each ministry. ● Although the system will facilitate case management, the varied number of units and entry points to file citizens’ demands represents an organizational design problem, which will not necessarily be solved by the new technological tool. Ministries may require additional analysis and solutions in organizational design and behavior along the way. ● Centralized management of databases from the CAPRES Innovation Secretariat and dissemination of campaigns to encourage reporting from the CAPRES Communications Secretariat, may represent an obstacle to operational efficiency in the organizational objectives of MINEDUCYT/MINSAL, since the management of complaints responds to a more operational nature of the services provided to the population, rather than to a general official governmental communication line. 4.4 EQ4: TO WHAT EXTENT HAS THE ASSISTANCE PROVIDED TO CIVIL SOCIETY ORGANIZATIONS CONTRIBUTED TO INSTITUTIONALIZING AND EXPANDING TRANSPARENCY AND CITIZEN OVERSIGHT? The GIP worked with a wide range of CSOs, which were reached through the modality of sub grant contracts. In addressing this EQ, the ET sought to classify and analyze the information and data collected according to three subgroups of CSOs: a) Academic Institutions and Think Tanks; b) Advocacy Organizations, and c) Grassroots Organizations. The main findings and conclusions regarding the scope of GIP’s contributions to the institutionalization and expansion of CSOs’ role in promoting transparency and citizen oversight over the use of public funds are outlined below 4.4.1 Findings Academic Institutions and Think Tanks Enhanced participation of Academic Institutions and Think Tanks in the promotion of a nationwide anti-corruption agenda. A grant signed with Francisco Gavidia University (UFG) and Dr. José Matias Delgado University (UJMD) supported the creation of a dialogue platform facilitated by the two universities. Under their lead role, with the active participation of 20 GIP’s other CSOs partners (FUSADES, ISD, FUNDE, Acción Ciudadana, and the Foundation for Democracy, Transparency, and Justice [DTJ]) and with the participation of representatives of the private sector, a National anti-corruption Agreement14 was drafted. Representatives from eighty (80) CSOs signed the Agreement, and a public campaign with the participation of young men and women was launched to disseminate its content and to promote signatures of support from the general public.15 The project presented and discussed the Agreement with representatives of the Legislative Assembly, government institutions, independent regulatory agencies, municipal authorities, and presidential and vice-presidential candidates participating in the 2019 election. During the KIIs conducted with CSO representatives, the ET recorded a unanimous positive assessment of the dialogue platform. CSOs valued the platform highly as a mechanism that allowed the creation of new alliances and collaborative relationships among a wide spectrum of stakeholders, who previously worked separately despite having a common agenda. UFG created an anti-corruption Specialized Monitoring Unit. UFG created a specialized unit to follow up on implementation of the anti-corruption policies and regulations included in the National Agreement. The University Monitoring Center (CUM) published the first report on the subject in July 2021.16 The ET was not able to identify the extent to which the report’s findings were disseminated, there is no public record of the number of downloads of the report nor how many times its content has been quoted or if any follow-up action has been taken. The two UFG representatives interviewed by the ET were confident of the long-term sustainability of the CUM, noting that the initiative cemented the UFG’s leading role, in transparency and anti-corruption issues within the consortium of private universities, and has consolidated the consortium support to continue this line of work. Adoption of technological innovation tools to promote transparency and accountability: The CUM, in close dialogue with FUNDE and ALAC, installed a digital platform known as Centraleaks to report corruption cases in an anonymous way. Due to the confidential nature of the online tool, the ET did not have access indicators that allowed it to assess the use of the platform. Increased research capacities in anti-corruption issues and the publication of specialized studies: With the GIP’s support, researchers from the UFG and UJMD Universities conducted and published a study of daily corruption practices in public services.17 The study showed a high-level of victimization among the public services’ users and identified more than 75 frequent petty corruption practices, with the highest concentration registered in the provision of health, education, security, and municipal public services. The study provided valuable inputs to the GIP-supported design of the system to streamline and manage the complaint cases at MINEDUCYT and MINSAL, as previously mentioned. The GTIAC’s working agenda also took the study into consideration. 14 UFG, UJMD ed. 2018. Acuerdo de País contra la Corrupción, San Salvador, 29 noviembre 2018 15 https://www.ujmd.edu.sv/firma-acuerdo-de-pais-contra-la-corrupcion-yomesumo/ 16 Universidad Francisco Gavidia, Centro de Monitoreo de Medidas y Políticas Anticorrupción, Primer Informe, July 2021https://anticorrupcion.edu.sv 17 USAID, UFG, UJMD. 2018. Estudio sobre el Fenómeno de la Corrupción en El Salvador y sus Efectos en la Ciudadanía. Dec 2018. 21 Advocacy Organizations Development of new skills and advocacy strategies. GIP used technological innovation tools, such as the development of apps and the promotion of open data, and “Hackathons contests,” to promote youth participation and awareness of transparency and accountability issues. The GIP’s partnership with the Association for the Connection to Development of El Salvador (CONEXION) and Transparency, Social Controller, and Open Data (TRACODA) were crucial to reaching new audiences within this population group and creating synergies with other CSOs. The interview with the director of Acción Ciudadana highlighted a clear example of these synergies. He identified the alliance with TRACODA as a key factor in acquiring new knowledge on open data issues and learning how to reach young people. He also credited the support received from with the GIP for Acción Ciudadana’s development of new competencies for monitoring and auditing political parties’ finances. An additional area of new expertise identified by the ET is the development of a strategic litigation capacity in ALAC. In an interview, the ALAC’s director credited the GIP support for developing its institutional capacity in this area. Strengthening of CSOs’ strategic alliances and collaborative relationships: The CSO representatives interviewed were united in their appraisal of the strengthening of existing strategic alliances18 and the facilitation of new alliances between CSOs, the private sector, and international organizations. They expanded their contacts with accountability and anti-corruption international mechanisms (i.e., the Follow-Up Mechanism for the Implementation of the Inter￾American Convention against Corruption [MESICIC]) and were able to establish new contacts with international organizations. CSO representatives also pointed out that GIP support was crucial to building bridges with government institutions between 2015 and 2019. During the KIIs with TRACODA’s director and the CCR representative, both highlighted setting up the Citizen Observatory of the Court of Accounts of the Republic (OCCCR) as a good practice. However, changes in the context in which public institutions operate, including the CCR, meant that there are fewer political incentives to continue with their engagement with citizen oversight practices. Grassroots Organizations Increased awareness among vulnerable populations of access to public information as a citizen’s right: The GIP’s partnership with the ISD was a key factor to reach vulnerable groups with awareness campaigns through community radio broadcasts. This partnership also allowed the systematic inclusion of these groups in transparency training programs. ISD has a wide network of partners, among them Open Future (Futuro Abierto), GMIES, and CCNIS. 18 Two long-standing alliances are: 1) the Consortium for Transparency and Combating Corruption is comprised of FUNDE-ALAC, the Democracy-Transparency-Justice Foundation(Fundación Democracia, Transparencia y Justicia, DTJ), FUSADES, the Centre for Legal Studies, (Centro de Estudios Jurídicos CEJ), and TECHO El Salvador; and, 2) Grupo Promotor por la Ley de Acceso a la Información Pública that supported the approval of the Law of Access to Public Information since 2009 and continues to advocate for the Right to Know, originally constituted by FUSADES, IIDC, UJMD and its Institute of Legal Research, the Journalists’ Association of El Salvador (Asociación de Periodistas de El Salvador – APES), ASDER and the National Association of the Private Sector (ANEP). 22 The ET conducted three FGDs (with young women, representatives of the indigenous population, and members of the LGBTQI+ community) that were organized with the support of ISD. The observations gathered from the FGDs can be summarized as follows: the FGD participants value the new knowledge acquired regarding their right to access public information and the practical aspects of the training sessions, which included the actual exercise of presenting an access-to￾information demand; and they also appreciate the efforts made to adjust the content of the training materials to their needs, interests and cultural sensitivities (this last point was emphasized by CCNIS). Even though they recognized that the health regulations imposed by the COVID-19 pandemic made it impossible to conduct in-person training sessions, FGD participants highlighted the limitations they face in terms of the cost and availability of digital connections. The ET also conducted a KII with a representative of the Association of the Blind of El Salvador. In addition to the value assigned to the Braille-based training, he emphasized that their participation in a GIP-supported activity enabled them to have access and to be heard by government officials. Limited follow up to the training programs: When asked if they had presented a new demand for public information after the training program concluded, none of the FGD participants responded positively. The main reason given is that they still need technical assistance from specialized CSOs. Some participants in the young women’s FGD said that they were part of a training-of-trainers program and, even though they felt confident in their capacity to reproduce the program, they lacked the financial resources to do so. 4.4.2 Conclusions Academic Institutions, Think Tanks, and Advocacy Organizations ● New specialized knowledge and competencies, as well as stronger skills and opportunities for dialogue and forging consensus, are two clearly identifiable contributions GIP has made to CSO strengthening. As a result, GIP have expanded their areas of influence (i.e., strategic litigation, financial oversight of political parties, promotion of open data policies and practices, adoption of new digital tools). ● Strengthened technical competencies and skills to forge strategic alliances have been institutionalized, as shown by the sustained publication of reports, the permanence of the networks built (as expressed by the joint communications issued), and their continuous engagement in working together on a common agenda. ● The collaborative relationship that was being built with public institutions (IAIP and CCR) has suffered setbacks, and the prospects of being reactivated in the medium term are limited. ● The limited number of outcome indicators included in the IP’s MEL Plan, as well as the monitoring tools applied by its partners, does not allow an assessment of how the specialized publications are being used by the intended audiences, the scope of the 23 awareness campaigns implemented, or how they follow up on agreements reached on different dialogue platforms.19 Grassroots Organizations ● The GIP’s partnership with the ISD allowed for the effective inclusion of vulnerable populations through their own organizations. The opinions gathered during the FGDs and through the OLS aimed at young women are positive in terms of the value assigned to the knowledge acquired. ● The cost of and accessibility to digital connection are obstacles that limit the participation of beneficiaries in rural areas. ● Even though the ET got positive feedback from the beneficiaries regarding the content of the courses, it also met requests to adapt the methodology for the online format as well as demands to refresh the training sessions and provide further technical support when presenting requests to the PIOs. 4.5 EQ4.1: WHAT HAVE BEEN THE KEY CHALLENGES FACED BY CSOS TO OVERSIGHT THE USE OF PUBLIC FUNDS? 4.5.1 Findings The CSO representatives interviewed showed consensus in identifying the following obstacles to their work: ● A limited culture of questioning political power as well as citizens’ poor perception of the benefits that their engagement with transparency and accountability activities could bring to their daily life. Two of the CSOs interviewees, expressed the opinion that “transparency issues continue to be an elite concern.” ● The weakening of the regulatory institutions responsible for compliance with the transparency and accountability legal framework (specifically, IAIP, CCR, and FGR) was most noticeable since the replacement of three out of five IAIP commissioners and the threat to dilute the LAIP with the reforms presented by the government to the Legislative Assembly. ● Public policies and priorities regarding transparency and accountability adopted by the central government changed in the context of the COVID-19 pandemic. Exceptions to compliance with the legal dispositions regulating public acquisition were presented by the executive branch of government and approved by the Legislative Assembly. ● Of the CSOs representatives interviewed, all emphasized the rising political and security risks they are facing. They highlighted the language used by government’s representatives to dismiss their work, harassment in the official media, and threats 19 IP’s M&E system is activity oriented. It contains eight indicators and targets to monitor GIP’s interventions with its CSOs partner: five of them are formulated in terms of the number of activities conducted; two measure the number of participants in trainings and CSOs-led events, One indicates the number of dialogue platforms supported. Based on these indicators, GIP shows a strong performance, as the targets assigned for all eight indicators were overachieved. 24 of politically motivated judicial persecution.20 Further risks are associated with the potential approval of the “Foreign Agent Law” that the government presented to the Legislative Assembly in November 2021, which has been denounced at the national and international level as a tool to “control, monitor, cancel the legal status and criminalize human rights organizations and independent media outlets.”21 4.5.2 Conclusions ● In the context of long-term cultural barriers to the questioning of public authorities, less responsive public institutions can jeopardize the progress made regarding citizens’ awareness and willingness to engage with transparency and accountability issues. ● With the current deterioration of the political context, the main challenges faced by CSOs are the legal threats and blocking of much-needed financial support from international donors, related to the potential approval of the Foreign Agent Law. There is also an environment of intimidation against CSOs linked to the use of surveillance spyware against them, as corroborated by independent agencies.22 4.6 EQ4.2: TO WHAT EXTENT IS CSOS’ ENGAGEMENT LIKELY TO CONTINUE AND BE SCALED UP AFTER THE GIP ENDS? 4.6.1 Findings All of the CSO representatives interviewed expressed in general terms their commitment to continue to develop their respective areas of work. As addressed in the EQ4 findings and conclusions, a solid ground for continuity can be found in the institutionalization of the newly developed expertise and in the consolidation of strategic alliances built around a common agenda of transparency, accountability, and anti-corruption objectives. Specific information regarding this point was offered during the interviews with two CSO representatives. ● The two UFG representatives interviewed emphasized the university’s commitment to continue its engagement with transparency and accountability issues. They pointed out that the CUM has been formally integrated into the university’s organizational structure and institutional planning system with financial resources being allocated. They also highlighted the fact that the University 20 The Inter-American Commission of Human Rights (OAS/IACHR) has denounced the dismantling of the democratic institutions and the hostile environment against civil society stakeholders and independent journalists in El Salvador. This environment has been characterized by “intimidating or criticizing messages, disqualification of their work and stigmatization speech promoted from government spheres against the CSO leaders or journalists”. http://www.oas.org/es/cidh/informes/pdfs/2021_ElSalvador-ES.pdf 21 https://www.fidh.org/es/region/americas/el-salvador/el-salvador-la-sociedad-civil-en-peligro 22 https://www.theguardian.com/news/2022/jan/13/pegasus-spyware-target-journalists-activists-el￾salvador?CMP=Share_iOSApp_Other 25 Chancellor, in the capacity of the President of the National Association of Private Universities, has established a wide support network for future activities. ● When interviewed, the ALAC director informed the ET that the organization will prioritize and expand its interventions in the area of advocacy and strategic litigation before international organizations and that requests for financial support have been presented to international aid agencies. 4.6.2 Conclusions ● The CSOs have expressed a strong commitment to continue their efforts, within their respective organizational mandate and expertise, to promote transparency, social oversight, and government accountability as key components within a wider anti-corruption strategy. ● In assessing the likelihood of scaled-up engagement, two limiting factors have to be considered: CSOs’ dependency on financial support from international donors, as discussed above and the increasingly hostile political environment in which they operate. Both factors will gain further weight if the Foreign Agent Law is approved by the Legislative Assembly. 4.7 EQ4.3: TO WHAT EXTENT HAS THE ASSISTANCE PROVIDED BY GIP CONTRIBUTED TO IMPROVING THE ORGANIZATIONAL CAPACITIES OF BENEFICIARY CSOS? 4.7.1 Findings As previously stated, GIP has contributed to the institutional strengthening of CSOs through the development of new areas of expertise, and by strengthening their communication strategies. Its interventions also contributed to strengthening CSOs by facilitating consolidation of strategic alliances among them and supporting platforms for dialogue and collaboration with government institutions. As documented in IP reports, one workshop was held with the participation of GIP’s CSO partners. The purpose of the workshop was to analyze the current context in which they operate and identify the challenges in advancing the transparency and anti-corruption agenda.23 However, there is no indication that other areas of organizational strategic planning have been addressed, such as Stakeholder Mapping, Theory of Change, and Results Based Approach, which, in turn, could lead to a more robust system of outcome indicators. In one specific case, support was given for the organization’s legal registration, as stated during the KII with the director of Transparency, Social Controller, and Open Data (TRACODA). 4.7.2 Conclusions ● As mentioned in previous sections of this report, the ET was able to corroborate specific contributions made by the GIP interventions to strengthen CSOs’ institutional capacities. 23 Proyecto Pro-Integridad Pública, USAID, 2021: Estado de situación y retos de la transparencia y las políticas anticorrupción en El Salvador. Una mirada desde organizaciones de sociedad civil. Presentado por: Hana Sztarkman Arauz 14 de abril de 2021. 26 ● Strengthening CSOs’ organizational capacities for strategic planning and monitoring is an ongoing task pending the adoption of outcome indicators that assess whether the expected changes from their work are being met and, in doing so, enable CSOs to prioritize their interventions. 4.8 EQ5: HOW DO RECIPIENTS AND DECISION-MAKERS OF THE ASSISTED CSOS, MUNICIPALITIES, GOES INSTITUTIONS, AND INDEPENDENT REGULATORY ENTITIES (TEG, PGR, AND CCR) VALUE THE TECHNICAL ASSISTANCE PROVIDED? 4.8.1 Municipalities Findings The PIOs and ARMOs highly valued the training and technical assistance received. All PIOs and ARMOs interviewed stated that they had expanded their knowledge of the LAIP, which empowered them to request and demand information from their colleagues in the different units of the mayors’ office with authority. The new acting PIOs and ARMOs asked for future training exercises. As can be seen in Table 4, 90 percent of the municipal PIOs who responded to the online survey either “agreed” or “strongly agreed” that the topics of the trainings received were relevant (94 percent among women, 86 percent among men), 87 percent that the training were of good quality (81 percent women, 93 percent men), 83 percent that the training formats were adequate (88 percent women, 79 percent men), and 87 percent feel confident to explain or train someone else on the topics received (94 percent women, 79 percent men). Table 4: Perceived Quality of Training Received by Municipals PIOs (N=30) Quality Dimension Strongly Agree Agree Neither Agree Nor Disagree Disagree Strongly Disagree Topics’ relevance 56.7% 33.3% 0.0% 3.3% 6.7% Good quality content 43.3% 43.3% 3.3% 3.3% 6.7% Suitable format 40.0% 43.3% 6.7% 3.3% 6.7% Confident to train someone else 30.0% 56.7% 10.0% 3.3% 0.0% Source: Evaluation OLS. N = Total number of respondents In general, municipal PIOs feel that the new knowledge has been effectively applied and that it will bring about positive changes in their municipality. As can be seen in Table 5 and Figure 1, 93 percent of the respondents either “agreed” or “strongly agreed” that they have been able to apply the new knowledge in their daily work (94 percent women, 93 percent men), and 83 percent that their municipality has been able to implement better institutional integrity mechanisms (88 percent women, 79 percent men). There is a contrast between the strong appreciation of their ability to apply the new knowledge at the individual level in day-to-day work and the perception of low institutional support and the probability attached to the creation of a new culture. As can be seen in Table 5, 60 percent of respondents “strongly agreed” that they have been able to apply the new knowledge in their daily work (63 percent women, 57 percent men), while only 17 percent “strongly agreed” that they have received institutional support and that the new knowledge will produce a new culture that rejects corruption (25 percent women, 7 percent men). 27 28 Table 5: Perception of Municipal PIO on the Application of New Knowledge (N=30) Application Dimension Strongly Agree Agree Agree Nor Disagree Disagree Strongly Disagree Was able to apply the new knowledge during my daily work 60.0% 33.3% 3.3% 0.0% 3.3% Received institutional support to apply the new knowledge 16.7% 46.7% 16.7% 16.7% 3.3% Institution was enabled to implement better institutional 23.3% 60.0% 6.7% 6.7% 3.3% integrity mechanisms The new mechanisms adopted will substantially reduce corruption and improve efficiency 20.0% 53.3% 16.7% 6.7% 3.3% Ethical behavior of people within my institution will be improved 26.7% 50.0% 13.3% 6.7% 3.3% New internal culture that rejects corruption within my institution 16.7% 60.0% 16.7% 3.3% 3.3% will be built The new institutional integrity mechanisms can be sustainably 23.3% 50.0% 16.7% 6.7% 3.3% applied Neither Source: Evaluation OLS. N = Total number of respondents. Figure 1: Perception of Municipal PIO on the Application of New Knowledge (N=30) Source: Evaluation OLS. N = Total number of respondents. 29 4.8.2 Municipalities Conclusions ● The trainings are highly appreciated by the PIOs and ARMOs in the municipalities in terms of learning the essential aspects of their functions and the content of LAIP, especially when they are starting their functions. ● The PIOs and ARMOs seem to be more confident in applying the new knowledge at the individual level in their day-to-day work. In the future, they could benefit from receiving further training on how to manage even greater institutional support. 4.8.3 Central Government Agencies Findings The MOP appreciates having received a guiding model and monitoring tools to facilitate the adoption of transparency mechanisms. All of the interviewees positively valued the IIM as a guiding model for measuring and structuring their efforts related to the fight against corruption. Two of the three people interviewed highly valued the presence of a person assigned by the GIP to work at the MOP offices. MINEDUCYT and MINSAL value the online platform that will enable them to manage the citizens’ complaints in a more efficient way. KIIs with all four representatives from both ministries revealed they positively valued the mapping of the actors involved in the complaints process and the automation that the system promises, once installed. The PIOs and ARMOs of the central government agencies seem to highly value the content of the trainings. As can be seen in Table 6, 87 percent of the PIOs of the central government agencies who responded to the online survey either “agreed” or “strongly agreed” that the topics of the trainings received were relevant (82 percent women, 100 percent men), 87 percent that it was of good quality (82 percent women, 100 percent men), 81 percent that the training formats were adequate (82 percent women, 80 percent men), and 75 percent felt confident to explain or train someone else on the topics received (73 percent women, 80 percent men). As can be seen in Table 7, 83 percent of the ARMOs from central government agencies who responded to the online survey either “agreed” or “strongly agreed” that the training topics received were relevant (89 percent women, 78 percent men), 72 percent that it was of good quality (67 percent women, 78 percent men), 72 percent that the training formats were adequate (67 percent women, 78 percent men), and 67 percent felt confident to explain or train someone else on the topics received (56 percent women, 78 percent men). Table 6: Perceived Quality of Training Received by PIOs of Central Government Agencies (N=16) Quality Dimension Strongly Agree Agree Neither Agree Nor Disagree Disagree Strongly Disagree Topics’ relevance 56.3% 31.3% 12.5% 0.0% 0.0% Good quality content 50.0% 37.5% 12.5% 0.0% 0.0% Suitable format 56.3% 25.0% 18.8% 0.0% 0.0% Confident to train someone else 31.3% 43.8% 18.8% 6.3% 0.0% Source: Evaluation OLS. N = Total number of respondents. 30 Table 7: Perceived Quality of Training Received by ARMOs of Central Government Agencies (N=18) Quality Dimension Strongly Agree Agree Neither Agree Nor Disagree Disagree Strongly Disagree Topics’ relevance 33.3% 50.0% 11.1% 0.0% 5.6% Good quality content 22.2% 50.0% 16.7% 5.6% 5.6% Suitable format 16.7% 55.6% 22.2% 0.0% 5.6% Confident to train someone else 16.7% 50.0% 22.2% 5.6% 5.6% Source: Evaluation OLS, N = Total number of respondents PIOs and ARMOs from central government agencies perceive that the new knowledge has been applied and will bring about positive changes in their institutions. As can be seen in Tables 8 and 9, 94 percent of the PIOs surveyed either “agreed” or “strongly agreed” that they have been able to apply the new knowledge in their daily work (91 percent women, 100 percent men), and 89 percent of the ARMOs felt the same (89 percent women, 89 percent men). There seems to be a higher confidence in terms of the ability to implement the knowledge among the PIOs than among the ARMOs. As can be seen in Tables 8 and 9, 11 percent of the ARMO respondents “strongly disagreed” on most questions related to the applicability and to the expectation that the new knowledge will bring about positive changes. Meanwhile, this category is 0 percent (zero) for all questions at the PIO level. This suggests that there is a group of ARMOs who find it more difficult to apply the newly acquired knowledge. Table 8: Perception of PIO of Central Government Agencies on the Application of New Knowledge (N=16) Application Dimension Strongly Agree Agree Neither Agree Nor Disagree Disagree Strongly Disagree Was able to apply the new knowledge during my daily work 56.3% 37.5% 0.0% 6.3% 0.0% Received institutional support apply the new knowledge to 56.3% 18.8% 25.0% 0.0% 0.0% Institution was enabled to implement better institutional 50.0% 31.3% 18.8% 0.0% 0.0% integrity mechanisms The new mechanisms adopted will substantially reduce corruption and improve efficiency 43.8% 43.8% 12.5% 0.0% 0.0% Ethical behavior of people within my institution will be improved 31.3% 50.0% 18.8% 0.0% 0.0% New internal culture that rejects corruption within my institution 37.5% 43.8% 18.8% 0.0% 0.0% will be built 31 Application Dimension Strongly Agree Agree Neither Agree Nor Disagree Disagree Strongly Disagree The new institutional integrity mechanisms can be sustainably 37.5% 37.5% 25.0% 0.0% 0.0% applied Source: Evaluation OLS. N = Total number of respondents. Table 9: Perception of ARMO of Central Government Agencies on the Application of New Knowledge (N=18) Application Dimension Strongly Agree Agree Neither Agree Nor Disagree Disagree Strongly Disagree Was able to apply the new knowledge during my daily work 27.8% 61.1% 5.6% 0.0% 5.6% Received institutional support to apply the new knowledge 27.8% 33.3% 22.2% 5.6% 11.1% Institution was enabled to implement better institutional integrity mechanisms 22.2% 55.6% 11.1% 0.0% 11.1% The new mechanisms adopted will substantially reduce corruption and improve efficiency 27.8% 44.4% 16.7% 0.0% 11.1% Ethical behavior of people within my institution will be improved 22.2% 38.9% 27.8% 0.0% 11.1% New internal culture that rejects corruption within my institution will be built 16.7% 55.6% 16.7% 0.0% 11.1% The new institutional integrity mechanisms can be sustainably applied 16.7% 55.6% 16.7% 0.0% 11.1% Source: Evaluation OLS. N = Total number of respondents 4.8.4 Central Government Agencies Conclusions ● The training and technical assistance provided to central government institutions seem to have helped them to learn more about institutional integrity, as well as about implementation processes and mechanisms. ● There appears to be no substantive difference between the level of PIOs’ and ARMOs’ satisfaction between municipalities and central government agencies. 4.8.5 Independent Regulatory Institutions Findings The TEG’s staff show an intermediate level of satisfaction and point out challenges in the applicability and specialization of the training topics. All investigators and legal collaborators who participated in the FGD stated that the training received was helpful and that 32 they have acquired new investigation techniques. They pointed out the need to develop even more specialized topics during future trainings. At the same time, they also raised the need to take their workload into consideration, to better manage their time. Most members of the Ethics Commissions positively value the training received. As can be seen in Table 10, 87 percent of the Ethics Commission members who responded to the online survey either “agreed” or “strongly agreed” that the topics of the trainings received were relevant (100 percent women, 60 percent men), 87 percent that they were of good quality (100 percent women, 60 percent men), 87 percent that the training formats were adequate (100 percent women, 60 percent men), and 80 percent felt confident to explain or train someone else on the topics (90 percent women, 60 percent men). Table 10: Perceived Quality of Training Received by Members of Governmental Ethics Commissions (N=15) Quality Dimension Strongly Agree Agree Neither Agree Nor Disagree Disagree Strongly Disagree Relevant topics 40.0% 46.7% 0.0% 0.0% 13.3% Good quality 33.3% 53.3% 0.0% 0.0% 13.3% Suitable format 13.3% 73.3% 0.0% 0.0% 13.3% Confident to train someone else 20.0% 60.0% 6.7% 0.0% 13.3% Source: Evaluation OLS. N = Total number of respondents. The members of the Ethics Commissions consider that they have been able to apply the new knowledge and that it will bring about positive changes in their institution. As can be seen in Table 11, 80 percent of respondents either “agreed” or “strongly agreed” that they have been able to apply the new knowledge in their daily work (100 percent women, 40 percent men) and 93 percent that a new internal culture rejecting corruption will be developed (90 percent women, 100 percent men). There seems to be limited institutional support for the applications of the new knowledge. As can be seen in Table 11 and Figure 2, only 67 percent of respondents either “agreed” or “strongly agreed” that they have received institutional support to apply the new knowledge (70 percent women, 60 percent men). Table 11: Perception of the Members of the Ethics Commissions on the Application of New Knowledge (N=15) Application Dimension Strongly Agree Agree Neither Agree Nor Disagree Disagree Strongly Disagree Was able to apply the new knowledge during my daily work 20.0% 60.0% 6.7% 6.7% 6.7% Received institutional support apply the new knowledge to 13.3% 53.3% 13.3% 6.7% 13.3% Institution was enabled to implement better institutional integrity mechanisms 33.3% 46.7% 13.3% 0.0% 6.7% 33 Application Dimension Strongly Agree Agree Neither Agree Nor Disagree Disagree Strongly Disagree The new mechanisms adopted will substantially reduce corruption and improve efficiency 13.3% 73.3% 6.7% 0.0% 6.7% Ethical behavior of people within my institution will be improved 20.0% 66.7% 13.3% 0.0% 0.0% New internal culture that rejects corruption within my institution will be built 20.0% 73.3% 6.7% 0.0% 0.0% The new institutional integrity mechanisms can be sustainably applied 20.0% 66.7% 13.3% 0.0% 0.0% Source: Evaluation OLS. N = Total number of respondents. Figure 2: Perception of by Members of Governmental Ethics Commissions on the Application of New Knowledge (N=15) Source: Evaluation OLS. N = Total number of respondents. PGR valued the quality of the training and the potential for cultural change that the IIM model represents. The PGR representative interviewed stated that one of GIP’s main contributions was to have transmitted an understanding of ethics as a work ethos. They considered this the basis for the cultural change PGR must apply in the provision of its services. They also found the training methodologies appropriate and stated that the contents presented met high quality standards. The ET did not find solid or systematized evidence during the interviews on how these beginnings of cultural change begin to translate into changes perceived by citizens during the delivery of their services. No clear incentive was identified at the CCR to promote citizen oversight. The CCR representative interviewed could not clearly explain the benefits that the institution saw in 34 establishing relationships with CSOs, nor what the institution's prospects are for promoting this type of action in the future. 4.8.6 Independent Regulatory Institutions Conclusions In general, the level of satisfaction with the contents of the training received with the probability of their applicability is high. There tends to be a little more satisfaction with the contents and formats of the training and a little less satisfaction with the institutional support to apply the new knowledge acquired. 4.8.7 Civil Society Organizations Findings Academic organizations consider that they have a greater capacity to build networks than before. All key informants interviewed considered the support received for building alliances and working with other CSOs to promote diversification and complementarity in capacity-building efforts to be valuable. The specialized and advocacy organizations regard the knowledge acquired and the technical assistance received as timely, pertinent, and of high added value. All of those interviewed evaluated the Tetra Tech team as highly professional, collaborative, and responsive to CSOs’ needs. They highlighted the flexibility that Tetra Tech had in adapting to changes in the national situation. The grassroots organizations are satisfied with the competency development approach of the trainings. All of the participants in FGDs conducted with vulnerable populations and young women agreed that they now feel more empowered as they have acquired new practical knowledge about their rights and tools to access information. The participants also considered the efforts to adapt the training materials to the culture and needs of their respective groups to be very positive. Members of the indigenous population organization, CCENIS, made a particularly strong case. They expressed their appreciation for the opportunity given to them to be involved in the preparation of the materials. However, the participants of all three FGDs raised the need to consider the difficulties of connectivity (access and cost) and the need to improve online pedagogy, pointing out the difficulties in maintaining attention levels during long periods of time, for example, during the courses that were held in the 2019-COVID pandemic. Young women that answered the OLS seem to be moderately satisfied with the trainings on access to public information, transparency, and citizen oversight. As can be seen in Table 12, 56 percent of young women who responded to the online survey either “agreed” or “strongly agreed” that the trainings they received were of good quality. The same percentage felt empowered after the training to exercise their right to access to public information and of citizen oversight. 35 Table 12: Perceived Quality of Training Received by Young Women Beneficiaries on DAIP and Citizen Oversight (N=9) Quality Dimension Strongly Agree Agree Neither Agree Nor Disagree Disagree Strongly Disagree Good quality of training on access to public information 44.4% 11.1% 0.0% 11.1% 33.3% Good quality of training citizen oversight on 44.4% 11.1% 0.0% 11.1% 33.3% Empowered to exercise right access to public information of 33.3% 22.2% 0.0% 11.1% 33.3% Empowered to exercise right citizen oversight of 22.2% 33.3% 0.0% 11.1% 33.3% Confident else to train someone 22.2% 33.3% 0.0% 11.1% 33.3% Source: Evaluation OLS. N = Total number of respondents. On the other hand, young women answering the OLS seem to have used or applied the knowledge acquired in the trainings. As can be seen in Figure 3, 67 percent of the young women who responded to the OLS requested public information after the training and 89 percent participated in citizen oversight exercises in their municipality. Also, 89 percent said that the training did incorporate topics and practical exercises to promote gender equality and the inclusion of vulnerable groups as well. It is worth noting that more than half of the young women (56 percent) perceived that participating in citizen oversight exercises could get them in trouble with the municipal authorities and 11 percent felt the same within their community. 36 Figure 3: Perceived Applicability of the Training Received by Young Women Beneficiaries on DAIP and Citizen Oversight (N=9) Source: Evaluation OLS. N = Total number of respondents. 4.8.8 CSOS Conclusions ● CSOs feel greater capacity and potential for working in networks and coordinating among themselves. This creates a future basis for fine-tuning and specializing their capacity to influence the public agenda. ● CSOs recognize that the national situation has been, and possibly will continue to be, highly changeable. This represents a challenge in adapting to changes without losing the scope and effectiveness of interventions in the long term. 4.9 EQ5.1: WHAT ARE THE HINDRANCES AND BARRIERS THAT RECIPIENTS FACE IN APPLYING THE NEW KNOWLEDGE GAINED THROUGH GIP? A specific set of questions was introduced in the KIIs and FGDs to explore what recipients perceived as barriers and obstacles in applying the new competencies. These are summarized below. 4.9.1 Findings Little knowledge of the new municipal authorities about LAIP and personnel changes in technical positions. During the interviews, the municipal PIOs and ARMOs pointed out that many new municipal council members and recently appointed heads of unit are unaware of the LAIP and its processes therefore they do not collaborate to respond to citizens' requests for public information. 43 percent of the OIPs and ARMOs in the 7 municipalities visited (3 out of 7 in both cases) were replaced after the 2021 municipal elections. 37 Little commitment from the new authorities at MOP to continue efforts on institutional integrity. As mentioned before, the GIP-supported activities at the MOP have been suspended since the change of authorities in 2019. The new authorities do not display the same level of enthusiasm for the organizational reforms related to transparency and the fight against corruption. In addition, 66 percent of the KIIs suggested by the implementer (2 out of 3) no longer work at the institution, suggesting a turnover of key personnel trained by GIP in MOP. The centralized management of MINEDUCYT and MINSAL´s complaints system. Since the management and operational support of the complaints systems is centralized in CAPRES, these ministries depend on CAPRES to start implementing it and will not be able to set their own institutional priorities and operate the system accordingly. The organizational structure of MINEDUCTY/MINSAL related to the receipt and handling of citizen´s complaints. Within the organizational structure of MINEDUCYT/MINSAL, there are different units with overlapping mandates regarding the reception and processing of citizens’ demands and complaints. Even though the system will streamline the case management, its application will require full commitment and training of the staff at these units to implement changes in organizational structure and organizational behavior. The uneven literacy level regarding the use of online platforms presents an obstacle for potential users of the MINEDUCYT/MINSAL system. The lack of a modern civil service career law that includes civil servant performance assessment and protection against unfair dismissal. Middle management civil servants and technical staff are cautious and insecure about expressing opinions on the future implementation of transparency measures in their institutions for fear of losing their jobs or being transferred. The lack of resources allocation to support IT-enhanced processes. The achievements observed regarding the enhanced use of information technology (IT)-supported processes and online platforms require an increased allocation of institutional resources—in terms of both personnel and technical equipment—in order to cope with technical demands and maintenance of these systems. The immediate future of the national political context is uncertain and there is great caution on the part of the regulatory institution’s authorities, whose independence is being curtailed. Their commitment and future actions will clearly depend on the evolution of this context. The main barrier voiced by the CSOs is the increased perception of political risks, resulting from increased harassment and politicized judicial persecution. Such a scenario drastically reduces the possibilities for collaboration and response by public institutions, increases uncertainty about their financial sustainability, and limits their capacity to provide technical assistance to citizens in their demands for access to information. 4.9.2 Conclusions A common thread of the main barriers identified is the lack of continuity of public policies and priorities from one government administration to the next, as well as the high rotation of middle management and technical staff that affects government institutions. The second thread stems from the risks of political persecution impacting the CSOs working on transparency and anti￾corruption issues. 38 4.10 EQ6: WHAT INTERVENTIONS PROMOTED GENDER EQUALITY AND SOCIAL INCLUSION DURING THE IMPLEMENTATION OF GIP? WHAT FACTORS ENABLED OR INHIBITED THE INTEGRATION OF THESE TOPICS? 4.10.1 Findings The ET identified a three-component strategy adopted by the IP to promote gender equality and social inclusion throughout the implementation of GIP: 1) training sessions and technical support for public institutions to help them develop institutional strategies on gender equality and the inclusion of vulnerable groups; 2) awareness campaigns; and 3) training for women and vulnerable groups, with emphasis on socio-economically disadvantaged women, reached through their own organizations. Training sessions and technical support: As reported by the MINEDUCYT and MINSAL representatives interviewed, GIP took data and indicators regarding gender and vulnerable groups GIP into consideration in the design of the complaints management system. MINSAL’s representative added that gender equality and social inclusion policies will be incorporated into the Ministry’s Gender Office Guidance. TEG’s President valued the support received from GIP to disseminate the Tribunal’s gender policy in close coordination with ISDEM and the Human Rights Ombudsperson, as well as the support received for the publication of the government’s Ethics Law in Braille format. As Tetra Tech documented in its reports, a gender policy was being finalized with IAIP when their partnership was suspended. Awareness campaigns using different media platforms: The ISD director highlighted the development of a communication campaign to raise awareness and promote citizen participation in social oversight activities among vulnerable groups. Partnerships were built with the network of community radios (ARPAS) and the municipalities. ISD also partnered with CONEXION, a GIP-supported CSO, for the development of the “discover the key” (#Descubrelaclave) and “knowing my rights” (#conociendomisderechos) campaigns that targeted youth groups. A dialogue platform organized by GIP’s CSO partners enabled discussion of the impact of corruption on women’s rights. GIP specifically designed training programs on access to public information rights and social oversight to meet the needs of young women and vulnerable groups. The ET reviewed the training materials specifically designed for the indigenous, LGBTQ+, and young female audiences. Their content is well structured and presented clearly and follows a methodology for development of competences, that combines conceptual content with practical exercises. Enabling factors: GIP’s mapping of key stakeholders and the partnership with ISD (and its partners GEMIES, CCNIS, and Futuro Abierto) played a central role in facilitating implementation of gender equality and social inclusion interventions. As noted above, this partnership allowed the inclusion of young women and vulnerable groups through their own organizations and, in so doing, empowered them individually and as organized groups. It also facilitated their active participation in the preparation of training materials, as was the case with CCNIS. An additional enabling factor has been the inclusion of three specific ERs in GIP’s MEL Plan, with targets identified for each fiscal year of operation in terms of the number of CSO-led activities to increase citizen participation, awareness, and use of the transparency mechanisms. The 39 standard indicators included in the MEL Plan reported disaggregated data on women, men, and LGBTQI+ people participating in GIP’s initiatives across project components. Limiting factors: As reported during the interviews with MINEDUCYT, MINSAL, and MOP representatives, there is a lack of sensitivity among government staff members regarding gender equality, which is perceived as something that is irrelevant to a public servant’s functions. The municipal PIOs and ARMOs interviewed made a similar observation. One of the government representatives raised a specific limiting factor, referring to a CAPRES communication guideline that restricts the use of gender equality language. 4.10.2 Conclusions Working in partnership with the central government institutions, regulatory agencies and municipalities targeted by the project, GIP has systematically implemented a strategy to promote gender equality and the inclusion of vulnerable groups. Through its alliance with the ISD, and by facilitating the coordination between CSO organizations with different expertise, GIP was able to effectively reach the final beneficiaries and their organizations. While the inclusion of ERs specifically designed for the monitoring gender and social inclusion interventions helped to maintain a clear focus on these issues, the way the indicators were formulated (the number of CSO activities, numbers of mechanisms, number of people reached) is not conducive to gauge how the increased awareness has led to change in attitudes, to a more active engagement, or how new competencies are being used. Despite the technical assistance provided by GIP to public institutions’ staff members on gender equality and social inclusion, and the support given for the development of policies and guidelines, there is no clear evidence of how this has translated into concrete measures nor how they have affected the delivery of public services. 5. RECOMMENDATIONS 5.1 GOVERNMENT & INDEPENDENT REGULATORY INSTITUTIONS ● GIP should maintain an agenda focused on building capacities on the IIM at the technical level and maintaining contacts and networks within the TEG, PGR, and CCR, to the extent that institutional relations with them are possible. Cultural changes at the institutional level requires long-term effort; maintaining training initiatives for the technical teams of these institutions will allow them to consolidate and sustain the GIP’s achievements. They can also be a vehicle for maintaining contact networks between mid-level officials and civil society organizations, journalists, and the media. ● In the case of the IAIP and TEG, the training work should continue to be carried out with PIOs and members of the Ethics Commissions, who are officials that do not belong to the control institutions but are spread across all ministries, municipalities, and other public institutions in a decentralized manner. The role of PIOs spread throughout the country at the municipal level can be an important mechanism to detect and take on emblematic corruption cases at the national level. ● GIP should consider the introduction of ISAS, like those employed by the IIM, to keep track of the implementation of policies and guidelines on gender equality and inclusion of vulnerable groups that have been designed with the GIP support, and to monitor 40 eventual changes in attitudes among the staff of government and regulatory institutions. 5.2 CIVIL SOCIETY ORGANIZATIONS ● GIP needs to explore the possibility to develop protection mechanisms and support measures against political persecution as these will be high on the agenda of CSOs in the next 24 months.24 Organizations could benefit from receiving training on issues of legal and digital protection and learning about the practical experiences of other organizations and social actors at international level that have been successful in maintaining their citizen oversight within hybrid or autocratic democratic systems. These types of measures will help to counteract the incentive for self-censorship that many organizations have begun to apply. ● USAID should train CSOs about public opinion formation processes, such as survey reading, perception management, semiotic interpretation, social communication tools, and traditional and digital media management, as well as training to enhance the professionalization of their communications teams. Without an evolution in these areas, it will be difficult to improve their impact on public opinion. ● To further enhance the CSOs’ capacity to create an informed public opinion, USAID should share international experiences of interorganizational coordination and investigative journalism as a means of enhancing the effectiveness of their own communication strategies. To improve the scope of their awareness campaigns, and engage with citizens nationwide, CSOs would also benefit from learning about experiences using new social media channels. ● USAID should train CSOs on international lobbying expertise to improve their advocacy strategies, such as stakeholder analysis, risk mapping, and designing of fundraising strategies. ● The change in the working context for CSOs to one of political persecution will require the current generation of leaders to change their paradigms and understand the need to develop a more sophisticated and mature advocacy capacity, as well as fostering a new generation of leadership in CSOs. The last year of GIP implementation could serve to promote discussion with current leaders and prepare the groundwork for a new cycle and type of work for the medium term. ● To face the challenges linked to the evolution of the political context, the CSOs will need to maintain and improve their capacity to work collaboratively with new stakeholders, particularly with representatives of political parties, the private sector, and the international community. Organizations will continue to benefit from having structured spaces for the exchange of views and reflection on the potential for collaboration. CROSS-CUTTING RECOMMENDATION ● Improvements to the IP’s M&E system, and that of its partners, can be introduced. Currently, GIP’s M&E system records data for 29 indicators with measurable targets assigned to them (four additional standard indicators have also been recorded). Out 24 Particularly during 2023, which will be a pre-electoral year of legislative, municipal and presidential elections at the same time. 41 of the 29 indicators, two are formulated in a way that allows for assessment of the expected outcomes of the intervention; the remaining 27 are focused on the number of activities, services, and products generated through implementation.25 While these indicators show a strong performance, with 72.4 percent of them being reported as overachieved and 13.8 percent as on target by September 2021, the introduction of outcome-oriented indicators would enable prioritization of the most effective interventions and enhance sustainability.26 Some examples of outcome-oriented indicators are: Government and Independent Public Institutions Gender and social inclusion indicators:  Institutional resources (financial and personnel) allocated to implement the policies and guidelines regarding gender and social inclusion that has been designed with GIP support.  Level of compliance with data disaggregation requirements of the designed management systems.  Action Plans (with clear measures, resources and responsibilities) designed and implemented to address the specific challenges faced by women and vulnerable groups as identified in the daily corruption study. Trainings to develop new competencies:  Number of cases investigated that reach the sentencing phase applying the new technics learned (TEG).  Number of days to process citizens’ complaints presented to government institutions.  Progressive targets of case resolution rate.  Positive resolution rate. Civil Society Organizations Trainings to develop new competencies:  Number of training course repetitions as follow up to the training of trainers’ program.  Number of requests for access to public information presented by women and vulnerable groups trained. Indicators of awareness, reach and engagement in social/digital and traditional media:  Coverage and quotes in traditional media of the reports produced by CSO (TV, radio, newspapers) 25 The two outcome indicators are: Indicator 1.8: Percentage of access to information requests for which selected government agencies provided response within the legal timeframe; and Indicator 1.15: Percentage of access to information requests for which each target Municipality provided response within the legal timeframe. 26 As recommended by the USAID guidelines, it will be particularly useful, in close collaboration with the organizational stakeholder, to identify and prioritise the gaps between the actual and the desired performance and establish specific targets of performance improvements that are clearly linked to the technical assistance provided by the implementing partner. https://www.usaid.gov/sites/default/files/documents/CBLD-9_FAQ_Version_2_.pdf 42  Number of impressions (number of times your post appeared in someone's newsfeed or news section)  Audience growth rate (new followers / followers in your social media)  Social share of voice (mentions of your campaign / all other competing mentions)  Average interaction rate (likes + comments + shares / followers) 43 ANNEXES 44 ANNEX 1: EVALUATION STATEMENT OF WORK STATEMENT OF WORK (SOW) USAID/El Salvador Final Performance Evaluation of the USAID Government Integrity Project A. Purpose of the Evaluation The purpose of the Final Performance Evaluation of the USAID Government Integrity activity (hereinafter referred as “GIP”) is to inform USAID of activity achievements and challenges and determine if the intended outcomes were achieved and if the services were implemented as intended. In addition, the evaluation will serve to provide empirical evidence on management issues and will document learning and continuous improvement. The Mission will use the findings of this evaluation to make changes in the GIP during its remaining life and to have inputs for the design of future activities. As per ADS 201.3.6.4, and given the purpose mentioned above, this evaluation is a combination of two types of evaluations: a) Outcome Evaluation and b) Process or Implementation Evaluation. The principal audience of this evaluation will be USAID/El Salvador; particularly Mission Management, the Democratic Governance and Security Office (DGS), the Regional Office of Acquisition and Assistance, and the Regional Program Office (RPO); and Tetra Tech DPK. Other audiences inside USAID interested in the evaluation results are the Bureau for Latin America and the Caribbean (LAC); the Bureau for Policy, Planning, and Learning (PPL); and the Bureau for Development, Democracy, and Innovation (DDI). External audiences include beneficiaries as well as other donors. The Mission will disseminate the lessons learned and best practices identified in this evaluation to stakeholders to be considered for future similar activities. These stakeholders include the Government of El Salvador (GOES), private sector, academia, and other donors working in public transparency and accountability. This Final Performance Evaluation is scheduled for the fourth quarter of FY 2021, with approximately six months left of implementation of the activity. The evaluation will cover the period from March 3, 2016 through June 30, 2021.27 B. Background Information Activity Implementing Partner Award Number and Dates Funding Government Integrity Tetra Tech DPK AID-519-C-16-00001 March 3, 2016-March 2, 2022 $ 20,291,708.00 The purpose of GIP is to work in partnership with the GOES for effective implementation and compliance with regulations that enhance transparency and accountability in government. GIP targets assistance to strengthen oversight institutions, supports effective implementation in selected GOES agencies, and supports selected municipalities in achieving full implementation of the access to public information and ethics laws. GIP also partners with civil society to foster 27 USAID/El Salvador is negotiating modifications to the contract with Tetra Tech DPK to extend the life of the activity. The evaluation will be based on the scope of the current contract. 45 citizen participation, provide oversight of public resources and GOES decision making processes, advocate for outstanding reforms, and promote universal application of transparency tools among the population, especially vulnerable populations. These efforts contribute to institutionalizing and expanding transparency and citizen oversight, thereby increasing accountability and reducing opportunities for corruption. GIP’s development hypothesis is that if there are well-functioning transparency and accountability regulations and policies and an active and informed civil society, then accountability is effectively increased and opportunities for corruption are reduced in El Salvador, thus leading to higher levels of citizen security and development. GIP has two main objectives, and several components under each objective. Objective 1: Strengthened GOES’ transparency, accountability and professional standards. Objective 1 addresses the needs of key oversight institutions to effectively promote implementation of transparency and professionalism regulations, and assists select GOES institutions at the national and municipal levels to better respond to access to public information requirements, as well as to implement more expansive open government and accountability practices. Under this Objective, GIP also assists the GOES in complying with ethics standards and professionalizing the public service workforce through the promotion of civil service reform and optimization of select human resource management processes. Component A, Capacities of regulatory agencies that oversee transparency and ethics regulations increased: ● IAIP is able to consolidate its role as the implementing institution of the access to public information law ● IAIP is able to strengthen links with civil society ● TEG improves its investigation unit’s capacities through the establishment of precedent-based training programs, protocols, and procedures ● TEG is able to consolidate its role as the implementing institution of the ethics law ● The Tribunal implements at least three transparency and accountability mechanisms to allow citizens to monitor the implementation of the government ethics law in the base period such as public accountability sessions, transparency website, and statistic releases Component B, Central government’s transparency increased: ● Improved response to access to information law requests ● Implementation of transparency, accountability and/or anti-corruption mechanisms Component C, Professionalization of Central GOES public servants enhanced: ● Civil service reform promoted ● Civil service best practices (including merit-based recruitment, evaluation, and promotion) advanced by GOES ● Improved procedures to receive and submit complaints of violations to the Ethics Law to the Government Ethics Tribunal in at least 3 government institutions during the base period (3 years). 46 Option Period (two years): Improved procedures to respond to receive and submit complaints of violations to the Ethics Law in at least 1 additional government institution per year. Component D, Transparency and professionalism at the municipal level increased: ● Implementation of at transparency and accountability mechanisms in selected municipalities ● Improved response to access to information requests and complaints of violations to the ethics law in selected municipalities Objective 2: Citizen oversight, participation and awareness of anti-corruption reforms increased Objective 2 focuses on the demand side by improving Civil Society Organizations’ (CSOs) capacities to develop and implement monitoring and oversight tools, increase citizen participation, and expand awareness on transparency regulations among the general population, with emphasis in rural areas and among people with disabilities. Component A, Civil Society’s capacities to monitor public resources and combat corruption strengthened: ● Contribute to a more accountable government through the development of CSO led oversight/advocacy actions ● Support GOES-CSOs joint actions to combat corruption through at least three GOES-CSO dialogue platforms Component B, Transparency rights in vulnerable populations expanded: ● Increased citizen participation and awareness on transparency rights in targeted vulnerable populations through the development of transparency outreach initiatives ● Increased use of transparency mechanisms by targeted vulnerable populations through the implementation of oversight actions Under the USAID/El Salvador Country Development Cooperation Strategy (CDCS) 2020 – 2025, GIP contributes to the Development Objective 3 “Government Responsiveness, Accountability and Transparency Improved”. Specifically, GIP contributes mainly to the Intermediate Result (IR) 3.3 “Corruption reduced” and to two of its sub-IRs: 3.3.2 “Civil society capacity strengthened to advocate for increased transparency and exposed corruption” and sub-IR 3.3.3 “Transparency of government operations enhanced”. GIP contributes to the following high-level results of the CDCS: Improve demand for government services and accountability through continued civil society organization appeals to the IAIP and improve the national Transparency International’s Corruption Perception Index score by 10 points. GIP also aligns to the third line of action of the U.S. Strategy for Central America “Improving Governance”. C. Evaluation Questions USAID has several learning agendas that help both the Agency and the Mission to fill critical knowledge gaps. 47 One example of Agency-level learning agenda is the DRG Learning Agenda that includes the following learning question: How and under what circumstances can citizen engagement in community decision-making, advocacy, and monitoring influence reforms at higher levels of government? In the CDCS 2020-2025’s Learning Plan, USAID/El Salvador included the following learning question: To what extent have USAID interventions contributed to strengthening El Salvador's anti-corruption systems? The following six main evaluation questions, in priority order, have been identified by USAID/El Salvador to achieve the purpose of this evaluation and to contribute to the Mission and Agency’s learning agendas. The ET should answer them and clearly present them in the Final Report in terms of how they relate to the evaluation purpose. The ET should answer the sub-questions separately in the report and use the data collection and analysis of the sub-questions to answer the main questions. In addition, the ET should incorporate, to the extent feasible, analysis of possible differences associated with gender or social groups, particularly historically excluded groups (youth, people with disabilities, indigenous populations, etc.), and report these separately for men and women. As an example, the ET should consider gender, age, or other social-related barriers when analyzing the obstacles to apply new knowledge in the country. Evaluation questions: 1. To what extent has the GIP contributed to strengthening oversight institutions’ capacity to monitor, disclose, highlight, discipline, investigate, communicate, or bring attention to public resource use? 2. What have been the main accomplishments and challenges of the anti-corruption Interinstitutional Working Group (GTIAC)28 in terms of: Key actors/champions, coordination of the participant institutions, and sustainability? 3. To what extent has the activity contributed to better respond to access to public information requirements, implement more expansive open government and accountability practices, and assist compliance of ethics standards in: ● Selected municipalities; and ● GOES agencies (MOP, Ministry of Health, Ministry of Education, Court of Accounts, and Public Defender's Office)? 4. To what extent has the assistance provided to CSOs contributed to institutionalizing and expanding transparency and citizen oversight? 4.1 What have been the key challenges faced by CSOs to oversight the use of public funds? 4.2 To what extent is CSO’s engagement likely to continue and be scaled up after the GIP ends? 4.3 To what extent has the assistance provided by the GIP contributed to improving the organizational capacities of beneficiary CSOs? 5. How do recipients and decision-makers of the assisted CSOs, municipalities, GOES institutions, and independent regulatory entities (TEG, PGR, and Court of Accounts) value 28 GTIAC: Grupo de Trabajo Interinstitucional Anticorrupción de El Salvador 48 the technical assistance provided? 5.1 What are the hindrances and barriers that recipients face in applying the new knowledge gained through GIP? 6. What interventions promoted gender equality and social inclusion during the implementation of GIP? What factors enabled or inhibited the integration of these topics? D. Evaluation Methodology, Data Collection, and Analysis For this performance evaluation, a non-experimental mixed-methods design that combines a comprehensive, rigorous analysis of existing quantitative data with customized qualitative techniques designed to elicit primary data from a wide range of counterparts, partners, beneficiaries, and stakeholders is recommended. This approach allows for triangulation of complementary data to elucidate linkages between activity inputs and outputs. The ET should consider a range of possible methods and approaches for collecting and analyzing the data to address the evaluation questions thoroughly. The use of participatory methods and activities that will enhance collaboration and dialogue among counterparts is required. Further, data collection and analysis methods should follow applicable Institutional Review Board (IRB) guidance on data security to ensure safety and confidentiality of all individuals providing data or information for the purposes of the evaluation. The finalized evaluation methods and approaches, data collection plan and analysis will be included in the Evaluation Plan submitted to USAID for revision and approval before field visits and data collection begin (see Deliverables section below). The method(s) proposed must comply with the USAID Evaluation Policy and ADS 201. The data collection plan for this evaluation will include, at a minimum: i) a desk review of relevant documents; and ii) key informant interviews and/or focus group discussions promoting equal opportunity of participation of women and men when applicable. USAID/El Salvador expects both qualitative and quantitative data to be collected; and the results will be coded, triangulated, and analyzed for content. The ET is encouraged to propose additional/alternate data collection and analysis methods in the Evaluation Plan that they consider can yield stimulating, robust evidence in answering each of the evaluation questions. Data collection shall be systematic, and data must comply with the five data quality standards of validity, integrity, precision, reliability, and timeliness. Specific interview, survey, and/or focus group protocols will be appended to the Evaluation Plan and finalized with approval from USAID/El Salvador; the questions should be used to answer the evaluation questions listed in this document and address the purpose of this evaluation. All data collected in response to the evaluation questions must have as much level of disaggregation as possible. At a minimum, and per USAID Gender Equality and Women’s Empowerment Policy and ADS 205.3.8.2, all data must be disaggregated and analyzed by sex, as well as analyzed for any differences between effects on men and women or male and female participation. 49 D.1 Desk review of relevant documents USAID/El Salvador will provide the ET with all relevant documents, such as the contract between USAID and Tetra Tech DPK and its amendment(s) including the expected results, performance reports stating the results achieved, etc. The ET must review these documents and other existing literature provided by USAID/El Salvador and others in preparation for the initial team planning meetings and before meeting with local stakeholders for interviews. The ET is expected to review these, make their own contextual literature research, and create a Review Matrix to be delivered to USAID/El Salvador as part of the final Evaluation Report. The Review Matrix should indicate how key information extracted from reviewed documents and other methodologies is linked to each evaluation question. At a minimum, the ET shall review the following documents relevant to the Performance Evaluation. USAID/El Salvador will provide these documents to the ET: ● USAID/El Salvador Country Development Cooperation Strategy 2020 - 2025 ● Contract and its amendments ● Monitoring, Evaluation, and Learning Plan ● Annual work plans ● Monthly and Semi-Annual reports, including monitoring data ● Curriculum of the different trainings provided ● Baseline of compliance indicators in 13 municipalities ● Institutional Integrity Model and Municipal Integrity Model ● Small corruption studies ● USAID/El Salvador Youth Assessment ● USAID/El Salvador Gender and Inclusive Development Analysis and its Supplementary Analysis ● El Salvador’s Access to Public Information Law ● El Salvador’s Government Ethics Law The ET will conduct literature review of other related official documents. D.2 Review of performance and context data GIP has an activity-specific Monitoring, Evaluation, and Learning Plan that contains data collected on a number of standard and custom performance indicators during activity implementation. The ET will use monitoring data on performance indicators as part of the data analysis and should report on it in the Final Report as it relates to the evaluation questions stated above and satisfies relevant data quality standards. The ET will analyze relevant context data related to GIP and include them to the maximum extent possible when answering the evaluation questions. Examples of context indicators or sources related to this activity include Transparency International’s Corruption Perception Index, World Economic Forum’s World Competitiveness Report (Report (transparency), Americas Barometer by LAPOP, and other perception surveys. The ET is encouraged to use context data at the national and municipal level in their analysis, and other geographic analyses. D.3 Key informant interviews, surveys, focus group discussions The ET will interview stakeholders, through key informant interviews, group interviews, short surveys, and/or focus groups discussions. By ADS 201.3.6.8, the ET may directly collect and analyze feedback from GIP 50 beneficiaries. The ET will include both men and women in the stakeholders’ consultation processes, and, if applicable, representatives of vulnerable groups (youth, people with disabilities, etc.). USAID/El Salvador or Tetra Tech DPK will provide key informant contact information once the evaluation begins. At minimum, the ET will collect information from: ● Key USAID/El Salvador staff ● Tetra Tech DPK ● USAID’s Intermediate direct beneficiaries: o Representatives of the six public institutions members of GTIAC (IAIP, TEG, FGR, PGR, CCR-, and the Supreme Court of Justice (CSJ), Salvadoran Institute of Municipal Development -ISDEM-, and municipalities- CSOs (FUNDE-, Acción Ciudadana, CONEXION, TRACODA, UFG, and UJMD) If feasible, the ET will also collect information from other stakeholders: ● USAID staff in Washington D.C. ● USAID’s Ultimate indirect beneficiaries (citizens, academia, journalists, etc.) ● Think-tanks ● Other donors working in areas similar to GIP ● Other actors that can provide an insight into USAID programmatic impacts. A sampling plan describing the selection process; such as purposeful, random, or a combination of approaches;29 for organizations and stakeholders for key informant interviews, surveys, and focus group discussions must be included in the Evaluation Plan and Final Report. D.4 Direct observation In consultation with USAID/El Salvador and Tetra Tech DPK, the ET will select, if applicable, relevant opportunities for direct observation. Selection should be based on the sampling plan developed for the Evaluation Plan and included in the Final Report. Depending on the COVID-19 pandemic situation, remote observation is expected to take place. The ET may conduct direct observation at events hosted or by the activity during the fieldwork period of the evaluation. The ET can use these events to talk with stakeholders and beneficiaries, conduct interviews, and collect additional data as evidence to answer the evaluation questions. USAID/El Salvador and Tetra Tech DPK will provide the ET with a list of events once the evaluation begins, if appropriate. Up to May 31, 2021, Tetra Tech DPK has assisted the following 30 municipalities (in parenthesis, approximate distance in kilometer s from San Salvador downtown): 1. Ahuachapán (100) 2. Santa Ana (68) 3. Sonsonate (65) 4. Armenia (39) 29 Some sampling approaches include quota sampling, proximity sampling, convenience sampling, theoretical sampling, typical case sampling, etc. 51 5. Quezaltepeque (20) 6. La Libertad (35) 7. Zaragoza (21) 8. Santa Tecla (12) 9. San Salvador (0) 10.Mejicanos (4) 11.Ayutuxtepeque (5) 12.Cuscatancingo (5) 13.Ciudad Delgado (7) 14. Soyapango (9) 14.Ilopango (13) 15.Apopa (16) 16.Nejapa (17) 17.San Martín (23) 18.Tonacatepeque (27) 19.Guazapa (28) 20.Panchimalco (20) 21.Santo Tomás (17) 22.Olocuilta (28) 23.Zacatecoluca (64) 24.San Pedro Perulapán (26) 25.Cojutepeque (37) 26.Ilobasco (56) 27.San Vicente (58) 28.Usulután (114) 29.San Miguel (134) The ET will choose a representative sampling of these municipalities to visit. At a minimum, the ET should expect to visit public facilities where Tetra Tech DPK has provided assistance in the three urban hubs: Western, Central, and Eastern. All the facilities to be visited are located in urban areas and they are easily reachable by car departing from San Salvador 52 Team planning meetings An initial team-planning/kick off meeting will be held virtually between USAID/El Salvador and the ET and between Tetra Tech DPK and the ET before the submission of the Evaluation Plan so that USAID/El Salvador can clarify any questions from the ET on expectations, and guidelines. The expected results of this meeting are to: ● Clarify each team member's role and responsibilities ● Confirm the anticipated timeline and deliverables ● Discuss data collection tools and methodologies by evaluation question to be presented in the Evaluation Plan ● Identify communications logistics and how the ET, USAID/El Salvador, and Tetra Tech DPK will communicate with each other Additional meetings may be held as deemed necessary by USAID/El Salvador and/or the ET. E. Deliverables It is estimated that no more than 150 calendar days from the starting date of the evaluation will be required to complete a high-quality evaluation as required under this SOW. During that time frame, the ET shall submit the following deliverables: 1. An Evaluation Plan, in Word Gill Sans font30 size 12, to be completed by the ET 30 If the ET does not have Gill Sans family available; they must use any other approved font as per the USAID Graphics Manual and Partners Co-Branding Guide. 53 after the Team Planning Meetings, no later than 15 calendar days after the starting day of the evaluation. USAID will receive the Evaluation Plan via electronic mail and review it to provide comments no later than five (5) working days after receiving the document. The Evaluation Plan will provide details of how the various deliverables, tasks, and activities will be undertaken. It must include at least: ● Activity description. ● Evaluation design,31 and the explanation of why one design or mix of designs proposed is the most appropriate, its limitations, and how these limitations will be addressed; ● A matrix summarizing the following information per each evaluation question: - Method(s) for data collection, data source, the explanation of why one method or mix of methods is the most appropriate, its limitations and the ways to address them. o Technique(s) for data analysis,32 the explanation of why one analysis technique or mix of techniques is the most appropriate, its limitations and the ways to address them. ● Data Management Plan33 describing the capture of data (for example, interview notes or live recording), storage and transfer, and how all data will be handled in such a manner as to protect the identities of informants in any situations where there are comments could potentially have a negative impact on their employment or security. ● Timeline and/or Milestone Plan, including tentative starting time for data collection and duration of each activity conducted under the evaluation. ● Drafts of data collection protocols, such as questionnaires or focus group moderator guide(s), interview scripts, consent form,34 etc. ● ET composition, roles, and responsibilities. ● Location for the evaluation and Site visit plan, if applicable If the Evaluation Plan includes key informant interviews, surveys, and/or focus group discussions, the Evaluation Plan should include the following information: ● How the interviews/surveys will help to answer the evaluation questions ● Who will conduct the interviews/surveys and why they are qualified to do so ● What the rationale and methods are for deciding the number, timing, and location of the interviews/surveys ● How the participants will be selected and recruited ● How the interviews/surveys will be recorded ● How the interview/survey data will be analyzed and presented 31 Some examples of evaluation designs for performance evaluations include snapshot design, cross￾sectorial design, before-and-after design, time series design, case study design, panel design, etc. 32 Some examples of data analysis techniques include parallel, conversion, sequential, multilevel, data synthesis, content analysis, contribution analysis, etc. 33 For more information on data management planning, see ADS 579.3.1 34 If underage persons (less than 18 years old) participate in this performance evaluation, the ET must make sure to comply with all national regulations related to Child Protection. 54 The Evaluation Plan, particularly the data collection and analysis protocols, as well as interview and focus group guides, must be approved by USAID prior to the start of data collection and the field work. All interview protocols must be submitted in English and Spanish. The ET will have another five (5) working days to make any changes. Once the Evaluation Plan is approved, the ET will submit to USAID an electronic copy in PDF. Any subsequent change to the Evaluation Plan must be approved by USAID. The ET may provide USAID/El Salvador and GIP with a preliminary briefing on the Evaluation Plan prior to the beginning of data collection. 2. Brief weekly bullet reports of activities submitted to the manager of this evaluation by electronic mail due every Monday by the close of business. 3. A Preliminary Findings Briefing for USAID, Tetra Tech DPK, and other stakeholders that USAID/El Salvador consider necessary on the preliminary findings identified by the ET immediately after finalizing the data collection phase and before starting the draft report. According to the audience, the briefing may be conducted in English or Spanish. If the briefing is held in person, only the Team Leader and local Team members need to be present. If the briefing is held virtually, the whole team should attend. The Preliminary Findings Briefing will be used by the ET as a feedback exercise to prepare the Draft of the Final Report. This briefing will be done 60 calendar days after the approval of the Evaluation Plan. 4. Co-creation workshop(s) to draft the recommendations of the evaluation. The idea of the co-creation workshop(s) is to get inputs from USAID/El Salvador, Tetra Tech DPK, and other stakeholders on how the Mission can improve future transparency and anti-corruption activities, based on the findings and conclusions. The ET will define the number of workshops needed as well as the number of hours the workshops will last. Depending on the national sanitary conditions at the moment, the workshop(s) can be in-person or virtual. The ET will be ultimately responsible to draft practical recommendations. 5. A Draft of the Final Report in Word, Gill Sans font size 12, submitted for review due no later than 85 calendar days after the approval of the Evaluation Plan via electronic mail (20 calendar days after the preliminary findings briefing). RPO will be responsible for coordinating the peer-review process with different offices within USAID/El Salvador and USAID/Washington and for distributing it to the implementing partner and other stakeholders for comments. USAID/El Salvador will consolidate all comments and send the draft back to the ET within 10 working days. It is required that the ET use the USAID Evaluation Report Template. Another useful resource is USAID’s how-to note on preparing evaluation reports. At a minimum, and in accordance with the USAID Evaluation Policy and ADS 201, the Final Report and its draft versions must include the following sections: ● An abstract of not more than 250 words briefly describing what was evaluated, evaluation questions, methods, and key findings or conclusions. The abstract should appear on its own page immediately after the evaluation report cover 55 ● Executive Summary of the purpose, background, evaluation questions, findings, conclusions, and recommendations (no more than five pages); ● Purpose of, audience for, and anticipated use(s) of the evaluation. ● Description of the GIP activity, including the award number, award date, funding levels, and implementing partner; ● Brief background information: country and/or sector context; specific problem or opportunity the intervention addresses; and the development hypothesis, theory of change, or simply how the intervention addresses the problem ● Evaluation questions; ● Thorough description of the evaluation design and any challenge/limitations,35 with emphasis on the timeliness and methods for data collection and data analysis; ● Relevant data analysis tables; ● Findings and conclusions drawn from the analysis of the findings;36 ● Action-oriented, practical, and specific recommendations with defined responsibility for the action; ● A dissemination plan of findings, conclusions, and recommendations to intended users of the evaluation, and; ● Appendices: o Original SOW, annotated with any changes approved by USAID o Evaluation and data collection team composition, qualifications and experience, and roles on the team; with signed conflict-of-interest o disclosures for all real or perceived conflicts of interest o Data collection protocols and instruments including questionnaires and checklists o Review matrix of documents consulted o Meeting notes o Complete schedule of evaluation activities, meetings, and interviews - List of individuals and organizations contacted, and sites visited o Tables, graphs, pictures taken during site visits, maps USAID/El Salvador expects to receive a high-quality Draft Report from the ET. USAID/El Salvador will assess the quality of the Draft Report using the Evaluation Report Checklist and Review Template. 6. Final Evaluation Report in PDF, font Gill Sans size 12, no longer than 40 pages in its body, excluding the cover page; Table of Contents; List of Acronyms; and Appendices. The approved Final Report must adhere to USAID’s Evaluation Policy and ADS 201, Criteria to Ensure the Quality of the Evaluation Report,37 and must be submitted in English and Spanish and have incorporated USAID's comments, as appropriate. The Final Report will be due to USAID/El Salvador 10 working days 35 The ET must identify a) steps taken to mitigate limitations, and b) how/whether the limitations affect any particular finding, conclusions, or recommendations. 36 In moving from findings to conclusions, the analysis must be clear as to how findings are synthesized through different techniques such as divergence, convergence, and amalgamation; propensity; weighting; etc. 37 See ADS mandatory reference 201maa 56 after the ET receives comments on the draft and no later than 140 calendar days after the start of the evaluation. Three high-quality printed, bound copies in English and Spanish of the Final Report must be submitted to USAID/El Salvador within 10 calendar days of acceptance of the Final Report. Reports must be printed on both sides. 7. A One-Page summary of the evaluation purpose, findings, conclusions, and recommendations. The One-Page summary will be prepared in English and Spanish in PDF. 8. Any raw data (qualitative or quantitative) collected in electronic form and the respective codebooks in English is due no later than 10 calendar days after the acceptance of the final report. As per ADS 540, the ET must submit to the Development Data Library (DDL), in a machine-readable, non-proprietary format, a copy of any datasets, and their codebooks in English and metadata, that are used (or of sufficient quality) to produce an Intellectual Work. 9. Other deliverables as identified during the Team Meeting and agreed to by USAID/El Salvador and ET. All reports and papers will be considered draft versions until they are approved by USAID/El Salvador. These draft documents must be labeled with the word “DRAFT” in watermark. Findings must be presented as analyzed facts, strong qualitative and quantitative evidence and data, and not based on anecdotes, hearsay or the compilation of people’s opinion. To ensure unbiased findings, there is no guarantee that findings will be modified based on USAID suggestions. The ET will research, investigate, and corroborate as objective any suggestion before it is incorporated in the findings, and the change will be noted in the draft document so as to have a record of the change. All submitted reports and presentations must be thoughtful, well-researched, and well-organized documents, and objectively answer the evaluation questions. When writing the report, the ET must remember the different audiences. The style of writing should be easy to understand and concise, while making sure to address the evaluation questions and issues with accurate and data￾driven findings, justifiable conclusions and practical recommendations.38 The ET should clearly list any biases or limitations that exist during both data collection and analysis (selection bias, recall bias, unobservable differences between comparator groups, etc.). In addition, all real or possible conflicts of interest must be disclosed by each member of the ET in writing. When quoting an individual in any report, the ET must always give the context or circumstances of the quote. Correcting a grammatical error in the quote may be valid, but not rewording an entire phrase. When translating quotes from one language to another, the ET should do so in an idiomatic way and care must be taken to ensure that the tone of the translation is equivalent to the tone of the original. Quotes should be presented in their original language in report texts. All reports must comply with the USAID Graphic Standards Manual and the ADS Style and 38 See ADS mandatory reference 201maa 57 Format Guide. Once USAID has approved the Final Report and the One-Page Summary, the ET will make them 508 compliant and submit them to USAID’s Development Experience Clearinghouse (DEC).39 The ET will email USAID the DEC link for the evaluation reports. USAID may attach a Statement of Differences as an Annex to any Final Report if any differences remain in the final version. F. Evaluation Management ET This performance evaluation will use a combination of multidisciplinary international, regional, and/or local experts. The ET must include at a minimum the following three positions: ET Leader (80 days of LOE) Minimum Qualifications Education: Master’s degree in fields such as Political Sciences, Public Administration, Laws, International Relations, or other fields related to international development is required. Ph.D. or doctorate degree or professional with Doctoral candidacy is a plus. Formal training in monitoring and evaluation is a plus. Language Proficiency: American English Level IV and Spanish Level IV Work Experience: At least eight years of relevant prior experience conducting rigorous external evaluations using both quantitative and qualitative methods for development objectives and monitoring projects and programs overseas, preferably in Latin America. Knowledge and detailed understanding of Central America’s economic, social, cultural, and political characteristics and its development environment is necessary. Working experience in Central America is a plus. At least eight years of project management experience in development is required. Knowledge and/or experience in the field of transparency and anti-corruption in the region is a plus. Experience with management of multidisciplinary teams is a plus. Familiarity with U.S. Government objectives, approaches, operations, and policies, particularly as they relate to evaluations is a plus. Role: The ET Leader will be responsible for overseeing and coordinating all activities related to this performance evaluation and for ensuring the production and completion of quality deliverables in a professional manner, in conformance with this SOW. Public Administration Specialist (60 days of LOE) Minimum Qualifications Education: Appropriate advanced university degree, preferably a master’s degree in Public Administration, Political Sciences, Economics, Law, International Relations, or related fields. Language Proficiency: American English Level III and Spanish Level IV. Work Experience: At least five years of relevant prior experience assessing public organizations (municipal government and central government), preferably with emphasis in transparency and accountability. Experience conducting studies (assessments, evaluations, analyses, etc.) related to the role/contribution of governments in anti-corruption and transparency. 39 As per ADS 540, documents and development assistance projects materials produced or funded by USAID must be submitted for inclusion in the DEC. 58 Civil Society Specialist (60 days of LOE) Minimum Qualifications Education: Appropriate advanced university degree, preferably a master’s degree in Political Sciences, Sociology, Economics, Law, or related fields. Language Proficiency: American English Level III and Spanish Level IV. Work Experience: At least five years of relevant prior experience assessing organizational strengthening, preferably with emphasis in transparency and accountability, and advocacy. Experience conducting studies (assessments, evaluations, analyses, etc.) related to the role/contribution of civil society organizations in public policies. The ET should have considerable experience in designing, monitoring, and evaluating development assistance programs. They must have proficiency in MS Office Suite and have excellent written, proofreading, and oral presentation skills and the ability to conceptualize and write clearly and concisely with attention to details. Understanding of the Latin American context is necessary, with a preference for personnel with work experience specifically in Central America. At least one member must have experience in transforming qualitative data, analyzing quantitative data, and producing data visualization in an easily digestible format. All Team members will be required to provide in advance to USAID/El Salvador a signed statement indicating any conflict of interest. The Team Leader must be someone external to USAID. No ET members shall have been directly involved in the implementation of the activity in any stage. Anyone who has been directly employed by USAID/El Salvador, Tetra Tech DPK, or its partners in the last five years must not be considered as part of the ET. Logistics The ET will be responsible for all logistics support necessary under this SOW, including field office administration, all travel arrangements (with required USAID/El Salvador clearances), team planning facilitation and appointment scheduling, coordination with all partners and stakeholders involved, administrative services (computer support, printing and copying), report editing and dissemination, and for complying with provisions set forth in this SOW. USAID/El Salvador will provide limited support to the ET. This support, if needed, may include assistance in arranging high-level meetings; access to the U.S. Embassy compound in El Salvador, as necessary; and access to all reports, data, and other relevant documents created by Tetra Tech DPK. Given the current political situation in El Salvador and USAID’s decision to redirect assistance away from certain institutions,40 including the Attorney General Office and the Institute of Access to Public Information, USAID/El Salvador will approve in advance any interview request to former and current Government Officials. USAID representatives may accompany the ET for some or all the evaluation, especially in high level meetings. The ET is expected to consider this when making logistical arrangements. At the moment of preparing this SOW, the sanitary conditions due to the COVID-19 pandemic may not allow the expatriate members of the ET to travel or collect data in-person. If conditions improve, and USAID/El Salvador considers that travelling is allowed, then all the 40 https://www.usaid.gov/news-information/press-releases/may-21-2021-usaid-redirects-assistance-salvadoran government-institutions-civil-society 59 members (expatriate and local) can work in the field. Field visits and in person interviews will be held to the extent possible and will be subject to sanitary conditions and protocols. When field visits or in person meetings are not feasible, the ET may use virtual methods. 60 ANNEX 2: BIBLIOGRAPHY Asamblea Legislativa de El Salvador. 2011. Ley de Acceso a la Información Pública, Decreto Legislativo No. 534. San Salvador 8 de abril 2011. Asamblea Legislativa de El Salvador. 2011. Ley Ética Gubernamental, Decreto Legislativo No. 873. San Salvador 13 de octubre 2011. Michel, James. Oct 2018. Institutional Integrity. An essential building block of Sustainable Reform Fusades. 2021. Boletín No 148, 6 razones por las cuales las propuestas de reformas a la LAIP equivalen a su derogatoria. Julio 2021. TEG, PGR, IAIP, CCR, CSJ, FGR. 2018. Carta de Entiendimiento para el Fortalecer la Cooperacion Interinsitutcional en la Lucha contra la Corrupcion del Grupo de Trabajo Interinstitucional Anticorrupción, GTIAC. San Salvador 4 de junio de 2018. ________. 2018. Sistematización del Encuentro Nacional Anticorrupción, GTIAC, San Salvador 28 y 29 agosto 2018. ________. 2019. Diplomado en Prevención y Combate Sistémico de la Corrupción en El Salvador, Módulos 1al 5, GTIAC, San Salvador 2018. ________. 2019. Diplomado en Prevención y Combate Sistémico de la Corrupción en El Salvador Guía de Formación de formadores responsables de la implementación del Diplomado en Prevención y Combate Sistémico de la Corrupción en El Salvador, GTIAC, San Salvador 2019. ________. 2020. Protocolo para la Comunicación, Coordinación y Colaboración Institucional en el Combate sistémico de la Corrupción. G TIAC, San Salvador, 25 de agosto 2020. Tetra Tech DPK. 2021. Estado de situación y retos de la transparencia y las políticas anticorrupción en El Salvador. Una mirada desde organizaciones de sociedad civil. April 2021. ________. 2021. Camarena, Adriana, Anchoring government accountability in a storm of tweets: Understanding USAID/Pro-Integrity’s contributions to public integrity in the current political context of El Salvador. Jul 2021. ________. 2016. FY16 USAID Government Integrity Project Work Plan. Sep 2016. ________. 2017. FY17 USAID Government Integrity Project Work Plan. Sep 2017. ________. 2018. FY18 USAID Government Integrity Project Work Plan. Sep 2018. ________. 2019. FY19 USAID Government Integrity Project Work Plan. Sep 2019. ________. 2020. FY20 USAID Government Integrity Project Work Plan. Sep 2020. ________. 2021. FY21 USAID Government Integrity Project Work Plan. Sep 2021. ________. 2017. Levantamiento de Línea de Base para indicadores de cumplimiento de la normativa de acceso a la información publica en 13 alcaldías de El Salvador y Estudio sobre los factores que condicionan la transparencia y la contraloría social a nivel local. 61 ________. 2017. Manual del Modelo de Integridad Intitucional (MII): Marco Conceptual, Guía de Implementación, Herramientas de Medición. Mar 2017. ________. 2017. Modelo de Integridad Municipal (MIM): Marco Conceptual, Herramientas de Medición y Guía de Implementación. Mar 2017. ________. 2016. USAID Government Integrity Project Monthly Report April 2016. April 2016. ________. 2019. USAID Government Integrity Project Monthly Report April 2019. April 2019. ________. 2020. USAID Government Integrity Project Monthly Report April 2020. April 2020. ________. 2021. USAID Government Integrity Project Monthly Report April 2021. April 2021. ________. 2016. USAID Government Integrity Project Monthly Report Aug 2016. Aug 2016. ________. 2019. USAID Government Integrity Project Monthly Report Aug 2019. Aug 2019. ________. 2020. USAID Government Integrity Project Monthly Report Aug 2020. Aug 2020. ________. 2107. USAID Government Integrity Project Monthly Report Aug 2107. Aug 2107. ________. 2019. USAID Government Integrity Project Monthly Report Dec 2019. Dec 2019. ________. 2020. USAID Government Integrity Project Monthly Report Dec 2020. Dec 2020. ________. 2107. USAID Government Integrity Project Monthly Report Dec 2107. Dec 2107. ________. 2016. USAID Government Integrity Project Monthly Report Dec 2016. Dec 2016. ________. 2018. USAID Government Integrity Project Monthly Report Feb 2018. Feb 2018. ________. 2019. USAID Government Integrity Project Monthly Report Feb 2019. Feb 2019. ________. 2020. USAID Government Integrity Project Monthly Report Feb 2020. Feb 2020. ________. 2021. USAID Government Integrity Project Monthly Report Feb 2021. Feb 2021. ________. 2107. USAID Government Integrity Project Monthly Report Feb 2107. Feb 2107. ________. 2018. USAID Government Integrity Project Monthly Report Jan 2018. Jan 2018. ________. 2019. USAID Government Integrity Project Monthly Report Jan 2019. Jan 2019. ________. 2020. USAID Government Integrity Project Monthly Report Jan 2020. Jan 2020 ________. 2021. USAID Government Integrity Project Monthly Report Jan 2021. Jan 2021. 62 ________. 2107. USAID Government Integrity Project Monthly Report Jan 2107. Jan 2107. ________. 2016. USAID Government Integrity Project Monthly Report Jul 2016. Jul 2016. ________. 2018. USAID Government Integrity Project Monthly Report Jul 2018. Jul 2018. ________. 2019. USAID Government Integrity Project Monthly Report Jul 2019. Jul 2019. ________. 2020. USAID Government Integrity Project Monthly Report Jul 2020. Jul 2020. ________. 2021. USAID Government Integrity Project Monthly Report Jul 2021. Jul 2021. ________. 2016. USAID Government Integrity Project Monthly Report Jun 2016. Jun 2016. ________. 2018. USAID Government Integrity Project Monthly Report Jun 2018. Jun 2018. ________. 2019. USAID Government Integrity Project Monthly Report Jun 2019. Jun 2019. ________. 2020. USAID Government Integrity Project Monthly Report Jun 2020. Jun 2020. ________. 2021. USAID Government Integrity Project Monthly Report Jun 2021. Jun 2021. ________. 2107. USAID Government Integrity Project Monthly Report Jun 2107. Jun 2107. ________. 2018. USAID Government Integrity Project Monthly Report Mar 2018. Mar 2018. ________. 2019. USAID Government Integrity Project Monthly Report Mar 2019. Mar 2019. ________. 2107. USAID Government Integrity Project Monthly Report Mar 2107. Mar 2107. ________. 2016. USAID Government Integrity Project Monthly Report May 2016. May 2016. ________. 2018. USAID Government Integrity Project Monthly Report May 2018. May 2018. ________. 2019. USAID Government Integrity Project Monthly Report May 2019. May 2019. ________. 2020. USAID Government Integrity Project Monthly Report May 2020. May 2020. ________. 2021. USAID Government Integrity Project Monthly Report May 2021. May 2021. ________. 2107. USAID Government Integrity Project Monthly Report May 2107. May 2107. ________. 2016. USAID Government Integrity Project Monthly Report Nov 2016. Nov 2016. ________. 2018. USAID Government Integrity Project Monthly Report Nov 2018. Nov 2018. ________. 2019. USAID Government Integrity Project Monthly Report Nov 2019. Nov 2019. 63 ________. 2020. USAID Government Integrity Project Monthly Report Nov 2020. Nov 2020. ________. 2016. USAID Government Integrity Project Monthly Report Oct 2016. Oct 2016. ________. 2018. USAID Government Integrity Project Monthly Report Oct 2018. Oct 2018. ________. 2019. USAID Government Integrity Project Monthly Report Oct 2019. Oct 2019. ________. 2020. USAID Government Integrity Project Monthly Report Oct 2020. Oct 2020. ________. 2107. USAID Government Integrity Project Monthly Report Oct 2107. Oct 2107. ________. 2016. USAID Government Integrity Project Monthly Report Sep 2016. Sep 2016. ________. 2107. USAID Government Integrity Project Monthly Report Sep 2107. Sep 2107. ________. 2016. May 2016. USAID Government Integrity Report, Monitoring and Evaluation Plan 2017. ________. 2018. Oct 2018. USAID Government Integrity Report, Monitoring and Evaluation Plan 2018. ________. 2016. 2016. Sep 2016. USAID Government Integrity Report, Semi-Annual Report Apr 2016 - Sep ________. 2017. 2017. Mar 2017. USAID Government Integrity Report, Semi-Annual Report Oct 2016 - Mar ________. 2017. 2017. Sep 2017. USAID Government Integrity Report, Semi-Annual Report Apr 2017 - Sep ________. 2018. 2018. Mar 2018. USAID Government Integrity Report, Semi-Annual Report Oct 2017 - Mar ________. 2018. 2018. Sep 2018. USAID Government Integrity Report, Semi-Annual Report Apr 2018 - Sep ________. 2019. 2019. Mar 2019. USAID Government Integrity Report, Semi-Annual Report Oct 2018 - Mar ________. 2019. 2019. Sep 2019. USAID Government Integrity Report, Semi-Annual Report Apr 2019 - Sep ________. 2020. 2020. Mar 2020. USAID Government Integrity Report, Semi-Annual Report Oct 2019 - Mar ________. 2020. 2020. Sep 2020. USAID Government Integrity Report, Semi-Annual Report Apr 2020 - Sep 64 ________. 2021. USAID Government Integrity Report, Semi-Annual Report Oct 2020 - Mar 2021. Mar 2021. ________. 2021. USAID Government Integrity Report, Semi-Annual Report Apr 2021 - Sep 2021. Sep 2021. UFG. 2021. Primer Informe del Centro Universitario de Monitoreo de Medidas y Políticas Anticorrupción. Jul 2021. USAID/El Salvador. Country Development Strategy 2013-2017 USAID. 2021. Informe de Seguimiento al Trabajo desarrollado en 30 Municipalidades de El Salvador. Feb 2021. ________. 2016. Program Description AID-519-C-16-00001 Tetra Tech DPK (Abstract and contract modifications). USAID Mission, San Salvador, El Salvador. ________. 2021. Sistematización de las dos ediciones del Diplomado en Gestión Documental y Archivos dirigido a las municipalidades en El Salvador. Feb 2021. ________. 2021. Statement of Work of USAID/El Salvador Final Performance Evaluation of the USAID Government Integrity Project. USAID Mission, San Salvador, El Salvador. UFG, UJMD ed. 2018. Acuerdo de País contra la Corrupción, San Salvador, 29 noviembre 2018 USAID, CCNIS, Futuro Abierto, GMIES, ISD. 2020. Diplomado en Acceso a la Información Pública y Contraloría Social. El Salvador, septiembre 2020 USAID, UFG, UJMD. 2018. Estudio sobre el Fenómeno de la Corrupción en El Salvador y sus Efectos en la Ciudadanía. Diciembre 2018. 65 ANNEX 3: LIST OF KII PARTICIPANTS Organization Person Position Date and Time Public Institutions TEG Néstor Castaneda President 8 December 2021 - am 11:00 TEG Ada Melvin Villalta Coordinator of the Department Unidad de Instrucción 4 November 2021 - 3.00 pm MOP Liz Aguirre Public Information Officer 10 November 2021 am - 10:00 MOP Mario Cerna Former Manager of Institutional Development Department 3 December 2021 - 8:30 am MOP Marta Díaz de Palomo Former Advisor to Ministerial Office the 2 December 2021 - am 11:00 MINEDUCYT Claudia Sánchez Director of Internal Audit Department 3 December 2021 - am 10:00 Technical Collaborator MINEDUCYT Rossana Pérez at the Office for the Protection of the Rights of the Educational 15 November 2021 am - 8:00 Sector MINSAL Marlene Santamaria Technical Collaborator at the Office "Oficina Derecho a la Salud" 25 November 2021 pm - 2:30 MINSAL Lorena de Bonilla Technical Collaborator at the Office "Oficina Derecho a la Salud" 30 November 2021 am - 9:00 CAPRES / Secre taría de Innovación Sigfredo Mancia Information Technology Technician at the Technological Infrastructure 3 December 2021 - am 11:00 Department Technical Collaborator CCR René Chinchilla at the Planning Department 5 November 2021 - 9:30 am PGR Paola Berdugo Director of the Integrity Department 5 November 2021 - 8:30 am Civil Society Organizations ISD Ramón Villalta Executive Director 11 November am 2021 - 8:00 CONEXION Gilberto Lara Director 9 November 2021 - 8:00 am Acción Ciudadana Eduardo Escobar Executive Director 11 November 2021 am - 10:00 TRACODA Carlos Palomo President 6 December 2021 - am 11:00 ALAC-FUNDE Wilson Sandoval Coordinator 8 December 2021 - 2:00 pm 66 Organization Person Position Date and Time FUNDE Jessica Estrada Director of the Transparency Area 16 November 2021 - pm 3:00 UFG William Rebollo Coordinator of the University Monitoring Center 16 November 2021 - am 9:00 UFG Roberto Morán General Director of the University Monitoring Center 23 November 2021 - pm 2:00 FUNDAMUNI José Rafael Hernandez Former Official 22 November 2021 - pm 3:00 ACES Carlos Obdulio Funes Representative 2 December 2021 - 2:00 pm Former Dean of the UJMD Juan Carlos Fernández Faculty of Graduate Studies and Continuing Education 24 November 2021 - pm 3:00 Qualified Observers CRISTOSAL Ruth Eleonora López Coordinator of anti￾corruption and Justice Department 15 November 2021 - pm 2:00 FUSADES Claudia Umaña President 15 November 2021 - am 10:00 FUSADES Javier Castro Executive Director 15 November 2021 - am 10:00 Municipalities Alcaldía Municipal de San Miguel Miguel Zelaya Public Information Officer 24 November 2021 - pm 2:00 Alcaldía Municipal de San Miguel Carlos Montoya Rubio Archive and Records Management Officer 24 November 2021 - pm 2:00 Alcaldía Municipal de San Pedro Perulapán Xiomara Natalí Sánchez Archive and Records Management Officer 17 November 2021 - am 8:00 Alcaldía Municipal de San Pedro Perulapán Naún Flores Public Information Officer 17 November 2021 - am 8:00 Asociación Comunitaria del Casco Urbano de San Pedro Perulapán (AFUSAP) José Roberto Mejía Representative 17 November 2021 - am 10:00 Asociación Comunitaria del Casco Urbano de San Pedro Emma Elena Teos Representative 17 November 2021 - am 10:00 Perulapán (AFUSAP) Alcaldía Municipal de San Martín Pablo Gamero Rodríguez Archive and Records Management Officer 26 November 2021 - am 8:00 67 Organization Person Position Date and Time Alcaldía Municipal de San Martín Raquel Ismenia Santos Archive and Records Management Deputy Officer 26 November 2021 - am 8:00 Alcaldía Municipal de La Libertad Fátima Azucena Fuentes Public Information Officer 22 November 2021 - am 8:30 Alcaldía Municipal de La Libertad Clara Luz Martínez Archive and Records Management Officer 22 November 2021 - am 8:30 Social Oversight Committee - ISD Tomás de Jesú s Funes Representative 22 November 2021 - am 10:00 Alcaldía Municipal de Armenia Néstor Hugo Santos Public Information Officer 23 November 2021 - am 8:30 Alcaldía Municipal de Armenia Patricia Victoria Molina Archive and Records Management Officer 23 November 2021 - am 8:30 Alcaldía Municipal de Ilobasco Antonio Serrano Mayor 19 November 2021 - am 8:30 Alcaldía Municipal de Ilobasco Johana Belén Rivas Bonilla Public Information Officer 19 November 2021 - am 9:00 Alcaldía Municipal de Ilobasco Oscar Ayala Roberto Archive and Records Management Officer 19 November 2021 - am 9:00 Alcaldía Municipal de Mejicanos Claudia Pérez Díaz Public Information Officer 24 November 2021 - am 8:30 Alcaldía Municipal de Santa Tecla Guillermo Eduardo Mejía Public Information Officer 26 November 2021 - pm 2:00 Tetra Tech GIP Staff Carlos Amaya Director 19 November 2021 - m 12:00 GIP Staff Henry Montano Coordinator of the MINEDUCYT and MINSAL Work Area 12 November 2021 - am 9:00 GIP Staff Celia Valdéz Anti-corruption Specialist 12 November 2021 - am 8:00 GIP Staff Miguel Peñalillo Coordinator Area of Work with CSOs 10 November 2021 - am 8:00 GIP Staff Patricia Navarro Coordinator of the Area of Work with Municipalities 11 November 2021 - am 9:00 GIP Staff Oscar Aguilar Coordinator of the Work Area with TEG, PGR, CCR, MOP 12 November 2021 - am 10:00 GIP Staff Walter Mejía Head of Monitoring and Evaluation Unit 10 November 2021 - am 11:00 USAID USAID Claudia Martínez USAID ProIntegrity COR 27 October 2021 - 9:00 am USAID Martin Schulz USAID ProIntegrity COR 28 October 2021 - 10:00 am USAID María Antonieta Zelaya USAID ProIntegrity COR 9 November 2021 - 9:00 am 68 69 ANNEX 4: LIST OF FGD PARTICIPANTS Organization Person Position Date and Time Indigenous Populations CCNIS - Atiquizaya Néstor Vladimir Pérez Representative 25 November am 2021 - 10:00 CCNIS Alfredo Rivera Representative 25 November am 2021 - 10:00 ADESCOMIIS - CCNIS Carmen Domingez Representative 25 November am 2021 - 10:00 ADESCOMIIS - CCNIS Balmore Zanco Representative 25 November am 2021 - 10:00 CCNIS Betty Pérez Valiente Representative 25 November am 2021 - 10:00 ADESCOIN - CCNIS Rubén Alonso Vásquez Representative 25 November am 2021 - 10:00 CCNIS - CICA Jesús Amadeo Martínez Representative 25 November am 2021 - 10:00 CCNIS Dora Estela Rivera Representative 25 November am 2021 - 10:00 LGTBIQ+ Population Estudiantes LGTBIQ+ UES Génesis Francisca García Vaquerano Representative 24 November am 2021 - 10:00 Liquidambar – Asociación de Mujeres Lesbianas Ales Lavinia Kenny Elizabeth Rosales Medrano Representative 24 November am 2021 - 10:00 Liquidambar – Asociación de Mujeres Lesbianas Ales Lavinia Rosa Alma Ramos Representative 24 November am 2021 - 10:00 Young Women Population Silvia Argueta Ayala 9 December 2021 - 9:00 am María Victyoria Salgado 9 December 2021 - 9:00 am Brenda Saraí Romero 9 December 2021 - 9:00 am Lendy Noemí Argueta 9 December 2021 - 9:00 am Roxa Abigaíl García 9 December 2021 - 9:00 am TEG´s Department: Unidad de Instrucción TEG Carlos Artola Instructor 15 December 2021 - 9:00 am TEG Eduardo Alvarenga Instructor 15 December 2021 - 9:00 am TEG Hérson López Amaya Instructor 15 December 2021 - 9:00 am TEG Roberto Munguía Instructor 15 December 2021 - 9:00 am TEG María Teresa Sagastume Legal Collaborator 15 December 2021 - 9:00 am TEG Natalia Canjura Legal Collaborator 15 December 2021 - 9:00 am TEG Vanessa Cañénguez Legal Collaborator 15 December 2021 - 9:00 am 70 Organization Person Position Date and Time TEG Wendy Mazariego Legal Collaborator 15 December 2021 - 9:00 am 71 ANNEX 5: LIST OF CO-CREATION WORKSHOP PARTICIPANTS Organization Person Position Date and Time ALAC-FUNDE Wilson Sandoval Coordinator 14 December 2022 - 10:00 am FUNDE Jessica Estrada Director of the Transparency Area 14 December 2022 - am 10:00 FUSADES Camila Figueroa Researcher of the Department of Legal Studies 14 December 2022 - am 10:00 Acción Ciudadana Eduardo Escobar Executive Director 14 December 2022 - am 10:00 GIP Staff Miguel Peñalillo Coordinador Área de Trabajo con CSOs 14 December 2022 - am 10:00 72 ANNEX 6: KII AND FGD GUIDES The following guide seeks to evaluate the services and products received within the GIP project framework (e.g., trainings, technical assistance, process design and tools) covering the following criteria: a) relevance of the service/product; b) quality and timeliness; c) applicability and contribution to improve the fulfillment of the institutional mandate; d) scope and challenges in the adoption and use of new knowledge and instruments; e) institutional resources allocated for their adoption and other questions related to sustainability; and f) inclusion of gender perspective and social inclusion. The guide is aimed at government officials, municipal officials, and CSO stakeholders, who have benefitted from GIP assistance. The questions will be used to facilitate both KIIs and FGDs with managerial and technical personnel. The formulation and emphasis of the questions will be adapted to the nature of the position of the interviewed, depending on whether it is a management position/decision-making level or technical level. In the formulation of the questions, the ET will consider the GIP component under which the assistance to the institution has been developed to improve precision and pertinence of the information. Introduction Regards Brief description of the evaluation objectives and presentation of the ET. Your participation is entirely voluntary and the information that you provide to us will be strictly confidential. The results will be presented in aggregated format and will not make any individual level reference. General Information Name Institution/Organization Name Position Start Year Gender Department Municipality GIP Component(s) associated with the institutional work (information to be annotated before the interview, the questions will be formulated specifically to the work component and type of institution) Type of Assistance Received Can you briefly explain the themes and the type of assistance received under the GIP framework (e.g., trainings, technical assistance, tools, and equipment donated, amongst others) 73 Relevance Could you please describe of your institution? the mandate Could you please indicate, in order of importance, the three main priorities of your institution? In relation to the priorities mentioned, how would you assess the relevance of the training and other services received to the mandate and functions of your organization and the role of the training participants from your organization? Assessment on Quality and Timeliness How would you describe the quality of the trainings content, processes and tools developed within the GIP framework? How do you assess the methodologies used and the delivery times or agility of the services? On a scale from 1 to 10 (10 being the highest value) what value would you assign to: ● The training methodology? ● The technical quality of the assistance? ● The delivery times of the technical assistance and related products? Assessments on the Institutional Performance Contribution How do you assess the applicability and/or usefulness of the knowledge, processes and tools developed? From its use/application, do you perceive improvements in the performance of your daily tasks? How would you describe the changes in terms of the ability to fulfill the mandate of your institution? (Specific question for Government Institutions and Municipalities) Do you observe cultural changes in your institutional environment to apply and respect ethics rules and implement transparency and accountability rules? How would you describe these changes? Achievements and Challenges in the Adoption of the New Knowledge, Processes and Tools In your knowledge opinion, have the new & tools been fully applied? Have you found difficulties in applying them? or challenges 74 How would you describe the effect of its application on the culture and work performance in your unit? Institutional Resources Tools Assigned for the Application of New Knowledge, Processes and Have you received support from your superiors for the implementation? Could you explain the kind of support? Have the new knowledge and tools been considered into the work plans of your unity? In what way does the new knowledge and tools have influence in your institutional planning process? Gender and Inclusion Perspective Do you consider that actions/measures were incorporated to promote gender equality during the implementation of the GIP? Could you give us three examples? Do you consider that actions were incorporated to promote the inclusion of vulnerable groups in citizen oversight and access to public information promotion? Could you examples? give us three specific Could you identify external factors (in the political, social, or economic environment) that promoted the inclusion of vulnerable groups? Could you identify external factors that limited the vulnerable groups inclusion? 75 ANNEX 7: OLS QUESTIONNAIRES INTRODUCTION Thank you for taking the time to respond to this online survey about your participation in USAID's Government Integrity Project (GIP). Your help will contribute to evaluate the results of the project to improve transparency and the fight against corruption in El Salvador. It will take you no more than 10 minutes to complete. INSTRUCTIONS It is necessary that you complete the questionnaire only once because the platform does not save the information partially. If you try to fill it out again after exiting, it will start again. CONFIDENTIALITY Your participation is completely voluntary and the information you provide will be kept strictly confidential. Your name will only be used to ensure that responses are not duplicated and will be removed from the database. Results will be presented as aggregate percentages and will not make any reference to the individual level. GENERAL INFORMATION Name Organization Department Position Age 18 – 29 years 30 - 35 years 36 - 59 years 60 years or more Sex Male Female Zone of Residence Rural Urban Department 14 departments Municipality 262 municipalities 1. OLS to members of Government Ethics Commissions / PGR functionaries / PIOS / ARMOS TRAINING How many courses did you receive as a member of _____________ at your institution within the Government Integrity Project? 0, 1, 2, 3, 3, 4, 5, More than 5 Please read the following statements and indicate how strongly you agree with each statement: The contents of the training I received as ___________ at my institution were relevant to my institution. Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree 76 The quality of the training I received as ___________ at my institution was good and adequate. Strongly Disagree Disagree Neither agree nor disagree Agree Strongly agree The format used for the training I received as ___________ at my institution was appropriate. Strongly Disagree Disagree Neither agree nor disagree Agree Strongly agree After the training I felt confident and prepared to explain or train someone else more about the content received. Strongly Disagree Disagree Neither agree nor disagree Agree Strongly agree APPLICATION OF KNOWLEDGE I have been able to apply the knowledge acquired in the training to exercise the functions as a ___________ during my daily work. Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree I have received the necessary support from my institution to apply the knowledge acquired. Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree 77 The training and/or technical assistance received enabled my institution to implement better institutional integrity mechanisms. Strongly Disagree Disagree Neither agree nor disagree Agree Strongly agree The new mechanisms adopted will reduce corruption and improve efficiency substantially within my institution. Strongly Disagree Disagree Neither agree nor disagree Agree Strongly agree The training and/or technical assistance received will substantially improve the ethical behavior of people within my institution. Strongly Disagree Disagree Neither agree nor disagree Agree Strongly agree 78 These new behaviors of people and better mechanisms will build a new internal culture that rejects corruption within my institution. Strongly Disagree Disagree Neither agree nor disagree Agree Strongly agree The new institutional integrity mechanisms will be able to be sustainably implemented in the coming years in my organization. Strongly Disagree Disagree Neither agree nor disagree Agree Strongly agree 2. OLS to young women beneficiary population TRAINING How many courses did you receive under the Government Integrity Project (also known as Pro-Integrity)? 0, 1, 2, 3, 3, 4, 5, More than 5 Please enter the name of the organization or university that invited you to participate. What type of training did you receive under USAID's Pro-Integrity Project? - Access to Public Information Training - Government Ethics Training - Training on Transparency - Training in Citizen Participation mechanisms - Training in Citizen Comptrollership Mechanisms - Training on Gender Equity - Training on Inclusion of Vulnerable Groups - Training for Trainers - All of the above - None of the above. Please explain what the training was about. The content and times used for the training contributed to improve my knowledge of the mechanisms for exercising my right of access to public information: - Strongly Disagree - Disagree - Neither agree nor disagree - Agree - Strongly agree 79 The contents of the training I received on transparency mechanisms and citizen comptrollership was good and adequate: - Strongly Disagree - Disagree - Neither agree nor disagree - Agree - Strongly agree The duration time and format of the training I received contributed to improve my knowledge in mechanisms for exercising my right to citizen participation and social/citizen comptrollership of public funds: - Strongly Disagree - Disagree - Neither agree nor disagree - Agree - Strongly agree After the training I felt confident and prepared to explain or train someone else more about the content learned: - Strongly Disagree - Disagree - Neither agree nor disagree - Agree - Strongly agree After the training I felt confident and prepared to exercise my right to request public information from state institutions and/or municipal authorities. - Strongly disagree - Disagree - Neither agree nor disagree - Agree - Strongly agree After the training I felt confident and prepared to exercise my right as a citizen comptroller. - Strongly disagree - Disagree - Neither agree nor disagree - Agree - Strongly agree TECHNICAL ASSISTANCE/ADVICE/ACCOMPANIMENT What type of technical assistance and/or advice have you received through the Pro-Integrity Project? - For the request of public information - To exercise citizen participation rights 80 - To exercise citizens' rights of social/citizen oversight - To empower citizen rights - To work in alliance with vulnerable groups (people with disabilities, indigenous people, LGBTI people, etc.) - To improve communication with electoral authorities - All of the above - None of the above In case you have made requests for public information in your Municipality and/or in a State institution: - You received a satisfactory response - You received an unsatisfactory response. Why was it unsatisfactory ? - You did not receive a response - You have never requested public information from the Municipality or any State institution. Have you participated in social/citizen comptroller exercises in your municipality? -Yes - No - I am not interested The tools provided in the Pro-Integrity project trainings have been your main motivation and incentive to get involved in social/citizen comptrollership exercises? -Yes -No -Not applicable Do you consider that citizen participation exercises contribute to: - Improve the transparency of municipalities - Guarantee good use of public resources - Exercise citizens' rights to receive public information and exercise social/citizen oversight - Inform municipal authorities of citizen demands and priorities. - To demand transparency and accountability from municipal authorities. - Demand transparency in the use of public resources - Exercise citizens' rights and improve the welfare of the municipality's inhabitants. - All of the above - None of the above You consider that participating in citizen/social comptroller exercises could cause you any of the following problems: - Problems with municipal authorities - Problems with groups or individuals in your neighborhood and/or municipality - Political persecution - Problems of access to public positions - All of the above - You are not a politician and therefore you are not interested in participating in these exercises. 81 - Because of your work, you do not have time to participate in these exercises. - None of the above GENDER EQUITY AND VULNERABLE GROUPS From your perspective, did the training received incorporate themes, actions and/or practical exercises to promote gender equity? - Yes - Partially - In a cross-cutting manner - They did not address gender equity issues. From your perspective, did the training received incorporate topics and/or practical exercises to promote the inclusion or participation of vulnerable groups? Yes Partially In a cross-cutting manner They did not address the issue of inclusion and/or participation of vulnerable groups. From your perspective, were there any factors that prevented the integration of gender equity and/or inclusion of vulnerable groups? - No In a cross-cutting manner - Yes - Explain what these factors were: