EVALUATION MIDTERM PERFORMANCE EVALUATION OF THE USAID/EL SALVADOR REGIONAL TRILATERAL COOPERATION PROJECT December 28, 2018 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Gary Woller, Team Leader, Randal Thompson, International Cooperation Specialist, and Gladys Soler, Research Associate, under the USAID/El Salvador Monitoring, Evaluation, and Learning Initiative. MIDTERM PERFORMANCE EVALUATION OF THE USAID/EL SALVADOR REGIONAL TRILATERAL COOPERATION PROJECT Submitted on: December 28, 2018 Prepared for the United States Agency for International Development under AID-OAA-I-15- 00024/TOAID-519-TO-16-00002 USAID/El Salvador Contact: Orlando Hidalgo ohidalgo@usaid.gov Submitted by: Gary Woller, PhD, Team Leader Randal Joy Thompson, PhD, International Cooperation Specialist Gladys Soler, Research Associate USAID Monitoring, Evaluation, and Learning Initiative Photo Caption: Participants in the Ruta N Trilateral Activity in Medellín, Colombia. Photo Credit: Taken by RTC Project DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ABSTRACT The Regional Trilateral Cooperation (RTC) project supports building trilateral cooperation between the United States Government (USG), provider countries in Latin America, and Northern Triangle countries in Central America. Together, these entities can support the technical expertise and knowledge transfer, while building and strengthening collaborative intra￾regional relationships. The RTC midterm performance evaluation examined 1) efficiency of project resource use, 2) whether collaborative relationships exist and are sustainable, and 3) how recipient countries use their new knowledge. The evaluation concludes that the trilateral exchanges facilitated by RTC have produced several important benefits including knowledge gained and applied, and cooperative relationships strengthened or formed with counterparts in the recipient and provider countries. These relationships have mostly endured and providers and recipients expect them to continue to endure—at least as contingent assets, if not as ongoing, active relationships. As a mechanism for facilitating, incentivizing, and demonstrating the value of trilateral exchanges, RTC has been highly effective. However, the effectiveness has been at a high a cost relative to other possible approaches for facilitating trilateral cooperation. Its perceived cost-effectiveness and value-added are constrained by its primary role as a logistical support project, although it may take on more roles to increase its value-added. Whether USAID/El Salvador should continue RTC into its option year depends on whether it can expand RTC’s current role and add more value. CONTENTS EXECUTIVE SUMMARY.................................................................................................................................. i Introduction........................................................................................................................................................ i Methodology....................................................................................................................................................... i Main Findings, Conclusions, and Recommendations.......................................................................... i I.0 EVALUATION BACKGROUND, PURPOSE, AND AUDIENCE ................................................... 1 1.1 Evaluation Background ......................................................................................................................... 1 1.2 Evaluation Purpose and Audience.................................................................................................... 1 2.0 EVALUATION QUESTIONS.................................................................................................................. 2 3.0 EVALUATION METHODOLOGY........................................................................................................ 2 3.1 Data Collection Methods.................................................................................................................... 2 3.2 Sampling Methods.................................................................................................................................. 4 3.3 Evaluation Team...................................................................................................................................... 4 3.4 Methodological Limitations................................................................................................................. 4 4.0 FINDINGS AND CONCLUSIONS....................................................................................................... 5 4.1 EQ1: To what extent are the beneficiary organizations applying the new knowledge transferred by SSC provider countries through the RTC project? ...................... 6 4.1.1 Findings ............................................................................................................................. 6 4.1.2 Conclusions ...................................................................................................................... 9 4.2 EQ1.1: What are the hinderances and barriers that beneficiaries face in applying this newly transferred knowledge?............................................................................................. 9 4.2.1 Findings ............................................................................................................................. 9 4.2.2 Conclusions ................................................................................................................... 11 4.3 EQ2: In terms of the use of resources—such as cost and staff time, timeliness to respond to needs, and other factors—what are the advantages and disadvantages of a centralized approach like RTC’s, and how to they compare to those of other activities with trilateral exchanges as a component or sub￾component of a development project?........................................................................... 11 4.3.1 Findings .......................................................................................................................... 11 4.3.2 Conclusions ................................................................................................................... 24 4.4 EQ3: To what extent has the RTC project contributed to building partnerships among the USG, the government(s) of the recipient country(ies), private sector and civil society of El Salvador, Guatemala, and Honduras (Northern Triangle of Central America), and those of SSC provider countries? ............... 25 4.4.1 Findings .......................................................................................................................... 25 4.4.2 Conclusions ................................................................................................................... 28 4.5 EQ3.1: How do the above recipient countries perceive the trilateral cooperation facilitated by the RTC project compared with traditional North-South Cooperation?............................................................................................................................ 28 4.5.1 Findings .......................................................................................................................... 28 4.5.2 Conclusions ................................................................................................................... 32 4.6 EQ3.2: To what extent have the beneficiary organizations built networking with their counterparts or private sector in the SSC provider countries that may lead to future collaboration beyond the life of the RTC project?....................... 33 4.6.1 Findings .......................................................................................................................... 33 4.6.2 Conclusions ................................................................................................................... 37 5.0 RECOMMENDATIONS........................................................................................................................ 37 6.0 ANNEXES ................................................................................................................................................ 39 Annex A: Evaluation Statement of Work ............................................................................................ 40 Annex B: Bibliography.................................................................................................................................. 53 Annex C: List of KII Participants............................................................................................................. 61 Annex D: KII Discussion Guides............................................................................................................. 67 Annex E: WBS Questionnaire .................................................................................................................. 83 Annex F: WBS Responses.......................................................................................................................... 95 Annex G: New Knowledge Applied and Relationships Formed by TLA Participants and RTC’s Role in Trilateral Activities................................................................................. 106 Annex H: Summary Description of Other Trilateral Cooperation Initiatives................... 122 Annex I: Trilateral Cooperation Activities in El Salvador by Selected Multilateral Donor Agencies................................................................................................................................... 133 LIST OF TABLES Table A1: Summary of Findings, Conclusions, and Recommendations................................................ii Table 1: Number of KIIs and People Interviewed by Stakeholder Group........................................... 3 Table 2: Number of WBS Respondents by Respondent Category....................................................... 4 Table 3: Perceived Quality of USAID-Facilitated Trilateral Exchanges (N=72) ................................. 6 Table 4: How Trilateral Exchange Beneficiaries Have Applied New Knowledge (N=50) ............... 7 Table 5: Hindrances and Barriers in Applying New Knowledge for Trilateral Exchange Beneficiaries (N=50)......................................................................................................................... 9 Table 6: Costs per Trilateral Exchange Under Alternative Cost Scenarios..................................... 12 Table 7: Mission and IP Support for Expanded RTC Role (N=12)..................................................... 16 Table 8: Mission and IP Support for Future Trilateral Cooperation and Continuing RTC (N=12) .......................................................................................................................................................... 18 Table 9: Perceived Quality of USAID-Facilitated Trilateral Exchanges (N=40) .............................. 23 Table 10: Additionality of Trilateral Exchanges in Strengthening and Forming Cooperative Relationships (N=49)................................................................................................................... 27 LIST OF FIGURES Figure 1: Increase in Trilateral Exchange Participants’ Knowledge ....................................................... 6 Figure 2: Level of Agreement Whether Trilateral Exchanges Produced Tangible Benefits (N=72) ............................................................................................................................................................. 7 Figure 3: Perceived Cost-Benefit of RTC by Mission and IP Staff (N=12)........................................ 13 Figure 4: Additionality of RTC in Incentivizing Participation in Trilateral Exchanges (N=72)...... 20 Figure 5: Quality of RTC-Facilitated Relationships Compared to Those Formed from Other Trilateral or Bilateral SSC Exchanges (N=41)........................................................................ 23 Figure 6: Effectiveness of USAID-Facilitated Trilateral Exchanges in Strengthening or Forming Cooperative Relationships (N=72)........................................................................................... 25 Figure 7: Trilateral Exchange Participants Strengthening or Forming Cooperative Relationships with Home Country and Foreign Country Counterparts (N=62).................................... 25 Figure 8: Perceived Benefits from Participating in USAID-Facilitated Trilateral Exchanges (N=72) .......................................................................................................................................................... 31 Figure 9: Intentions to Participate in Future RTC-Facilitated or Other Trilateral Exchanges (N=72)............................................................................................................................................. 32 Figure 10: Mission and IP Staff Willing to Invest in Trilateral Exchanges Without RTC............... 32 Figure 11: Number of New Relationships Formed During Trilateral Exchanges and Six Months After............................................................................................................................................... 34 Figure 12: Likelihood of Sustaining Cooperative Relationships Strengthened or Formed as a Result of USAID-Facilitated Trilateral Exchanges (N=48)................................................. 34 Figure 13: Factors Contributing to the Sustainability of Strengthened or New Cooperative Relationships (N=46) ................................................................................................................. 35 ACRONYMS ABC Portuguese Acronym for Agência Brasileira de Cooperação (Brazilian Cooperation Agency) AECID Spanish acronym for Agencia Española de Cooperación Internacional para el Desarrollo (Spanish Agency for International Cooperation for Development) AGCID Spanish acronym for Agencia Chilena de Cooperación Internacional para el Desarrollo (Chilean Agency for International Cooperation for Development) AMEXCID Spanish acronym for Agencia Mexicana de Cooperación Internacional para el Desarrollo (Mexican Agency for International Development Cooperation) ANVISA Portuguese acronym for Agencia Nacional de Vigilância Sanitária (Brazilian Health Regulation Agency) APC Spanish acronym for Agencia Presidencial de Cooperación (Colombian Presidential Agency of International Cooperation) APCI Spanish acronym for Agencia Peruana de Cooperación Internacional (Peruvian International Cooperation Agency) ARSA Spanish acronym for Agencia de Regulación Sanitaria (Honduran Health Regulation Agency) ASI Spanish acronym for Asociación Salvadoreña de Industriales (Salvadoran Industry Association) BMZ German acronym for Bundesministerium für wirtschaftliche Zusammenarbeit (Federal Ministry of Economic Cooperation and Development) CAM Central America and Mexico CDCS Country Development Cooperation Strategy CIDA Canadian International Development Agency COFEPRIS Spanish acronym for Comisión Federal Para la Protección Contra Riesgos Sanitarios (Mexican Federal Commission for the Protection Against Sanitary Risk) CONAMYPE Spanish acronym for Comisión Nacional de la Micro y Pequeña Empresa (National Commission for Micro and Small Enterprises) COP Chief of Party COR Contract Officer Representative CSO Civil Society Organization DAC Development Assistance Committee DDL Data Development Library DEC Development Experience Clearinghouse DFID Department for International Development ECA Economic Competitiveness Activity EQ Evaluation Question ET Evaluation Team EU European Union FAO Food and Agriculture Organization of the United Nations FUSADES Spanish acronym for Fundación Salvadoreña para el Desarrollo Económico y Social (Salvadoran Foundation for Economic and Social Development) FY Fiscal Year GIZ German acronym for Deutsche Gesellschaft für Technische Zusammenarbeit (German Corporation for International Cooperation) GOES Government of El Salvador HEA Higher Education for Economic Growth Activity IAIP Spanish acronym for Instituto de Acceso a la Información Pública (Institute of Access to Public Information) IBSA India, Brazil, and South Africa IBTCI International Business & Technical Consultants, Inc. IFAD International Fund for Agriculture and Development IOM International Organization for Migration IP Implementing Partner IsDB Islamic Development Bank JICA Japan International Cooperation Agency KII Key Informant Interview KOICA Korean International Cooperation Agency LAC Latin America and the Caribbean LOP Life of Project Lux-Dev Luxembourg Development Cooperation Agency M&E Monitoring and Evaluation MEL Monitoring, Evaluation, and Learning MENA Middle East and Northern Africa MINED Ministry of Education MINEX Spanish acronym for Ministerio de Relaciones Exteriores (Ministry of Foreign Affairs) MOF Ministry of Finance MOFCOM China’s Ministry of Commerce MOP Ministry of Public Works MOU Memorandum of Understanding MSME Micro, Small, and Medium Enterprise N/A Not Available NGO Non-Governmental Organization OAS Organization of American States OECD Organization for Economic Cooperation and Development OFID OPEC Fund for International Development PDS Project Development Support RNP Rwanda National Police RTC Regional Trilateral Cooperation Project SENA Spanish acronym for Servicio Nacional de Apredizaje (Colombian National Training Service) SETEPLAN Spanish acronym for Secretaría Técnica y de Planificación (Technical and Planning Secretariat) SIECA Spanish acronym for Secretaría de Integración Económica Centroamericana (Central American Integration System) SOW Statement of Work SSC South-South Cooperation STTA Short-Term Technical Assistance TA Technical Assistance TCS Trilateral Cooperation Secretariat TCTP Third Country Training Program U.S. United States UK United Kingdom UNDP United Nations Development Program UNICEF United Nations International Children’s Emergency Fund U.S. United States USAID/CAM United States Agency for International Development in Central America and Mexico USG United States Government VCMD Vice Ministry of Cooperation for Development WBS Web-Based Survey EXECUTIVE SUMMARY INTRODUCTION This Executive Summary presents an overview of the primary findings, conclusions, and recommendations from the midterm performance evaluation of the United States Agency for International Development in Central America and Mexico (USAID/CAM) Regional Trilateral Cooperation Project (RTC). The purpose of this evaluation is to inform USAID/CAM on 1) the efficiency of the use of project resources, 2) the level of leverage achieved, and partnerships built, and 3) how the new knowledge is being applied in the recipient countries. USAID/CAM will use the evaluation’s findings and recommendations both to decide how to exercise the project’s option period and to inform the potential design of future trilateral cooperation mechanisms. The evaluation answered the following three evaluation questions (EQs) and three sub-EQs: 1. To what extent are the beneficiary organizations applying the new knowledge transferred by South-South Cooperation (SSC) provider countries through the RTC project? 1.1 What are the hindrances and barriers that beneficiaries face in applying this newly transferred knowledge? 2. In terms of the use of resources such as cost and staff time, timeliness to respond to needs, and other factors, what are the advantages and disadvantages of a centralized approach like RTC’s, and how do they compare to those of activities with trilateral exchanges as a component or sub-component of a development project? 3. To what extent has the RTC project contributed to building partnerships among the United States Government (USG), the Government(s) of the recipient country(ies), the Private Sector and Civil Society of El Salvador, Guatemala, and Honduras (Northern Triangle of Central America), and those SSC provider countries? 3.1 How do the above recipient countries perceive the trilateral cooperation facilitated by the RTC project compared with traditional North-South Cooperation? 3.2 To what extent have the beneficiary organizations established networking with their counterparts or the private sector in the SSC provider countries that may lead to future collaboration beyond the life of the RTC project? METHODOLOGY The evaluation team (ET) employed a mixed-methods data collection design that included document review, key informant interviews (KIIs), an online stakeholder survey, and a review of RTC performance and financial data. The ET reviewed 108 documents about RTC and trilateral cooperation more broadly, conducted 45 KIIs with 51 key informants from seven key stakeholder groups, and received 72 complete responses to its online survey for a response rate of 33.3 percent. MAIN FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS A summary of the main findings, conclusions, and recommendations from the RTC midterm evaluation are presented in Table A1 below. ii Table A1: Summary of Findings, Conclusions, and Recommendations Findings Conclusions EQ1: To what extent are the beneficiary organizations applying the new knowledge transferred by SSC provider countries through the RTC project? Trilateral exchange activities were well planned and executed, met participants’ needs, imparted useful knowledge on timely/relevant topics by providers who were experts in their relevant fields, and generated recognized benefits for participants. RTC trilateral exchanges provided useful information and were of high quality according to measured quality criteria. Trilateral exchange participants increased their knowledge as a result of the exchanges. Knowledge acquired during trilateral exchanges was still considered useful six months after the exchange’s conclusion. Trilateral exchange activities were effective mechanisms for transferring new knowledge to participants who then disseminated this knowledge and applied it in ways that have generated multiple, tangible benefits. Trilateral exchange beneficiaries have applied the knowledge they acquired during the trilateral exchanges in numerous ways. Trilateral cooperation is, or can be, an effective mechanism for driving reform and innovation at the organizational, ministry, and policy levels. Key informants do not perceive there to be important gender issues related to trilateral exchanges. Gender is generally not an issue of high relevance to RTC stakeholders. The “right” people are being selected to participate in the exchanges, regardless of their sex. EQ1.1: What are the hindrances and barriers that beneficiaries face in applying this newly transferred knowledge? Trilateral exchange participants faced numerous impediments applying newly transferred knowledge, including funding and technological constraints, lack of higher￾level political support, and lack post-exchange follow-up. Lack of financing, advanced technologies, or capacities and lack of political support are important hinderances/barriers to applying newly transferred knowledge. However, the lack of post-exchange follow-up is a systemic limitation that potentially threatens the extent of gains that can be achieved from the trilateral exchanges as well as their sustainability. Sequential trilateral exchange activities offer the potential for consolidating gains and facilitating new knowledge application. Trilateral exchange participants do not face significant institutional opposition applying newly transferred knowledge; rather, the opposite appears to be true. To the extent hindrances/barriers exist, they are not insurmountable and have not prevented trilateral exchange participants from applying transferred knowledge. EQ2: In terms of the use of resources such as cost and staff time, timeliness to respond to needs, and other factors, what are the advantages and disadvantages of a centralized approach like RTC’s, and how do they compare to those of activities with trilateral exchanges as a component or sub￾component of a development project? The ET was unable to find cost information on trilateral exchanges as a component or sub-component of a development project to serve as a benchmark for answering EQ2. With this in mind, the per-trilateral exchange cost borne by Mission approaches $70,000, while the fixed per-exchange cost borne by the Mission exceeds the variable costs by about 50 percent at $41,066 to $27,223. RTC’s primary value-added contributions have been: 1) its funding support to IPs; 2) its professionally competent support in facilitating exchange logistics; 3) its incentivization of more trilateral exchanges than would have occurred otherwise, 4) the flexibility it has provided for doing trilateral exchanges to address unplanned-for priority issues where projects have ended or no project exists; and 5) its contribution to increasing the perceived legitimacy of trilateral cooperation relative to traditional development approaches. RTC’s logistical support has been excellent. Nonetheless, RTC is perceived by Mission staff as an expensive mechanism that delivers relatively low added value compared to its costs. How RTC’s costs compare to embedding trilateral cooperation as a project component or sub-component could not be determined due to a lack of benchmark data. With a per-exchange cost of nearly $70,000, consisting of 40 percent variable costs and 60 percent fixed costs (including RTC’s overhead allocations), the Mission can determine whether these costs are acceptable. RTC has evolved and is performing relatively few of the tasks envisioned in its original Statement of Work (SOW). RTC has since become more proactive in looking for opportunities to expand its services to the Mission and implementing partners (IPs). RTC is perceived as a relatively expensive mechanism for funding trilateral exchanges that could, in most but not all cases, be funded by individual projects, either with project or outsourced staff at a potentially lower cost. Mission and IP staff support an expanded role for RTC and identified several potential roles for RTC. This demand, however, remains largely latent. RTC is performing few of the roles envisioned for it in its SOW with the result that: 1) it is currently playing largely a logistical support role and is perceived as such by Mission and IP staff, and 2) it has a low perceived added value and cost-effectiveness. iii Findings Conclusions RTC faces constraints in playing a more expansive role; the Salvadoran Foundation for Economic and Social Development (FUSADES) and short-term technical assistance (STTA) offer opportunities for expanding RTC’s technical capacity, but with attendant opportunity costs. Mission and IP staff strongly favor the Mission continuing to support trilateral exchanges but are split as to whether this should be via RTC or another approach. If RTC is extended, they agree it should be assigned an expanded role, although how this expanded role would be financed given RTC’s budget structure is unclear. Mission and IP staff favor continued Mission support for trilateral exchanges and continuation of RTC, although with conditions. Transferring responsibility for trilateral exchanges to the projects may be less expensive but will likely result in fewer trilateral exchanges and greater logistical and other burdens borne by IPs, including smaller or less well-funded projects that may not be as capable of assuming their cost and administrative burden. Other results might be a loss in flexibility as to when and how to use trilateral exchanges where a project has ended or does not exist, a potential loss in planning and implementation efficiency and effectiveness, and the absence of a mechanism to collect, analyze, and disseminate information about exchange effectiveness and other lessons learned. Mission and IP staff are split as to whether projects can do their own trilateral exchange logistics. In lieu of RTC, Mission and IP staff further identified a number of potential exchange funding options. There exist various options for the Mission to continue supporting trilateral exchanges. Most of them, however, will require the Mission and projects to engage in strategic up￾front planning as to how and when to incorporate trilateral exchanges in Mission programming and project design and implementation; otherwise, the use of trilateral cooperation at the Mission is likely to be lower (and potentially much lower) than what its advocates desire. RTC has played an important role in incentivizing participation in trilateral exchanges beyond what would have occurred in its absence. Overall, RTC has been highly effective in creating demand for trilateral exchanges within the Mission and among IPs substantially beyond what would have occurred otherwise. Adopting an alternative approach for facilitating trilateral exchanges may lower their cost but with a number of important downsides relative to RTC, which include fewer trilateral exchanges, greater logistical burdens on IPs, and a loss in flexibility as to when and how trilateral exchanges can be used when projects have ended or do not exist. The supply of trilateral exchanges has, in turn, created higher demand for trilateral exchanges. Given financial constraints, RTC funding was critical to local beneficiaries’ trilateral exchange participation, particularly for the Government of El Salvador’s (GOES) ministries and offices. RTC is providing services that go beyond logistical support, and it is making efforts under its new management team to increase its role and value-added contribution. Many of these benefits or services go unnoticed by trilateral exchange participants, thus their perceptions of RTC’s value-added contribution may understate its actual Trilateral exchanges implemented under RTC have increased the perceived value of contribution. trilateral exchanges among Mission and IP staff. RTC trilateral exchanges compare favorably to other trilateral South-South exchanges in which trilateral exchange participants have participated in terms of facilitating new relationships and other quality criteria. RTC has played a role in many of the trilateral exchanges in addition to logistical support, such as vetting, travel, accommodations, etc. and includes trilateral exchange agenda setting, planning and preparation and (in certain cases) creating spaces and opportunities for trilateral exchange participants to meet, talk, and develop/strengthen relationships and facilitating discussions about post-exchange follow-up. iv Findings Conclusions EQ3: To what extent has the RTC project contributed to building partnerships among the USG, the Government(s) of the recipient country(ies), the Private Sector and Civil Society of El Salvador, Guatemala, and Honduras (Northern Triangle of Central America), and those SSC provider countries? Trilateral exchange participants strengthened and formed cooperative relationships with both home country and foreign country counterparts during trilateral exchanges. RTC trilateral exchanges have been an effective mechanism for strengthening or forging collaborative relationships with relevant home country, provider country, and other beneficiary country counterparts. In certain cases, trilateral exchange participation contributed to increased cooperation among USAID projects, among traditional adversaries in El Salvador, and among beneficiary counties. RTC has contributed important additionality in that the trilateral exchange participants would not have, for the large part, strengthened or formed these relationships without RTC. Relationships strengthened or formed during trilateral exchanges would not likely have occurred otherwise. Trilateral exchange participants have overwhelmingly leveraged these strengthened or new relationships to implement administrative, technological, policy, legal, etc. reforms Trilateral exchange participants have leveraged their relationships formed during within their organizations, among their partners, and within their countries. trilateral exchanges to further establish relationships and commitments, update laws and regulations, share information, strengthen their institution’s and partners’ capacity, and implement innovations. EQ3.1: How do the above recipient countries perceive the trilateral cooperation facilitated by the RTC project compared with traditional North-South Cooperation? Trilateral cooperation is a growing alternative to traditional North-South cooperation approaches, which offers a number of other potential comparative advantages, e.g., cost-effectiveness, contextual relevance, increased ownership, and more sustainable partnership networks. The literature affirms that trilateral cooperation is a legitimate complement or alternative to traditional North-South Cooperation. Within El Salvador, moreover, multilateral donor agencies are actively supporting trilateral cooperation with El Salvador and in the region, each using a different operational and financial model. SSC and trilateral cooperation are growing forms of donor cooperation in El Salvador and the region. There are a large number of operational and financial models for doing trilateral cooperation on which USAID/El Salvador can draw to inform its future plans in this area. There exists widespread agreement about the comparative advantages of trilateral cooperation relative to traditional North-South Cooperation. These comparative advantages, however, remain theoretical given the current lack of empirical evidence to support them. Survey respondents and key informants agree that trilateral cooperation is an effective development intervention strategy. There is room for USAID/El Salvador to increase its activity in working with other bilateral and multilateral donors to facilitate trilateral cooperation, including working more closely with the GOES Ministry of Foreign Affairs (MINEX) on identifying trilateral cooperation opportunities with other donor agencies. Mission and IP staff favor continued Mission support for trilateral exchanges, to participate in future trilateral exchanges or other trilateral exchanges and will continue investing in trilateral exchanges. Evaluation findings do not allow a conclusion about the relative effectiveness of trilateral cooperation. They do, however, allow the conclusion that RTC-facilitated trilateral exchanges have been effective, particularly in terms of learning how other countries in the region have addressed similar problems and in terms of forging relationships with provider country counterparts. Mission and IP staff value their participation in the RTC-facilitated trilateral exchanges. This is evidenced by their intention to participate in future RTC and non-RTC trilateral exchanges, invest their own funds in future exchanges, and by their near-unanimous advocacy for continued Mission support of trilateral cooperation. EQ3.2: To what extent have the beneficiary organizations established networking with their counterparts or private sector in the SSC provider countries that may lead to future collaboration beyond the life of the RTC project? Trilateral exchange participants have maintained the new relationships they formed during the trilateral exchanges for at least six months, and they are confident that they will maintain them over time. RTC trilateral exchanges have been an effective mechanism for strengthening and/or forming cooperative relationships. The evidence does not allow a conclusion that these relationships will be maintained over the long term; however, trilateral exchange participants are confident that they will be. v Findings Conclusions Factors that most contributed to the expected sustainability of relationships include high personal motivation, organizational support, and the value of counterpart knowledge. To the extent that these cooperative relationships are maintained over the long term, they are more likely to be maintained as ad hoc relationships in the form of a contingent asset on which relationship partners can draw on as needed, as opposed to an ongoing, Relationships strengthened or formed were more ad hoc to be drawn on as needed active collaboration. rather than ongoing; however, a number of relationships were of a more intensive, ongoing nature. Recommendations • USAID/El Salvador should continue to support trilateral cooperation at the Mission. The results from trilateral exchanges implemented under RTC have demonstrated that trilateral cooperation can be an effective instrument in the Mission’s toolbox of development programming. While the evaluation results did not allow a conclusion as to the relative effectiveness of trilateral cooperation compared to more traditional development approaches, they did allow a conclusion about its case-by-case effectiveness in terms of 1) transferring knowledge; 2) facilitating action based on transferred knowledge; and 3) strengthening and/or forming cooperative relationships with host country, provider country, or other beneficiary country counterparts. In addition to this, Mission staff and IPs overwhelmingly favor a continuation of Mission support for trilateral cooperation. • If USAID/El Salvador accepts the recommendation to continue supporting trilateral cooperation, it needs to decide which model to use. One option is to extend RTC into its option year. In light of cost-benefit considerations, however, the ET does not recommend extending RTC if its role would be solely or largely limited to logistical support, as has been the case to date. There exist lower-cost options to provide logistical support, such as tasking projects with the responsibility for funding and organizing trilateral exchanges, whether with own or out-sourced staff, contracting with a local firm for logistical support, extending RTC but staffing it entirely with local (or regional) staff, or creating a trilateral-support fund administered by the Program Office (or other entity) to which projects may apply for funding to support trilateral exchanges, etc. The annexes to this evaluation provide information and links to different models of trilateral cooperation. While assessing these different models was outside of the evaluation’s scope, the ET encourages the Mission to investigate these other models to see which have more or less relevance to the Mission’s situation. • If, on the other hand, the Mission elects to extend RTC, it should revise RTC’s SOW to include an expanded set of tasks that are more likely to generate sufficient added value to compensate for its cost, with the caveat that the tasks should be compatible with RTC’s existing technical capacity and budget allocations. Alternately, the Mission should revise RTC’s SOW to expand its technical capacity and/or expand or reallocate its budget allocations. • In deciding how to revise RTC’s SOW, the Mission should facilitate discussions between RTC and the technical teams in the Mission to identify the areas of highest demand and best fit with RTC’s technical and budget constraints. • Two overlapping and mutually reinforcing areas in which RTC might add particular value, and which also appear consistent with its technical and budget constraints, are to work with trilateral exchange participants to plan and facilitate actions, both pre and post-trilateral exchange, to 1) strengthen/form relationships between trilateral exchange participants and leverage those relationships over time and 2) implement actions/reforms drawing on the knowledge acquired during the trilateral exchanges. This recommendation addresses three concerns about the ultimate effectiveness of trilateral exchanges, and RTC’s role in facilitating them, both reflected in the EQs and voiced by key informants: 1) the lack of pre and/or post-exchange planning on how to leverage the trilateral exchange to implement reforms, innovations, etc. in beneficiary organizations and countries; 2) strengthening or forming cooperative relationships and maintaining them over time; and 3) increasing the cost-effectiveness of RTC’s service delivery model. • USAID/El Salvador should take the initiative to build closer relationships with other bilateral and multilateral donors in the Latin America and the Caribbean (LAC) region that are actively engaged in trilateral and South-South cooperation activities in El Salvador and the Northern Triangle, including the European Union (EU), Spanish Agency for International Cooperation for Development (AECID), German Corporation for International Cooperation (GIZ), Luxembourg Development Cooperation Agency (Lux-Dev), Mexican Agency for International Development Cooperation (AMEXCID), Brazilian Cooperation Agency (ABC), Peruvian International Cooperation Agency (APCI), Colombian Presidential Agency of International Cooperation (APC), and Chilean Agency for International Cooperation for Development (AGCID) in addition to strengthening the Mission’s relationship with MINEX, which is responsible for coordinating GOES’ trilateral cooperation efforts within the country. The Donor Coordinator in USAID’s Program Office can take the lead in developing these relationships and can serve as the trilateral cooperation point of contact in the case that USAID decides not to extend RTC. In that case, he/she can further develop a listing of countries and donor organizations that have a comparative advantage and can offer opportunities for USAID projects, Mission technical offices, or the Mission/Embassy to collaborate. 1 I.0 EVALUATION BACKGROUND, PURPOSE, AND AUDIENCE 1.1 EVALUATION BACKGROUND This evaluation report presents primary findings, conclusions, and recommendations from the midterm performance evaluation of the United States Agency for International Development in Central America and Mexico (USAID/CAM) Regional Trilateral Cooperation Support Project (RTC). RTC is implemented by International Business and Technical Consultants, Inc. (IBTCI), with a local subcontractor, the Salvadoran Foundation for Economic and Social Development (FUSADES). RTC runs from August 1, 2016-July 31, 2019, plus an Option Year. The project’s total funding is $5,620,555.46. RTC provides support to build trilateral cooperation1 between the United States Government (USG), represented by USAID, South-South cooperation (SSC)2 provider countries (Mexico, Colombia, Chile, Peru, Argentina, Brazil, Costa Rica, and Uruguay), and receiving countries (Guatemala, Honduras, El Salvador, Mexico, Belize, Costa Rica, Panama, and the Dominican Republic). The project’s overall aim is to provide technical and logistical assistance to USAID regional and bilateral Missions and the projects they support. Assistance is provided through trilateral exchanges that share and transfer technical expertise, technology, knowledge, and information on model systems, processes, and policies, all while building and strengthening meaningful, sustainable, intra-regional relationships. Trilateral exchanges include observational or study tours, facilitated workshops, conferences, and delegations of experts. 1.2 EVALUATION PURPOSE AND AUDIENCE The purpose of this midterm performance evaluation is to inform USAID/CAM on 1) the efficiency of resources used under this mechanism, 2) partnerships built, and 3) how recipient countries apply new knowledge. The Mission will use the evaluation results to both decide on the project’s option period and to inform the design of future trilateral cooperation mechanisms. [See Annex A for the evaluation Statement of Work (SOW).] The principal audience of this midterm performance evaluation is USAID/CAM, particularly the Regional Program Office, which manages RTC, and the Technical Offices of Economic Growth and Democracy and Governance. Other USAID audiences include the Program Offices of other Missions in the Central American region (mainly Donor Coordinators), the Bureau for Latin America and the Caribbean, and the Office of Donor Engagement in the Bureau for Policy, 1 Trilateral Cooperation involves partnerships between two or more developing countries—in this case from Latin America—supported by a developed country (or countries) or multilateral organization(s) to implement development cooperation programs and interventions. 2 SSC involves establishment of meaningful and sustainable relationships between Latin American governments, non￾governmental organizations, academics, the private sector, and other relevant stakeholders in order to enable the long-term exchange of resources, technology, and knowledge between developing countries, also known as countries of the Global South. SSC can take place on a bilateral, regional, sub-regional, or inter-regional basis. 2 Planning, and Learning. The implementing partner (IP), IBTCI, will carry out the remaining project implementation based on the evaluation results. Lessons learned and best practices identified in this evaluation will be disseminated to stakeholders and will be used to inform the design of future trilateral interventions at USAID/CAM or other Missions. 2.0 EVALUATION QUESTIONS The evaluation answers three primary evaluation questions (EQs) and three sub-EQs: 1. To what extent are the beneficiary organizations applying the new knowledge transferred by SSC provider countries through the RTC project? 1.1 What are the hindrances and barriers that beneficiaries face in applying this newly transferred knowledge? 2. In terms of the use of resources such as cost and staff time, timeliness to respond to needs, and other factors, what are the advantages and disadvantages of a centralized approach like RTC’s, and how do they compare to those of activities with trilateral exchanges as a component or sub-component of a development project? 3. To what extent has the RTC project contributed to building partnerships among the USG, the Government(s) of the recipient country(ies), the Private Sector and Civil Society of El Salvador, Guatemala, and Honduras (Northern Triangle of Central America), and those SSC provider countries? 3.1 How do the above recipient countries perceive the trilateral cooperation facilitated by the RTC project compared with traditional North-South Cooperation? 3.2 To what extent have the beneficiary organizations established networking with their counterparts or the private sector in the SSC provider countries that may lead to future collaboration beyond the life of the RTC project? 3.0 EVALUATION METHODOLOGY The information below describes the evaluation methodology, including data collection methods, sampling methods, methodological limitations, and lists the evaluation team (ET) members. Primary fieldwork/interviews for the evaluation took place during October 29-November 16, 2018, with additional interviews conducted after the conclusion of the fieldwork. The web-based survey (WBS) ran from November 12-November 30, 2018. 3.1 DATA COLLECTION METHODS The evaluation used a mixed-methods data collection design consisting of four qualitative and quantitative data collection methods, these were 1) document review, 2) key informant interviews (KIIs), 3) WBS, and 4) review of RTC performance and financial data. Each of these is described below. 3 Document Review The ET reviewed 108 documents pertaining to RTC in particular and to trilateral and South￾South cooperation more generally. RTC provided some documents and the ET found others during its own literature search. (See Annex B for a comprehensive bibliography.) Key Informant Interviews KIIs are semi-structured individual or small group (2-3 persons) interviews conducted with key informants who are representative of principal project stakeholder groups. As seen in Table 1, the ET conducted 45 KIIs with 51 individuals from seven stakeholder groups—RTC (including IBTCI and FUSADES), USAID/El Salvador, IPs, trilateral exchange beneficiaries from the Government of El Salvador (GOES), trilateral exchange beneficiaries from civil society organizations (CSOs) in El Salvador, trilateral exchange providers, and donor/cooperation agencies (bilateral and multilateral). 3 (See Annex C for a list of persons interviewed along with organizational affiliations and Annex D the KII discussion guides.) For key informants based in El Salvador, the ET conducted face-to-face interviews. For key informants based outside of El Salvador, the ET conducted the interviews either by phone or by Skype. Table 1: Number of KIIs and People Interviewed by Stakeholder Group Stakeholder Group Number of KIIs Number of People Interviewed RTC 5 8 USAID/El Salvador 10 11 IPs 6 6 GOES 10 12 CSOs 2 2 Trilateral exchange providers 6 6 Cooperation/donor agencies 6 6 Total 45 51 Web-Based Survey The ET designed and administered a WBS using the Zoho Survey platform.4 The survey included questions about respondents’ perceptions of trilateral cooperation in general and about RTC and RTC trilateral exchanges specifically. Other questions probed the extent to which knowledge is applied; the hindrances and barriers faced when applying their new knowledge; the extent to which the RTC trilateral exchanges helped build relationships with counterparts in receiving and providing countries; and the sustainability of those relationships. (See Annex E for a copy of the WBS questionnaire and Annex F for the raw survey results.) Review of RTC Performance and Financial Data The ET reviewed two sources of secondary data for the evaluation. One source was the project’s reported results for the six indicators found in the RTC Monitoring, Evaluation, and Learning (MEL) Plan. The second source, provided by IBTCI, was RTC financial data. 3 The six donor/cooperation agencies included in this group included two trilateral exchange participants: World Bank/International Finance Corporation and the Peruvian Agency for International Cooperation. 4 Zoho Survey is an online survey creation tool, which is part of a larger suite of cloud-based applications included in the Zoho Office Suite. 4 3.2 SAMPLING METHODS Web-Based Survey The ET emailed all 216 trilateral exchange participants an invitation to complete the survey. Of these, 72 completed the survey—an overall response rate of 33.3 percent, broken down by the respondent categories shown in Table 2. Table 2: Number of WBS Respondents by Respondent Category Respondent Category Number of Respondents Percentage of Respondents USAID/El Salvador 5 6.9% USAID/El Salvador-funded projects 7 9.7% Government ministry or agency 32 44.4% Private sector organization 12 16.7% CSO 3 4.3% Academic institution 7 9.7% Other organization 6 8.3% Total 72 100% Key Informant Interviews For the KIIs, the ET used a combination of purposive and random sampling methods. Purposive sampling is a non-probability sampling approach. Participants are selected based on the characteristics of the target population and the objective of the study. As part of its purposive sampling approach, IBTCI initially identified 47 “priority contacts” across 21 trilateral exchanges and recommended them to the ET to interview. Of these 47 priority contacts, 36 agreed to be interviewed. During the fieldwork, the ET identified a set of additional key informants, including people at other bilateral and multilateral donor/cooperation agencies also involved with trilateral cooperation, and it randomly selected one or more people participating in the trilateral exchanges not covered by IBTCI’s list of priority contacts. The ET requested interviews with each these as well. In the end, the ET interviewed each person it contacted who agreed to be interviewed. 3.3 EVALUATION TEAM The ET was comprised of three people—Dr. Gary Woller, Team Leader; Dr. Randal Joy Thompson, International Cooperation Specialist; and Gladys Soler, Research Associate. The Team Leader oversaw and coordinated all activities related to the evaluation and ensured the production and completion of all deliverables. The other two team members assisted the Team Leader in designing the research instruments, conducting the KIIs and drafting the final report. The USAID MEL Initiative provided logistical support for the evaluation planning and fieldwork. 3.4 METHODOLOGICAL LIMITATIONS The evaluation methodology involved the methodological limitations described below. Evaluation findings based on the subjective perceptions of key informants: Because time was short, the ET was not able to verify all perceptions and anecdotes cited by key informants. The ET sought to compensate for this limitation through triangulation achieved by generating data from other sources, including the document review and WBS. Difficulty assessing the sustainability of relationships formed as a result of trilateral exchange participation: To address sustainability, the ET asked each key informant about the 5 relationships he/she formed as a result of the trilateral exchange and then probed to determine the extent to which these relationships were likely to endure and in what form. The WBS also asked questions about the relationships formed and how likely it was that they would be sustainable. Potential for sampling bias: Three evaluation methodology features potentially introduced sampling biases. First, with 237 trilateral exchange participants, the evaluation budget and time constraints meant it was not possible for the ET to interview a fully representative sample of exchange participants. Second, the list of priority contacts IBTCI provided raised the possibility that the people selected would have more favorable perceptions of the project. Third, online surveys can suffer from low response rates, and respondents are randomly distributed across potential respondent groups or categories. To address the first and second potential sources of sampling bias, the ET combined purposive sampling with random sampling. The team did this by interviewing additional key informants RTC had not identified and by randomly identifying participants from the trilateral exchanges not included in RTC recommendations. In the end, the ET conducted KIIs with participants from 21 of the 33 trilateral exchanges (63.6 percent). To address the third source of potential bias, the ET sent all trilateral exchange participants who had not responded to the survey three invitations to complete the survey. Invitations were sent out a week apart. For the WBS, the ET sent invitations to each of the 11 Mission staff and seven IP staff who participated in the KIIs. All seven IP staff completed the survey. However, only five Mission staff responded to the survey, despite the fact that the ET sent them two reminder emails. 4.0 FINDINGS AND CONCLUSIONS This section presents the evaluation’s findings and conclusions. For the KII findings, the ET focused on the primary themes that emerged during the KIIs or on issues that, given the context, the ET judged particularly important. The findings are supported by quotes taken from the ET’s interview notes, which give an illustrative flavor of what key informants said. For more comprehensive findings on how trilateral exchange recipients have applied their new knowledge (EQ1), the relationships formed by trilateral exchange participants (EQ3), and the sustainability of those relationships (EQ 3.1), see Annex G of this report.5 5 Note that all KIIs did not necessarily cover all the same topics or cover them in the same depth, depending on the flow of the interview, the relevance of a particular question to the key informant, and on the key informants’ ability or willingness to answer certain questions. Thus, the reported number of KIIs describing how key informants answered certain questions does not always line up with the number of KIIs conducted. 6 4.1 EQ1: TO WHAT EXTENT ARE THE BENEFICIARY ORGANIZATIONS APPLYING THE NEW KNOWLEDGE TRANSFERRED BY SSC PROVIDER COUNTRIES THROUGH THE RTC PROJECT? 4.1.1 Findings Trilateral exchanges were well planned and executed, met participants’ needs, imparted useful knowledge on timely/relevant topics by experts in their relevant fields, and generated recognized benefits for participants. As seen in Table 3, over three￾quarters of respondents either “agreed” or “strongly agreed” that the knowledge transferred during the trilateral exchanges was useful. In excess of 70 percent likewise “agreed” or “strongly agreed” that trilateral exchanges addressed important and timely topics, used qualified experts, met participants’ needs, had well-organized logistics (RTC did an excellent job managing logistics), and required an appropriate amount of time. Table 3: Perceived Quality of USAID-Facilitated Trilateral Exchanges (N=72) Quality Dimension Strongly Disagree Disagree Neither Agree Nor Disagree Agree Strongly Agree Don’t Know Useful knowledge 6.9% 2.8% 0.0% 19.4% 56.9% 13.9% Topics important and timely 5.6% 1.4% 4.2% 11.1% 68.1% 9.7% Qualified experts 5.6% 2.8% 1.4% 23.6% 54.2% 12.5% Met needs of participants 5.6% 2.8% 5.6% 27.8% 51.4% 6.9% Logistics well organized 5.6% 4.2% 8.3% 25.0% 45.8% 11.1% Time appropriate 5.6% 2.8% 8.3% 36.1% 40.3% 6.9% Source: WBS Trilateral exchange participants increased their knowledge as a result of the exchanges. RTC’s MEL Plan, implemented by its sub-contractor FUSADES, tracked trilateral exchange participants’ knowledge acquisition by asking how much they knew about the relevant topics both prior to the exchange and after the exchange. Answers were tracked on an ascending 10-point scale. As seen in Figure 1, participants reported an increase from their pre-exchange score of 5.8 to a post-exchange score of 8.2, indicating that the trilateral exchanges had filled approximately one-half of the perceived knowledge gap. The pre and post-exchange scores for study tours and workshops/seminars followed a similar trend. Figure 1: Increase in Trilateral Exchange Participants’ Knowledge Source: RTC MEL Plans Knowledge acquired during trilateral exchanges was still considered useful six months after the exchanges’ conclusion. FUSADES returned to the same trilateral exchange 5.8 6.1 5.8 8.2 8.0 8.2 0 5 10 Study Tour Workshop/Seminar Total Before After 7 participants six months later and asked how useful they found the trilateral exchanges using an ascending 5-point scale in three areas: 1) their institution, 2) their area of work, and 3) their personal performance. Respondents gave a 4.8 rating in each area indicating a high level of perceived usefulness, even after the passage of time. Trilateral exchanges produced tangible benefits for participants. In terms of overall effectiveness, approximately one-third (72.2 percent) of survey respondents either “agreed” (27.8 percent) or “strongly agreed” (44.4 percent) that the trilateral exchanges produced tangible benefits for them or their organization (Figure 2). Figure 2: Level of Agreement Whether Trilateral Exchanges Produced Tangible Benefits (N=72) 5.6% 5.6% 5.6% 27.8% 44.4% 11.1% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% Strongly Disagree Disagree Neither Agree or Disagree Agree Strongly Agree Don’t Know Source: WBS Trilateral exchange beneficiaries have applied their new knowledge in numerous ways. As seen in Table 4, the majority of trilateral exchange beneficiaries shared the knowledge they acquired during the exchanges with others both inside (86.0 percent) and outside (60 percent) their organization; implemented the new knowledge in their job (78 percent); or promoted changes within their organization (56 percent). Notably, almost no beneficiaries indicated being unwilling or unable to apply the new knowledge in any way. Those who have yet to apply their new knowledge noted that they either plan to apply it or may apply it in the future. Finally, 56.0 percent and 38.0 percent of beneficiaries, respectively, have solicited help from, or provided help to, people they met during the trilateral exchanges. Table 4: How Trilateral Exchange Beneficiaries Have Applied New Knowledge (N=50) Hinderance/Barrier No And Won’t No But Might No But Will Yes Don’t Know N/A Shared knowledge in my organization 0.0% 0.0% 4.0% 86.0% 10.0% 0.0% Implemented knowledge in my job 0.0% 2.0% 10.0% 78.0% 8.0% 2.0% Maintained contact with people I met 0.0% 16.0% 12.0% 66.0% 6.0% 0.0% Shared knowledge outside my organization 0.0% 20.0% 10.0% 60.0% 8.0% 2.0% Promoted changes in my organization 2.0% 8.0% 16.0% 56.0% 10.0% 8.0% Solicited help from people I met 2.0% 18.0% 10.0% 56.0% 10.0% 4.0% Promoted political changes 2.0% 24.0% 16.0% 34.0% 4.0% 20.0% Provided help to people I met 0.0% 28.0% 14.0% 38.0% 8.0% 12.0% Source: WBS Turning now to qualitative findings, in all but one of the beneficiary KIIs, key informants described specifically how they or their organization implemented the knowledge gained during the trilateral 8 exchange. (See Annex G for a summary of the specific ways the key informants or their organizations, applied their new knowledge post-exchange, as described to the ET during KIIs.) Text Box 1 The Ministry of Finance (MOF) leveraged the knowledge it acquired from participating in the Tax Collection and Anti-Corruption trilateral exchange, part of the Fiscal Policy trilateral effort implemented with the Fiscal Policy and Expenditure Management Program, to execute a number of administrative and legal reforms. Its Investigative Intelligence Unit and Transparency and Anti￾Corruption Unit refined the investigations/research and verification processes with better procedural guarantees for investigating growth of capital/assets by public servants. The ministry further improved its risk management process by establishing more assertive penalties for public officials’ criminal/corruptive behavior, redefined and reclassified actions subjected to penalization, developed more efficient tax collection processes (based on a platform shared by Colombia during the exchange) and is working with its Communications Unit to establish an electronic mechanism for submitting complaints through a system that protects complainants’ integrity. Finally, the MOF revised a proposal for the new National Tax Strategy, currently awaiting congressional approval. Key informants do not perceive important gender issues related to trilateral exchanges. Key informants routinely struggled to identify gender issues related to the trilateral exchanges in which they participated. While key informants nearly uniformly agreed that it is important or desirable for women to have equal opportunity to participate in trilateral exchanges, ensuring this outcome was generally not a high priority. Rather, they prioritized selecting participants who held key positions of authority, would be responsible for implementing the acquired knowledge (beneficiary organizations), or who possessed the relevant expertise (provider organizations). Women included in these categories felt participation was encouraged, and no barriers to participation existed. However, when women were not included in these categories, it was not a reason for concern. This point is reflected in the following quote, “There are no gender issues. We tried to bring women in key positions and will take this into consideration, but this is not a big issue with RTC.” – USAID/El Salvador By the end of Fiscal Year (FY) 2018, RTC reported that it had assisted 190 men, or 86 percent of its life of project (LOP) target of 221, and 98 women, or 44 percent of its LOP target of 221. Thus, while RTC is on track to meet its performance target related to male trilateral exchange participation, it is lagging behind its performance target for female participation. RTC has no control over the gender mix of trilateral exchange participants who are selected by the participating organizations without RTC’s input. Furthermore, none of the key informants interviewed expressed concern about this gender imbalance. Instead, the consensus view was that the “right” people were participating in the trilateral exchanges. “I do not see any gender issues that came up. Nothing indicated that women could not participate in these events or were being excluded in anyway.” – USAID/El Salvador “As for gender, we have a goal of 40 percent female or marginal group participation. We cannot control who went to meeting, but when setting up different activities, and where it is possible, we try to ensure that there is a mixture of both men and women and other groups were relevant. – USAID IP 9 4.1.2 Conclusions • RTC trilateral exchanges provided useful information and were of high quality according to measured quality criteria. • Trilateral exchanges were effective mechanisms for transferring knowledge to participants who disseminated this knowledge both inside and outside their organizations and applied it to generate tangible benefits for their organizations. • Trilateral cooperation is, or can be, an effective mechanism for driving reform and innovation at the organizational, ministry, and policy levels. • Gender is not a priority criterion in selecting trilateral exchange participants, nor is it generally an issue of high relevance to RTC stakeholders. The “right” people are being selected to participate in trilateral exchanges, regardless of their sex. 4.2 EQ1.1: WHAT ARE THE HINDERANCES AND BARRIERS THAT BENEFICIARIES FACE IN APPLYING THIS NEWLY TRANSFERRED KNOWLEDGE? 4.2.1 Findings Lack of financing, advanced technologies, limited capacities, low political support, and a lack of time are the primary hinderances/barriers to applying new knowledge. The WBS found that the most important hindrances or barriers facing trilateral exchange beneficiaries when applying the acquired knowledge was a lack of funding. This was followed at a distance by technologies or capacities too advanced to apply, a lack of political support from higher-level government officials, new knowledge not relevant to the country context, and a lack of time to apply the new knowledge (see Table 5). Table 5: Hindrances and Barriers in Applying New Knowledge for Trilateral Exchange Beneficiaries (N=50) Hinderance/Barrier 1st Most Important 2nd Most Important 3rd Most Important Total Lack of financing 46.0% 20.0% 10.0% 76.0% Requires technologies or capacities too advanced 20.0% 16.0% 10.0% 46.0% Lack of political support 12.0% 8.0% 24.0% 44.0% Knowledge not relevant to country 6.0% 16.0% 12.0% 34.0% Lack of time 8.0% 12.0% 10.0% 30.0% Not sure how/need more capacity development 4.0% 6.0% 10.0% 20.0% Lack of institutional support from upper management 2.0% 4.0% 4.0% 10.0% Lack of support from immediate supervisor 0.0% 2.0% 2.0% 4.0% Knowledge not relevant to job 0.0% 2.0% 0.0% 2.0% Lack of personal motivation 0.0% 0.0% 0.0% 0.0% Knowledge not relevant to organization 0.0% 0.0% 0.0% 0.0% Source: WBS The KII findings corroborate the WBS findings. Where recipients struggled to apply what they learned during the trilateral exchange, they were most likely to cite one or more top four factors as the primary hindrance/barrier. The lack of financing and/or the inability to replicate the relevant technologies were particularly salient hinderance/barriers among recipients from GOES ministries. In turn, this adversely affected the ministries’ ability to apply technologies or practices learned during the trilateral exchanges. With regards to political obstacles, in the Integrity 10 Evaluation and Control Center, Civil Service Reform, and Single Taxpayer trilateral exchanges, the exchange contributed to drafting or revising reform bills that remained pending at the time of the KII, either in the President’s Office or the National Assembly, and in one case—Transfer Control Pricing trilateral exchange—GOES declared the proposed transfer pricing standards unconstitutional. Lack of post-exchange follow-up is another important hindrance/barrier to applying new knowledge. In KIIs, the only factor hindering post-exchange implementation mentioned frequently was the lack of post-exchange follow-up. Post-exchange follow-up was defined as organized efforts to reinforce learning acquired during the trilateral exchange, plan and coordinate response activities, or strengthen ties between exchange participants. Lack of follow through up to strengthen participant relationships, of course, further threatens the sustainability of relationships developed during the exchanges. While certain trilateral efforts involved follow-up sequential trilateral exchanges that built explicitly on each other (e.g., Ruta N and Civil Service Reform), as a rule, trilateral exchanges implemented under RTC have not included follow-up planning or other activities to consolidate gains achieved during the exchange. (Note: That is not a criticism of RTC, as it is not responsible for designing the trilateral exchanges nor has it been tasked, in practice, with facilitating follow￾up activities among exchange participants.) Ten (10) key informants specifically noted that more could have been achieved from their trilateral exchanges had they been followed, or preceded, by activities to plan and coordinate the post-exchange response. “I think that better follow-up is missing…follow-up of what we should be addressing after the experience we had. I perceived scattered efforts, when we should be aligned. All players…we should have a plan.” – CSO “If they do not produce a final product, they are partially successful…and if there is no action or implementation, knowledge stays in people’s heads. Of course, trilateral exchanges change minds and trigger improvements, but we have to assure the sustainability of the exchanges. – USAID IP “There have never been any follow-up meetings, [projects] have not set them up proactively or taken this to another level.” – USAID/El Salvador “We are not convinced that the mechanism offers sufficient value from these sorts of trips. The trips suffer from a lack of follow-up, which is their greatest weakness. Trilateral activities are successful in terms of the right people who go, and they are learning important things. But getting back, it’s hard to move beyond enthusiasm; they’re having difficulty getting from the trip to activation of what they want to implement.” – USAID/El Salvador Sequential trilateral exchanges offer the potential for consolidating gains and facilitating new knowledge application. Four key informants argued that the “ideal” trilateral exchange should be structured as a series of sequential activities that build on each other. The reasoning was that this would increase the likelihood that the exchange produces tangible benefits for recipients and long-term cooperative relationships among participants. One key informant from a CSO likened trilateral exchanges to trade shows. According to him/her, “With a single trade show, you don’t end up making many sales, but you make contacts. Over time after the second and third trade shows you begin to make sales because of the contact you have previously established.” 11 Another USAID/El Salvador key informant remarked that, while a single trilateral exchange can be “transformative” for those who participate, on their return they are but “one voice in a chorus,” and “they need to bring other people on board at some point.” Two of the trilateral efforts that involved sequential trilateral exchanges (Ruta N and Civil Service Reform) boasted of significant post-exchange results. Although these are not evidence of necessity (numerous other trilateral exchanges also produced significant post-exchange results), they illustrate the potential of sequential trilateral exchanges. As noted by a USAID IP participating in Ruta N, “It was the return visits to El Salvador that really solidified plans to actually implement the different activities that were taking place in Colombia. It led to greater institutional commitment to do it that was not present after the visit to Colombia.” 4.2.2 Conclusions • Lack of financing, advanced technologies, and capacities and political support from higher￾level government officials are important hinderances/barriers to applying newly transferred knowledge. However, the lack of post-exchange follow-up is a systemic limitation that potentially threatens both the gains that can be achieved from the trilateral exchanges and their sustainability. • Trilateral exchange participants do not face significant institutional opposition within their own organizations from their immediate supervisors or upper management in applying newly transferred knowledge. In fact, the opposite appears to be true. • To the extent hindrances/barriers exist, they are not insurmountable and have not prevented trilateral exchange participants from successfully applying newly transferred knowledge. 4.3 EQ2: IN TERMS OF THE USE OF RESOURCES—SUCH AS COST AND STAFF TIME, TIMELINESS TO RESPOND TO NEEDS, AND OTHER FACTORS— WHAT ARE THE ADVANTAGES AND DISADVANTAGES OF A CENTRALIZED APPROACH LIKE RTC’S, AND HOW TO THEY COMPARE TO THOSE OF OTHER ACTIVITIES WITH TRILATERAL EXCHANGES AS A COMPONENT OR SUB-COMPONENT OF A DEVELOPMENT PROJECT? 4.3.1 Findings The per-exchange cost borne by the Mission approaches $70,000, while the fixed per￾exchange cost borne by the Mission exceeds its variable costs by about 50 percent at $41,066 to $27,223. The ET was unable to find cost information on trilateral exchanges as a component or sub-component of a development project to serve as a benchmark for answering EQ2. However, using financial data provided by IBTCI, the ET was able to calculate the cost per exchange for a variety of cost scenarios covering project inception through FY2 (ending August 31, 2018) and 31 of the 33 trilateral exchanges examined in this evaluation.6 Using this data, the ET calculated the per-exchange costs shown in Table 6. 6 Excluded were the two trilateral exchanges completed after August 31: Roadmap III, which was implemented in September 2018, and Civil Service Reform Part II, which was implemented in October 2018. 12 Table 6: Costs per Trilateral Exchange Under Alternative Cost Scenarios Costs per Trilateral Exchange Amount Total USAID costs per exchange $68,290 Total costs per exchange with counterpart in-kind contributions $78,343 Travel, transportation, and per diem costs per exchange $18,783 Travel, transportation, and per diem costs w/ in-kind contributions per exchange $28,836 Variable costs per exchange $27,223 Variable costs with in-kind contributions per exchange $37,276 Fixed costs per exchange $41,066 The second and third rows in Table 6 show the total project cost ($2,116,982) per exchange and the total project cost plus the value of in-kind contributions ($2,428,630) per exchange.7 The fourth and fifth rows show the project travel, transportation, and per diem costs ($582,271) per exchange and the project, transportation, and per diem costs plus provider country in-kind contributions ($893,919) per exchange. The sixth and seventh rows show the total project variable costs per exchange ($843,925) and the total project variable costs plus in-kind contributions ($1,155,57) per exchange. Variable costs include travel, transportation, and per diem costs plus the portion of RTC staff time and salaries dedicated to planning and implementing the exchanges. Finally, row eight shows the total project fixed costs ($1,273,057) per exchange, which include costs for short-term technical assistance (STTA), allowances, equipment, and supplies, subcontractor costs, the portion of staff time and salaries not dedicated to trilateral exchanges, and other direct and indirect costs.8 As seen in Table 6, the total cost per-exchange borne by USAID/El Salvador was $68,290 and $78,343 if counterpart in-kind contributions are counted. Of particular interest in Table 6 is the comparison of per-exchange variable and fixed costs, the latter of which includes RTC’s overhead costs—a particular source of concern among Mission staff. RTC’s fixed per-exchange costs, borne by USAID/El Salvador, are 50 percent larger than its variable per-exchange costs. If counterpart in-kind contributions are included, then fixed per-exchange costs exceed variable exchange costs by about 10 percent. RTC is perceived as an expensive mechanism that delivers relatively low added value. Eight of the 11 Mission staff interviewed noted RTC’s perceived high cost structure as its primary weakness. In particular, they noted that RTC is structured as a project, complete with a MEL Plan and a project manager and is staffed by an international contractor, all of which imposed high overhead costs. At the time RTC was conceived and approved, moreover, “we were flush with resources to address CDCS [Country Development Cooperation Strategy] challenges…and we had 7 In-kind contributions include in-kind contributions (e.g., staff time) apportioned by provider country institutions and other support institutions, including both bilateral and multilateral donor/cooperation agencies. For the 31 trilateral exchanges covered by this financial analysis, the value of in-kind contributions totaled $311,648. The value of in-kind contributions varies by trilateral exchange and is thus counted as a variable cost in the ET’s financial calculations. 8 The calculations shown in Table 6 are intended as approximations, not precise calculations. The financial data provided to the ET did not always differentiate between fixed and variable costs, such that certain line-item costs included both fixed and variable components, including staff salaries, equipment and supplies, and sub-contractor (FUSADES) costs. In the case of staff salaries, RTC roughly estimated that 50 percent of the Chief of Party’s (COP) time was devoted to planning and implementing trilateral exchanges, while 70 percent of RTC’s three local staff members’ time, on average, was devoted to planning and implementing trilateral exchanges. 13 money” but “money is getting tight again” (USAID/El Salvador). Had the funding situation remained constant from project inception, key informants indicated that they would be less concerned about RTC’s costs. However, given that the funding situation has worsened, the project’s costs have come into full relief, which key informants generally find hard to justify given their perception that RTC is a largely logistical support project. “What we are buying is the convenience to do logistical planning. Most implementing partners have robust staff and could do it…The project is an expensive convenience.” – USAID/El Salvador “The cost is very expensive, and it is an expensive way of doing business. You are paying double overhead for both a contractor and activity, not precisely double, but the idea is you are paying more than it would cost just one of them to do the trilateral activity. In this case, you are paying overhead for the mechanism, and you’re also paying for the time that the project devotes to it.” – USAID/El Salvador “IBTCI’s overhead was very hard…too high in comparison to what IBTCI really did. It is a huge amount. I have been very involved, and it is too much money.” – USAID/El Salvador The above KIIs findings are reflected in the WBS findings, in which seven Mission staff said that RTC costs were too high, regardless of or relative to its benefits (see Figure 3). In contrast, IPs expressed a more optimistic perception of RTC’s cost-benefit relationships; three of seven respondents said that the costs were appropriate to the benefits, while one said its costs were too high relative to its benefits. (IP staff were much less likely either to know RTC’s cost structure than Mission staff.) Figure 3: Perceived Cost-Benefit of RTC by Mission and IP Staff (N=12) 3 1 0 1 0 0 1 0 3 3 3 2 0 4 3 0 1 2 3 4 5 Cost too high regardless of benefits Cost too high relative to benefits Cost appropriate regardless of benefits Cost appropriate relative to benefits Don't know USAID USAID IP Total Source: WBS The challenge RTC faces in demonstrating its value-added is captured by the following observation made by a member of the Mission technical team: “I thought that the original Statement of Work for RTC included identifying opportunities for exchanges and matching the needs of different USAID implementers and counterparts to expertise in other countries. However, RTC has not done this, instead it is large travel agency. It is very cooperative and very responsive, it executes its logistical function very well; however, all of this could be done by the IP, typically at a much lower cost. Its comparative advantage was supposedly to help develop opportunities with countries. It started doing this with its assessment and inventory of what other countries could provide. It was supposed to take this, and design activities based on expertise in other countries. But this did not happen.” 14 RTC has evolved such that it is performing relatively few of the tasks envisioned in its SOW, with adverse implication for its added value. RTC’s low perceived value-added among Mission staff is a function of its perception as a logistical support project (or as facetiously referred to by one Mission contact, “the Carlson Wagonlit contract”). This was not, however, the intention of RTC’s designers, as described in its original SOW. The potential, but as yet unrealized, benefits of the RTC approach are enumerated in detail in the SOW and consist of a series of tasks falling under the following concepts: 1) assess the enabling environment to implement trilateral cooperation among selected countries; 2) provide logistical support to trilateral activities; and 3) coordinate and help implement trilateral program activities. As RTC has evolved, however, it has performed relatively few of the tasks described in its SOW. Performing these tasks would presumably add significant value above the limited tasks RTC is currently performing. RTC did produce the assessment requested in its SOW (in April 2017). The assessment included a number of recommendations delineating ways in which RTC could add value by undertaking tasks to support the Mission in identifying, planning, implementing, and assessing trilateral exchanges. Nonetheless, few Mission and IP staff were aware of the assessment and fewer still had read it. The ET was unable to find any official or unofficial response to the assessment which, in the end, has exerted no influence over RTC’s direction or the tasks it performs. As stated by one USAID/El Salvador key informant, “The assessment was done and delivered, and then everyone walked away.” In terms of coordination and implementation assistance, the SOW enumerated a long list of potential tasks that RTC could perform (pp. 12-13). These tasks included post-exchange follow￾up, organizing complementary (extracurricular) activities to facilitate dialogue and coordination, and to stimulate the exchange of ideas; analyzing, identifying, and recommending trilateral exchange options; researching, developing, and recommending concepts for potential trilateral exchanges; assessing operational procedures and practices that can be piloted in El Salvador and other countries; or subcontracting with providers to deliver training and formal education activities. To date, however, RTC has not performed any of these roles. The point of enumerating the contents of the SOW here is to highlight the fact that RTC was designed to be a traditional human and institutional capacity development project and, thus, envisioned as playing a significantly larger role than it is currently playing. Had this vision come to fruition, RTC would have delivered a significantly enhanced added value and potentially justified its unique administrative structure and attendant costs. As the situation currently stands, however, RTC has played a largely logistics support role and is thus seen in this light by Mission and IP staff. RTC’s limited role is a function of, among other things, a missing champion, indifferent and ineffective promotion (initially), and limited technical expertise. In interviews with key informants, the ET sought to determine how this outcome came about. Three possible, mutually reinforcing answers stood out. First, the people responsible for promoting and designing RTC left the Mission prior to or early in RTC’s implementation, leaving RTC without a “champion” advocating for a more expansive role. Second, while the SOW called for proactive engagement by RTC to “analyze project objectives and assess, identify and recommend trilateral intervention options to address specific development problems” (p. 12), RTC instead became a demand-driven mechanism that has largely been limited to responding to expressed demand from 15 the Mission and Embassy in the form of trilateral exchange concept notes. This outcome, in turn, has been driven by a reluctance of Mission staff to ask RTC to take on a more expansive role given that RTC is thinly staffed and lacks technical background or the expertise to weigh in knowledgably on technical issues. Third, the original RTC management team did not proactively promote the mechanism or position it to play this more expansive role. While it produced the assessment requested in the SOW that advocated for a more expanded role, there was little and/or ineffective follow-up to the assessment or promotion of this expanded role. During the project’s second year, IBTCI attempted to engage the Mission in a workshop geared at thinking more strategically about how to use RTC; however, the workshop never materialized. Of the above explanations, the first two were outside of RTC’s control. While the third was a function of early RTC management policies, the current management team is working to reverse this outcome through more proactive engagement with Mission and IP staff. “Implementing partners and counterparts possess all of the technical knowledge, they are able to identify the issues and experts in their field. It is hard for RTC to do this. Were RTC to attempt this, it may create conflicts with the technical experts who actually know the field and the other experts better than RTC staff, who are not experts in the relevant fields.” – USAID/El Salvador “These are very high-level officials, who is going to reach them? RTC is not in a position to make contact with such high-level officials...” – USAID IP Latent demand exists for a more expansive RTC role, but this has yet to be translated into effective demand. 9 To be fair, other Mission and IP staff did express a desire for RTC to play a more active technical role, including identifying trilateral cooperation opportunities and cooperation counterparts in other countries. This is reflected in the recommendation expressed by a USAID IP that RTC expand its services “because this is a large project with six components, and it can be difficult to establish contacts, find opportunities and so forth. A project such as RTC could play an important role in this.” This demand, however, has yet to manifest itself in meaningful effective demand. In response, RTC’s current management team is working to become more proactive in providing an expanded set of services that would offer greater added value to the Mission and IPs. While RTC has made some progress in this area—according to one Mission key informant, “…now I think they [RTC] are taking a more active role…they are more interested in identifying places and people…this is a 180 degree change from before”—its efforts remain constrained by a lack of effective demand. Thus, its perceived value-added contribution continues to lie predominantly in the area of logistical support. RTC has become more proactive in looking for opportunities to expand its services to the Mission and IPs. The new RTC management team has been more proactive in meeting with Mission and IP staff in an attempt to understand their objectives and needs and to explore opportunities for collaboration beyond logistical support. As documented in Annex G and illustrated in Text Box 2, below, RTC has played a role in many of the trilateral exchanges that 9 Latent demand refers to existing but unexpressed or unrealized demand. Effective demand refers to expressed and realized demand. 16 goes well beyond mere logistical support by providing vetting, travel, accommodations, trilateral exchange agenda setting, planning and preparation. In certain cases, RTC has gone so far as to create spaces and opportunities for exchange participants to meet, talk, and develop/strengthen relationships. RTC has also facilitated discussions about post-exchange follow-up. Notwithstanding, RTC’s contribution remains primarily in the area of logistical support, while key informants were unable to identify ways in which RTC had created value beyond logistical support, and no key informant could cite anything RTC had done to facilitate relationships among trilateral exchange participants. Text Box 2 During the One-Stop-Shop Customs Border Crossings trilateral exchange, part of the Regional Trade Facilitation trilateral effort, RTC arranged a trip to Tijuana for exchange participants (customs directors, customs operations officers, and customs risk management officers). The group visited a border facility that served as a one-stop-shop for traders sending goods across the border. Traders needed to complete only one set of paperwork and undergo one physical inspection instead of spending hours going through two different countries’ processes. While there, participants viewed examples of close cooperation between the United States (U.S.) and Mexico customs officers and discussed personal experiences with inefficient and time-consuming customs procedures. Motivated by this experience, the participating customs directors met in a closed-door session during the exchange to discuss cooperation opportunities. After the trilateral exchange, the customs directors expressed their gratitude for the exchange by stating that it was one of the most fruitful coordinating conversations in which they had participated and, although they had known each other previously, the exchange strengthened their relationship and opened opportunities for greater cooperation down the road. Mission and IP staff support an expanded role for RTC. The WBS results in Table 7 show that both USAID and IP staff, with one exception in each case, either “agreed” or “strongly agreed” that RTC should take on an expanded role beyond providing logistical support. The survey results are consistent with what Mission and IP staff told the ET during KIIs. Of the 11 Mission staff interviewed, seven favored an expanded RTC role and four did not. Among IPs interviewed, five favored an expanded RTC role, while one was indifferent. Table 7: Mission and IP Support for Expanded RTC Role (N=12) Quality Dimension Strongly Disagree Disagree Neither Agree Nor Disagree Agree Strongly Agree Don’t Know USAID 0 1 0 4 2 1 IP 0 0 1 1 5 0 Total 0 1 1 2 7 1 Source: WBS Mission and IP staff identified several potential roles for RTC. In KIIs, Mission and IP staff offered more insights into what they saw as potential expanded roles for RTC. Their suggestions include those listed below (offered by way of examples not endorsement). Note that several of the suggestions include tasks envisioned in the RTC SOW, and some suggestions offered by key informants were general in nature and lacked implementation specifics. • Advise the Mission and IPs on trilateral exchange opportunities and the existence and availability of counterpart experts in relevant fields. 17 • Advise the Mission and IPs on how trilateral exchanges fit strategically into their programming and projects and how exchanges can help them achieve their objectives. • Help the Mission and IPs identify common or overlapping trilateral initiatives that address the objectives or needs of two or more projects. • Help strategically plan and coordinate trilateral exchanges in support of Mission strategy and objectives. • Help facilitate virtual trilateral “exchanges” via video conferencing or other information technologies. • Do knowledge management by capturing and disseminating lessons learned, best practices, innovative approaches, etc. • Provide support services during trilateral exchange implementation, e.g., note taking. • Facilitate planning prior to trilateral exchanges so as to maximize learning. • Facilitate and post-exchange follow-up activities. • Maintain a database of contacts made during trilateral exchanges and help facilitate relationship maintenance among contacts via, for example, a Facebook page, alumni network or events, blog site, or community forum. RTC faces constraints in playing a more expansive role. A noted constraint to RTC undertaking a more expansive role is its thin staffing structure, which consists of a project manager, logistics manager, project administrator, and accountant. The current staffing structure presents two challenges to RTC. First, there are no technical specialists in areas covered by current Mission programming, plus the current Mission portfolio covers multiple technical areas that would be difficult for RTC to address. Second, given that the primary criticism of RTC is its high overhead burden, increasing RTC staff would increase this burden, although would also provide higher added value. FUSADES and STTA offer opportunities for expanding RTC’s technical capacity, although with attendant opportunity costs. One possible way to address RTC’s technical constraints is to expand FUSADES’ current role to include one or more of the above functions. FUSADES was originally envisioned to play a larger role, including research on trilateral exchange opportunities and counterparts, and possibly assuming implementation of the project from IBTCI during the project’s option year. Yet the Mission subsequently reduced FUSADES’ yearly budget from $100,000 to $25,000 and limited its role to monitoring and evaluation (M&E). This was due to, among other things, its perceived low added value relative to its costs. Were FUSADES to assume a larger role, however, its budget would need to be increased commensurately, requiring reallocation from other project functions. Another possibility is to draw on the line item in RTC’s budget for STTA to hire outside experts (international, regional, or local) to provide the technical assistance (TA) required in an expanded project role. RTC’s original budget included a yearly line item for STTA totaling $67,208 over three years—$44,808 in FY1, $14,935 in FY2, $7,468 in FY3, and $0 in FY4—presumably to provide the services envisioned in the RTC SOW. So far, however, it has spent only $28,232 on STTA, all during FY1 for a consultant to work on the trilateral assessment requested in RTC’s SOW. Nonetheless, whether the current STTA budget allocation is sufficient to provide meaningful TA is a legitimate question that would need to be addressed. As with expanding FUSADES’ role, reallocating budget back to STTA would involve an opportunity cost by decreasing the budget elsewhere. 18 Mission and IP staff favor continued Mission support for trilateral exchanges and continuation of RTC, although with conditions. As seen in Table 8, both Mission and IP staff responding to the WBS expressed strong support for the Mission to continue supporting trilateral cooperation in the future. In KIIs, both Mission and IP staff likewise unanimously advocated for continued Mission support of trilateral cooperation. In the WBS, Mission staff expressed strong support for continuing RTC, albeit with conditions. A single staff member recommended ending RTC once its current contract period ends, due to its high cost. However, KII results presented a different picture in that five of the Mission staff interviewed supported continuing RTC, while the remaining six supported its termination at the conclusion of its existing contract.10 The main reason given for terminating RTC was its perceived high cost and the belief that projects had internal capacity to conduct trilateral exchanges, either by outsourcing logistics or keeping them in-house. Finally, among IP staff, two recommended continuing RTC, with conditions, while five recommended continuing RTC without conditions. In KIIs, IP staff uniformly supported continuing RTC. Table 8: Mission and IP Support for Future Trilateral Cooperation and Continuing RTC (N=12) Quality Dimension Continue Supporting Trilateral Cooperation Continue RTC Yes No Don’t Know No Yes w/ Conditions Yes USAID 4 0 1 1 4 0 IP 7 0 0 0 2 5 Total 11 0 1 0 6 5 Source: WBS For the Mission staff who recommended continuing RTC with conditions, one respondent simply recommended revising the project to reduce its costs. Another recommended that RTC offer technical services but reduce costs by eliminating the project manager position. A third recommended that RTC provide more technical services, including identifying opportunities and counterparts, creating more robust trilateral exchange agendas, facilitating discussions among exchange participants, and facilitating exchange follow-up plans with action steps. IP recommendations included providing technical support; identifying cooperation partners; disseminating best practices, lessons learned, and other information of interest; and improving logistical support and identifying trilateral opportunities and contacts. Mission and IP staff are split as to whether projects can do their own trilateral exchange logistics. For those key informants supporting RTC’s termination, the majority opinion was that projects had the capacity to undertake trilateral exchanges on their own, either using project staff or outsourcing the logistics to external contractors. Eight Mission and three IP staff felt that projects could undertake trilateral exchanges on their own, while three each of 10 While KII and survey findings involving Mission staff consistently supported each other, the numbers reported in each do not always line up, for two reasons. First, several Mission staff who participated in the KIIs did not complete the survey. Second, depending on whether they completed the survey or participated in the KII first, the period of reflection between them may have produced different, or more nuanced, responses. 19 Mission and IP staff felt that projects, and particularly smaller or less well-funded projects, were not capable of assuming the cost and administrative burden of doing trilateral exchanges. “Larger projects would likely say that they can do it on their own, that it is not much of a heavy lift to organize the logistics and agenda and so forth for the trilateral exchanges on their own.” – USAID/El Salvador “Most implementing partners have robust staff and could do it. But the number of activities would be much lower than without the project.” – USAID/El Salvador “In the absence of RTC, they [projects] could rely on the implementers to figure out the logistics, but they could do it either with existing staff or via outsourcing.” – USAID/El Salvador “In the absence of RTC, you have to build it into the project, but it requires a lot of time and money from contactors and takes away from what they should be doing. If you want them to focus on technical aspects, it may be difficult to have trilateral activities built in.” – USAID/El Salvador Mission and IP staff identified a number of potential trilateral exchange funding options absent RTC. Key informants identified a number of possible project funding sources outside of RTC, such as the project training budget, budget allocated for work with sectoral associations, or budget allocated for knowledge management. One key informant suggested a trilateral fund administered by the Mission Program Office that projects could submit proposals to for trilateral exchanges. Other possibilities raised in KIIs included replacing the current expat project manager with a local project manager or replacing RTC with a local logistical support contract awarded to a local contractor—both suggestions were believed to entail lower overhead costs than the current funding model. This recommendation reflects a point raised in four KIIs that a full-blown “project” (complete with project manager, MEL Plan, and fully burdened overhead rates) is not required to provide the logistical support that has comprised the bulk of RTC services. Adopting an alternative approach for facilitating trilateral exchanges may lower their cost but with downsides relative to RTC. The obvious upside to adopting one of the above approaches, or another approach to promoting trilateral cooperation at the Mission, is a potentially lower cost per trilateral exchange. There are, however, downsides. One downside is that projects will increasingly bear the financial, staff, and time costs of implementing trilateral exchanges, which is almost certain to reduce their number. As one Mission key informant noted, “Projects can handle it. However, as budgets become tighter, they have less capacity to handle this, and, honestly, when budgets become tight, this is seen as a secondary priority.” Another downside is the loss of flexibility that constitutes one of RTC’s most important value￾added contributions. If doing trilateral exchanges is left up to individual projects, the U.S. Embassy, Mission and projects lose the flexibility RTC provides to do trilateral exchanges where project funding has ended or where no project exists. These scenarios have accounted for over one-half of the trilateral exchanges to date. To compensate for the loss of flexibility offered by RTC, one key informant suggested that the Mission might tap into its Project Development Support (PDS) fund to support ad hoc trilateral exchanges. (The PDS fund, or its equivalent at USAID/El Salvador, is a flexible funding mechanism that can be used to fund ad hoc or other priority activities not anticipated in Embassy or Mission programming allocations.) 20 Another potential downside would be the loss of institutional knowledge that RTC has acquired in planning and executing trilateral exchanges. Experience has allowed RTC to move up the learning curve and implement its duties efficiently and effectively. IPs doing few trilateral exchanges would presumably struggle to reach RTC’s level of expertise with comparatively adverse implications for things like protocol adherence, and travel and security arrangements, all of which are particularly critical for events with high-level government officials. Without a centralized mechanism, moreover, there would be no way of collecting, analyzing, and disseminating information across trilateral exchanges related to their effectiveness, relationships formed and sustained, lessons learned, etc., which currently fall under RTC’s (specifically FUSADES’) purview. Nor would there be a mechanism to build linkages with stakeholders and IPs across trilateral exchanges. (The ET found little evidence to indicate that these were occurring to any significant degree, they thus exist as potential benefits of a centralized mechanism under a more expansive SOW than realized benefits of the centralized mechanism to date). Finally, it would also mean that the Mission and projects would need to specify early in their programming and project design process how trilateral cooperation fits within their plans. There are ways to increase flexibility at the margin, such as creating a trilateral fund or embedding trilateral exchanges in other budget line items, but either option is likely to result in a loss in flexibility that RTC provides. With the above said, the ET found a number of ways RTC has contributed important value-added to the trilateral exchanges implemented by the Mission, as described below. RTC has played an important role in incentivizing participation in trilateral exchanges beyond what would have occurred in its absence. As seen in Figure 4, 65.3 percent of WBS respondents said they either would not have (18.1 percent) or “probably” would not have (47.2 percent) participated in the same trilateral exchanges if not for RTC. Another 48.7 percent either would not have (9.7 percent) or “probably” would not have (38.9 percent) participated in another trilateral exchange without RTC. Figure 4: Additionality of RTC in Incentivizing Participation in Trilateral Exchanges (N=72) 18.1% 47.2% 20.8% 9.7% 4.2% 9.7% 38.9% 27.8% 16.7% 6.9% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% No Probably no Probably yes Yes Don't know Participate in Same Trilateral Exchanges Participate in Other Trilateral Exchanges Source: WBS In KIIs, key informants had various responses when asked about their participation in trilateral exchanges in RTC’s absence. Four indicated that they would have participated in the same trilateral exchanges; five would have participated in the same trilateral exchanges but, because of RTC, they participated in more trilateral exchanges than otherwise; four would have participated 21 in the same trilateral exchanges, but in a scaled-down form; and 25 would not have participated in any trilateral exchange in RTC’s absence. Indeed, one of the most consistent themes emerging from the KIIs with Mission and IP staff was that RTC was a key incentive and was critical to facilitating participation in more trilateral exchanges than would have occurred without RTC. Both Mission and IP staff saw the funding mechanism made available to projects unable to undertake the administrative burden of vetting, getting clearances, securing visas, and arranging logistics, as value-added. The mechanism encouraged them to undertake more trilateral exchanges than they otherwise could have. The discrepancy between the LOP target for the number of trilateral exchanges in RTC’s MEL Plan, (24), and the number of trilateral exchanges actually conducted through FY2, (33), with another 23 planned over the remaining LOP, 11 is further evidence of RTC’s role in stimulating demand for trilateral exchanges. This result implies that RTC has been significantly more effective in creating demand for trilateral exchanges than anticipated at its inception. “In the absence of this mechanism, we would not have been doing trilateral exchanges to the same extent. It is probably the case that there would have been much less emphasis on trilateral exchanges, and the number of them would probably fall.” – USAID/El Salvador “We would have done this without RTC, but having the mechanism made it easier and agile to propose and implement. We would not have done as many without RTC.” – USAID/El Salvador “Before RTC, we used to do one or two trilateral activities a year. With RTC, we’ve done maybe nine.” – USAID/El Salvador “Without RTC we would still be doing trilateral activities, but fewer of them.” – USAID IP “I do not think it would have been possible for my project to participate in the absence of RTC. Our project was actually just starting its implementation. I think there may have been a possibility of going to Colombia by our own means, but further in the project’s implementation.” – USAID IP The supply of trilateral exchanges has created demand for trilateral exchanges. The supply of trilateral exchanges has to demand for trilateral exchanges. According to one Mission key informant, “For the bottom line, because the Mission is committed to doing trilateral exchanges, there has been growing demand by technical offices to seek out [trilateral] opportunities because the contract is there.” Or, as similarly stated by a USAID IP, “In the past, we did not use trilateral activities. Right now? Because the mechanism is there, this is like a luxury. We have a very tight budget…if the mechanism exists, we should use it.” Given financial constraints, RTC funding was critical to local beneficiaries’ participation in South-South exchanges. Among the trilateral exchange participants at GOES ministries, a recurring theme was the environment of financial austerity within the central government, which made participation in any of the South-South exchanges facilitated by RTC 11 The RTC FY 2 Report lists 35 trilateral exchanges completed by the end of the fiscal year. This number, however, includes two trilateral exchanges—High Risk Court Models and Tertiary Prevention—which were postponed and remained pending at the time of the evaluation. 22 impossible. Dependence on external financing was not limited to the GOES, however, as key informants representing local organizations like CSOs, also consistently indicated that their participation in such South-South exchanges would have been highly unlikely without external support. “We have had an austerity policy in the Executive Branch for about 15 years. These exchanges are not possible for our institution to finance.” – GOES “We have two options. One is to do exchange visits to other countries. The other option is to bring experts here. Both of them, however, are not possible with resources existing at the ministry.” – GOES “We would not have participated in this trip without USAID assistance.” – CSO RTC provided useful flexibility for funding trilateral exchanges in unplanned-for circumstances. RTC has provided both a useful and flexible mechanism for organizing trilateral exchanges for projects that have ended, are ending, or that are otherwise out of money to address issues of Mission or Embassy priorities or unanticipated emerging issues. Of the 33 trilateral exchanges completed to date, 17 (51.5 percent) were not associated with a particular USAID-funded project but, instead, came out of the Economic Growth Office, the Democracy and Governance Office, the Office of the Mission Director, or the Office of the U.S. Ambassador. For example, the International Organization for Migration (IOM) Regional Conference for Reintegration of Returning Migrants was a trilateral activity designed to address an emerging policy issue not anticipated in earlier Mission programming. Presumably, many of these trilateral exchanges would not have been possible without a central funding mechanism like RTC in place. At the project level, RTC has allowed some projects to pursue alternative development strategies not included in their original work plans. In other cases, RTC has allowed projects to reallocate funds to pursue project objectives using more diverse strategies. “Doing this with USAID funds was helpful because it gave us the chance to move to other actions that were not in the work plan to support the goals we have. RTC gave us an opportunity to do this that would not have been possible otherwise.” – USAID IP “[RTC] kind of forces us to be innovative to make space to develop trilateral activities. Originally, such activities were not in anybody’s work plan.” – USAID/El Salvador “I would have done it without RTC, as it was already in the work plan. But with Peru, because the project is ending, we would not been able to do it, so RTC was critical in that case.” – USAID/El Salvador “They could not have done the logistics, since the project was ending, and it didn’t have funds, so RTC allowed it to keep operating.” – USAID/El Salvador Trilateral exchanges implemented under RTC have increased their perceived value among Mission and IP staff. RTC increased demand for trilateral exchanges among Mission and IP staff. Further, five Mission key informants noted that it has also increased the perceived value of trilateral exchanges more generally, which enhanced their legitimacy as a development intervention. Whether this translates into RTC becoming higher priority or trilateral exchanges being used more frequently—particularly when budgets are tight—remains to be seen. 23 RTC logistical support has been excellent. Professionalism and competence in handling trilateral exchange logistics is a primary added value RTC has offered since inception—including several high-visibility events involving high-level government officials. Almost without exception, key informants stated that the logistics were done well; that RTC staff were responsive, professional, and competent; that interactions with RTC staff were efficient and cooperative; and that activities unfolded without major hitches. RTC trilateral exchanges compare favorably to other trilateral South-South exchanges. Of the 72 WBS respondents, 40 (55.6 percent) had participated in other trilateral or South-South exchanges before RTC. As seen in Table 9, respondents were most likely to rate their RTC trilateral exchanges the “same” as their non-RTC exchanges, although nearly equal numbers of respondents rated the RTC trilateral exchanges as either “better” or “much better” than other South-South trilateral exchanges. Table 9: Perceived Quality of USAID-Facilitated Trilateral Exchanges (N=40) Quality Dimension Much Worse Worse Same Better Much Better Don’t Know Not Applicable Cost to participate 2.5% 12.5% 20.0% 27.5% 15.0% 7.5% 15.0% Time to participate 0.0% 2.5% 42.5% 30.0% 17.5% 2.5% 5.0% Relevance/importance 0.0% 2.5% 42.5% 22.5% 27.5% 5.0% 0.0% Addressed participants’ needs 0.0% 0.0% 40.0% 22.5% 30.0% 5.0% 2.5% Expertise of providers 0.0% 5.0% 37.5% 20.0% 32.5% 2.5% 2.5% Usefulness 0.0% 12.5% 27.5% 30.0% 22.5% 7.5% 0.0% Organization and logistics 0.0% 12.5% 30.0% 27.5% 22.5% 7.5% 0.0% Facilitate new relationships 0.0% 10.0% 25.0% 27.5% 27.5% 10.0% 0.0% Creation of tangible benefits 5.0% 10.0% 27.5% 22.5% 27.5% 7.5% 0.0% Source: WBS RTC trilateral exchanges were more effective in facilitating new relationships than other trilateral or South-South exchanges. The WBS asked respondents to compare relationships formed from RTC trilateral exchanges with those formed from other South-South trilateral exchanges. Figure 5 shows that 61 percent said that new relationships facilitated by RTC were either “better” (39.0 percent) or “much better” (22.0 percent) compared to 31.8 percent who said that they were either “the same” (24.4 percent) or “worse” (7.3 percent). Figure 5: Quality of RTC-Facilitated Relationships Compared to Those Formed from Other Trilateral or Bilateral SSC Exchanges (N=41) Source: WBS 24 4.3.2 Conclusions • RTC is perceived as a relatively expensive mechanism for funding trilateral exchanges that could, in most (but not all) cases, be funded by individual projects at a potentially lower cost, either with project or outsourced staff. • Due to a lack of benchmark data, how RTC’s costs compare to other trilateral cooperation models, including embedding it as a component or sub-component in an existing project, could not be determined. With a Mission per-exchange cost of nearly $70,000—consisting of 40 percent variable costs and 60 percent fixed costs (including RTC overhead allocations), the Mission can determine whether these costs are acceptable or unacceptable. • RTC is currently performing few of the roles envisioned in its SOW. Results of this are 1) it is currently playing largely a logistical support role and is perceived as such by Mission and IPs staff; and 2) it has a low perceived added value and cost-effectiveness. • Mission and IP staff favor a continued role for the Mission to support trilateral exchanges but are split as to whether this should be via RTC or some other mechanism. If RTC is extended, the Mission and IP staff agree it should be assigned an expanded role, although given RTC’s budget structure, it is unclear how this expanded role would be financed. • The perception is that making IPs responsible for trilateral exchanges is a less expensive alternative. However, this involves the following tradeoffs—a reduced number of exchanges; a transfer of the logistical burdens to IPs, some of which are less equipped to assume them; a loss in flexibility in when and how to do trilateral exchanges where projects have ended or do not exist; potential loss in planning and implementation efficiency and effectiveness; and the absence of a mechanism to collect, analyze, and disseminate information about exchange effectiveness and lessons learned. • Various options exist for the Mission to support trilateral exchanges in the absence of RTC. Most of them, however, require the Mission and projects to engage in strategic up￾front planning for how and when to incorporate trilateral exchanges in programming, project design, and implementation. Absent this planning, their use will be less effective. • RTC-facilitated trilateral exchanges were of comparatively high quality relative to other forms of trilateral cooperation or SSC. • RTC’s primary value-added contributions have been 1) its funding support to IPs; 2) its professionally competent support in facilitating exchange logistics; 3) its incentivization of more trilateral exchanges than would have occurred otherwise (RTC has been highly effective in creating demand for trilateral exchanges within the Mission); 4) the flexibility it has provided for doing trilateral exchanges to address unplanned-for priority issues where projects have ended or no project exists; and 5) its contribution to increasing the perceived legitimacy of trilateral cooperation relative to traditional development approaches. • RTC is providing benefits and services beyond logistical support and, under its new management team, is making efforts to increase its role in this regard. Many of these benefits or services go unnoticed by trilateral exchange participants, thus their 25 perceptions of RTC’s value-added contribution probably understate its actual contribution. 4.4 EQ3: TO WHAT EXTENT HAS THE RTC PROJECT CONTRIBUTED TO BUILDING PARTNERSHIPS AMONG THE USG, THE GOVERNMENT(S) OF THE RECIPIENT COUNTRY(IES), PRIVATE SECTOR AND CIVIL SOCIETY OF EL SALVADOR, GUATEMALA, AND HONDURAS (NORTHERN TRIANGLE OF CENTRAL AMERICA), AND THOSE OF SSC PROVIDER COUNTRIES? 4.4.1 Findings Trilateral exchange participants strengthened and formed cooperative relationships with both home country and foreign country counterparts. Over 70 percent of survey respondents “agreed” or “strongly agreed” that participating in RTC trilateral exchanges either strengthened existing relationships or helped form new relationships with counterparts (see Figure 6). Figure 6: Effectiveness of USAID-Facilitated Trilateral Exchanges in Strengthening or Forming Cooperative Relationships (N=72) 5.6% 6.9% 6.9% 22.2% 48.6% 9.7% 4.2% 6.9% 6.9% 29.2% 45.8% 6.9% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree Don’t know Strengthened existing relationships Formed new relationships Source: WBS Figure 7, below, shows that 70 percent of trilateral exchange providers and beneficiaries either strengthened existing or formed new cooperative relationships with their home country counterparts (75.8 percent) or with their foreign country counterparts (69.4 percent). Figure 7: Trilateral Exchange Participants Strengthening or Forming Cooperative Relationships with Home Country and Foreign Country Counterparts (N=62) 75.8% 19.4% 4.8% 69.4% 25.8% 4.8% 0.0% 20.0% 40.0% 60.0% 80.0% Yes No Don't know Own Country Other Country Source: WBS With three exceptions, key informants who participated in trilateral exchanges affirmed they either strengthened existing relationships or established new relationships as a result, whether 26 with their home country or foreign country counterparts—foreign county counterparts including both provider countries and other Northern Triangle beneficiary countries. According to a Mission key informant, “There is indisputably a secondary effect of these trilateral exchanges. The secondary effect is the effect of people forming relationships and work with each other afterwards.” As a result of the relationship formed between the El Salvador Institute of Access to Public Information (IAIP) and its provider counterpart in Peru during the Open Government Trilateral Exchange Activity, IAIP became the provider institution to its Peruvian counterpart by providing it TA related to the implementation of new Tribunal of Transparency. According to a key informant at IAIP, “The Tribunal of Transparency in Peru was recently developed, similar to something El Salvador already had. So, El Salvador is helping them to establish it. They [Peru] just named the institution, but now they need to figure out how to implement it.” Although in approximately half of the interviews, the key informants noted already knowing and having a relationship with their provider country counterparts, participation in the trilateral exchange still served to strengthen those relationships. However, in nearly all cases, the trilateral exchange further helped participants form new counterpart relationships, particularly in cases where, according to one key informant, “They [foreign counterparts] are frequently rotating in and out of their positions, so it is good to meet the new ones who have taken their place.” (See Annex G for more detailed information on the relationships formed between exchange participants.) These results were corroborated in a 2015 Organization for Economic Cooperation and Development (OECD) survey, in which all 60 organizations reported forming mutually beneficial relationships by participating in trilateral exchanges (OECD, 2016). “The encounter facilitated meeting key people and experts on topics relevant to our institution’s mission and day-to-day tasks.” – GOES “I could say that I established 4-5 relationships with colleagues from other countries. For example, I met a Colombia sanitary authority representative, and we have kept emailing, and I have made important consultations regarding manufacturing and supplier sanitary regulation procedures.” – GOES “Study tours are important. In our case, for example, Ruta N triggered effective relationships with universities and with the Industria Joven Project.” – CSO “Some of the people there I already knew…we refreshed the relationship. Other colleagues, I met for the first time, and it was good knowing who they are, what they do, and where they are at…There is always competition and new scientific products and substances, which depending on the country, are sometimes given different category or market treatment…so it is important having these contacts.” – Provider Trilateral exchange participation contributed to increased cooperation among USAID projects. The two USAID projects that participated in the Ruta N trilateral exchange activity—Higher Education for Economic Growth Activity (HEA) and Economic Competitiveness Activity (ECA)—affirmed that participation in the exchange enabled them to establish a closer relationship to pursue shared objectives collaboratively. One key informant from an IP said, “Another big advantage of this trilateral activity was that it opened up a space for us to collaborate more activity with the other USAID project. Normally, this type of collaboration is difficult to achieve, but this opened the window for this type of collaboration to occur, and it facilitated it.” Speaking on the same 27 topic, a Mission key informant observed that “the coordination between the projects in innovation was not so good beforehand but is better afterwards.” Trilateral exchange participation contributed to increased cooperation among traditional adversaries in El Salvador. The Ruta N trilateral exchange contributed to the formation of an incipient relationship between GOES [specifically, the Ministry of Education (MINED)] and Salvadoran Industry Association (ASI) representatives. In the words of a Mission key informant: “Suddenly we have a qualitative impact in the spectrum. [MINED] is at the same table with [ASI]. The dynamic makes this happen…This is qualitative change…The visits brought more personal relationships across the leadership of the two organizations. There is now an open line of communication between them…so next time an opportunity comes up with USAID, it increases the likelihood that they will take it.” Trilateral exchange participation contributed to increased cooperation among beneficiary counties. In two cases, trilateral exchange participation contributed to forming productive cooperative relationships between beneficiary country counterparts. For example, the U.S.-Mexico Customs and Border Facility trilateral exchange activity—part of the Regional Trade Facilitation trilateral effort—helped forge closer ties between the Customs Directorates in El Salvador and Guatemala, including drafting an agreement on a detailed cooperation plan between the two agencies. Relationships strengthened or formed during trilateral exchanges would not likely have occurred otherwise. To measure the added value of the RTC trilateral exchanges in strengthening or forming cooperation relationships, the WBS asked respondents to rate the likelihood relationships would have been forged or strengthened without participation in the RTC trilateral exchange. As seen in Table 10, about three-fourths of respondents said that they would not have or “probably” would not have strengthened or formed the same or similar relationships without RTC. In contrast, about one-quarter said they “probably” would have or would have done formed or strengthened the same cooperative relationships without RTC. Table 10: Additionality of Trilateral Exchanges in Strengthening and Forming Cooperative Relationships (N=49) Additionality No Probably No Probably Yes Yes Don’t Know Strengthened or formed same relationships 14.3% 63.3% 20.4% 2.0% 0.0% Strengthened or formed similar relationships 12.2% 61.2% 24.5% 2.0% 0.0% Source: WBS Again, the KIIs confirm survey results. In two-thirds of cases, key informants either said they would not have or would likely not have formed the same relationships without participating in the RTC trilateral exchanges. “We made particularly useful contacts, one from Mexico and one from Argentina. We would not have made these contacts without our participation in the trilateral activity.” – GOES “It would have been very hard to make these contacts without participation in the trilateral activities. We tried to make contact before but with little success.” – GOES 28 “Without RTC, it would have been very difficult to develop the same relationships in the same context.” – USAID IP Trilateral exchange participants have leveraged their new or strengthened relationships for multiple purposes. Of WBS respondents who said they either strengthened or formed cooperative relationships during an RTC-facilitated trilateral exchange, 77.6 percent said that they have taken advantage of these relationships within their organization for a variety of reasons that fall under the following six broad categories (with the number of mentions in parentheses)—Establish Relationships and Commitments (9); Update Laws and Regulations (8); Share Information (5); Strengthen Institutional Capacity (5); Strengthen Partner Capacity (5); and Implement Innovations (3). The specific responses informants gave for each of these categories are in Annex F with the other WBS results. 4.4.2 Conclusions • RTC trilateral exchanges have been an effective mechanism for strengthening or forging collaborative relationships with relevant home country, provider country, and other beneficiary country counterparts. • RTC has contributed important added value in that the trilateral exchange participants would generally not have strengthened or formed these relationships without RTC. • Trilateral exchange participants have overwhelmingly leveraged these strengthened or new relationships to implement administrative, technological, policy, legal, and other reforms within their organizations, among their partners, and within their countries. 4.5 EQ3.1: HOW DO THE ABOVE RECIPIENT COUNTRIES PERCEIVE THE TRILATERAL COOPERATION FACILITATED BY THE RTC PROJECT COMPARED WITH TRADITIONAL NORTH-SOUTH COOPERATION? 4.5.1 Findings Trilateral cooperation is a growing alternative to traditional North-South Cooperation approaches and offers a number of other potential comparative advantages. To help answer EQ3.1, the ET believes that placing it within the context of the broader debate about trilateral cooperation will be useful. Of particular interest are the motivations behind trilateral cooperation and the comparative advantages relative to traditional North-South Cooperation. The time and space that can be devoted to this discussion are limited, so only a sense of this debate is provided here. Readers interested in learning more about it are encouraged to reference the bibliography found in Annex B of this report.12 Motivated in part by the economic growth and development in several middle-income countries (Brazil, Colombia, Mexico, Paraguay, Costa Rica, and Peru) that position them as potential development partners aligned with traditional Northern bilateral and multilateral development agencies, trilateral cooperation has received increased attention in recent years. This has been accompanied by a renewed focus on knowledge sharing as a potentially powerful engine for development. According to the 2015 OECD survey results, TA is the primary modality for 12 The relevant literature often adopts the term “triangular” cooperation instead of “trilateral” cooperation. The two terms are typically used synonymously, even if some (e.g., Rhee, 2011) argue for a distinction between them. 29 trilateral cooperation, including joint research, training programs, expert visits, study tours, scholarships, internships, and volunteering in developing countries. Trilateral cooperation offers a number of potential advantages relative to traditional North-South cooperation. In the OECD survey, providers saw deepened collaboration with beneficiary countries as the primary advantage, while beneficiary countries saw financial and technical support from provider (or pivotal) countries and Northern partners as the primary advantage. Other presumed comparative advantages of trilateral cooperation mentioned prominently in the literature include these listed below. • Trilateral cooperation is cheaper and more cost effective, while learnings from other developing countries are more relevant and culturally appropriate (Kumar, 2008; Fordelone, 2009). • Trilateral cooperation promotes partnerships for development and facilitates knowledge sharing and joint learning via creation of horizontal partnerships of peer countries that strengthen regional and other networks and increase intercultural understanding. • Beneficiary countries may be able to assert more country ownership and negotiate better development outcomes given the involvement of other southern partners [Spanish Agency for International Cooperation for Development (AECID), 2010; Ashoff, 2010]. • Trilateral cooperation bridges the divide between North-South Cooperation and SSC by combining comparative advantages of different partners and taking advantage of complementary strengths to create synergies (Kharas, 2007; Pantoja and Elsner, 2009). • Trilateral cooperation sets mechanisms in place and develops provider country capacity to continue cooperation after phasing out Northern assistance. • New development partners can serve as experts on diverse topics so that future SSC can be more efficient, cost-effective, and better adapted to the beneficiary countries’ context. Empirical evidence to substantiate the presumed comparative advantages of trilateral cooperation is lacking. Notwithstanding its presumed comparative advantages, critical, empirically-based analyses of trilateral cooperation remain scarce (see, for example, Abdenur, 2007; AECID, 2010; Altenburg and Weikert, 2007; Chahoud, 2008; Fordelone, 2009; Kumar, 2008; and Mehta and Nanda, 2005). Notably, there is little empirical evidence of its impact and the added value from the beneficiary country’s perspective. Nor is there evidence on whether or not it promotes effective partnerships (Cabral and Weinstock, 2010; McEwan and Mawdsly, 2012). What is “known” about the relative effectiveness of trilateral cooperation is based largely on third-source and ad hoc information (AECID, 2010). South-South and trilateral Cooperation are growing forms of donor cooperation in El Salvador and the region. In addition to USAID, the European Union (EU), Luxembourg Development Agency (Lux-Dev), AECID, German Corporation for International Development (GIZ), Japan International Cooperation Agency (JICA), United Nations Development Program (UNDP), Food and Agriculture Organization (FAO), Embassy of Brazil, and Embassy of Mexico reported working with El Salvador and other countries in the Latin America and the Caribbean (LAC) region via trilateral cooperation. These include El Salvador as the provider country in at least two cases (Embassy of Mexico and AECID). The text box below describes how Lux-Dev engages with El Salvador in trilateral cooperation, while Annex I describes the trilateral 30 cooperation efforts between El Salvador and the EU, GIZ, and AECID. Finally, and to place USAID/El Salvador’s trilateral cooperation efforts within a global context, Annex H describes a number of other trilateral cooperation approaches identified during the ET’s literature review— including their organizational and funding models and related links for readers who would like more information. Text Box 3 Lux-Dev works entirely through the Ministry of Foreign Affairs (MINEX) and supports trilateral cooperation through budget assistance. To date, it has provided 1.2 million Euros in trilateral assistance. Lux-Dev and other donors contribute to a common fund in the MINEX. The MINEX and other GOES ministries design trilateral assistance projects and use this fund to finance them. El Salvador manages the entire project cycle, not the donors. The donors contribute funds, review project proposals, monitor the projects, and work with various ministries throughout project implementation. A commission inside the MINEX, the Office of International Cooperation, is comprised of officials from the Ministry and the Technical and Planning Secretariat (SETEPLAN) and meets with donors to obtain their commitment to support the fund and to support trilateral cooperation. The commission reviews projects with donors and obtains their commitment. The commission meets once every trimester to review projects. According to Lux-Dev, El Salvador has more South-South and Trilateral Assistance than any other country in the region. Lux-Dev has an open dialog with GOES ministries and other donors, and it always informally discusses opportunities to collaborate in trilateral assistance. USAID is also working with other bilateral and multilateral donor/cooperation agencies to facilitate trilateral cooperation between El Salvador and other LAC countries. RTC has collaborated with the Mexican Agency for International Development Cooperation (AMEXCID) on 11 trilateral exchanges; the Colombia Agency for International Cooperation (APC) on four trilateral exchanges; the Chilean Agency for International Cooperation for Development (AGCID) on two trilateral exchanges; and the Peruvian Agency for International Cooperation (APCI), World Bank, and International Monetary Fund on one trilateral exchange each. RTC’s Contracting Officer Representative (COR), moreover, works with cooperation agencies in provider countries to coordinate trilateral exchanges and identify government officials to participate in the exchanges. RTC estimates that provider country counterparts have contributed approximately $350,363 of in-kind support for the trilateral exchanges. This is in comparison to $585,851 contributed by RTC,13 or 35.0 percent of the total trilateral exchange cost. At the time of the evaluation, however, the ET found no evidence that any of these provider country cooperation agencies took the initiative to continue providing TA to El Salvador via trilateral or bilateral SSC using their own funds. The Donor Cooperation Study completed by MEL Initiative found that MINEX has expressed the desire for a closer relationship with USAID and has requested that USAID meet to discuss trilateral cooperation activities. Nonetheless, at the time of the evaluation fieldwork, MINEX reported that its position on this issue had not changed. 13 This amount does not break down the contribution by type of contributor, so there was no way to determine how much of this amount was provided by the provider country cooperation agency. 31 WBS respondents and key informants agree that trilateral cooperation is an effective development intervention strategy. Consistent with the findings in the above literature review, 79.2 percent of WBS respondents either “strongly agreed” (62.1 percent) or “agreed” (18.1 percent) that trilateral cooperation is an effective development strategy. For their part, key informants, without exception, also agreed that trilateral cooperation is an effective development intervention strategy. With regards to its specific benefits, WBS respondents said that learning how other countries solved similar problems was the most important benefit of trilateral cooperation. This was followed by learning new methods for solving problems and establishing cooperative relationships with experts in other countries (Figure 8). Figure 8: Perceived Benefits from Participating in USAID-Facilitated Trilateral Exchanges (N=72) 41 9 6 5 6 5 10 24 13 12 4 9 6 12 21 11 15 7 57 45 40 28 25 21 0 20 40 60 80 100 120 Learning how other countries have solved similar problems Learn about new methods to solve problems Establish cooperative relationships with experts in other countries Learn about contextually appropriate solutions from other countries Establish cooperative relationships with organizations in other countries Identify low cost solutions to problems 1st Most Important 2nd Most Important 3rd Most Important Total Source: WBS Trilateral exchange participants plan to participate in future exchanges or other trilateral exchanges. Almost without exception, key informants who participated in RTC trilateral exchanges said that they are planning to, or would be willing to, participate in future RTC trilateral exchanges. This result is corroborated in the WBS results shown in Figure 9 where 87.5 percent of respondents said that it was either “likely” (22.2 percent), “probable” (15.3 percent), or “certain” (50.0 percent) that they would participate in future USAID-facilitated trilateral exchanges. Another 76.4 percent said that it was either “likely” (31.9 percent), “probable” (26.4 percent), or “certain” (18.1 percent) that they would participate in future non RTC-facilitated trilateral exchanges. 32 Figure 9: Intentions to Participate in Future RTC-Facilitated or Other Trilateral Exchanges (N=72) 0.0% 1.4% 22.2% 15.3% 50.0% 11.1% 0.0% 2.8% 31.9% 26.4% 18.1% 20.8% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% No Not likely Likely Probably Yes Don't know Future RTC Trilateral Exchange Future Non-RTC Trilateral Exchange Source: WBS Mission and IP staff will continue investing in trilateral exchanges. One measure of how much trilateral exchange participants value trilateral exchanges is whether they would invest in them out of their own budgets. The survey asked Mission and IP staff whether they would continue investing in trilateral exchanges were RTC to end tomorrow. Of the 12 who answered this question, four said “yes,” three said “no,” and five answered “don’t know” (see Figure 10). Figure 10: Mission and IP Staff Willing to Invest in Trilateral Exchanges Without RTC 1 3 4 2 1 3 2 3 5 0 2 4 6 8 10 12 14 USAID USAID IP Total Yes No Don't know Source: WBS The KIIs revealed stronger support among Mission and IP staff for investing their own funds in trilateral exchanges, 10 (about one-half) of whom said they would invest their own funds. According to one USAID/El Salvador key informant, “Our office will continue with the trilateral approach regardless of whether the project is extended. Perhaps not in the same amount, but there is now a recognition that the approach is valuable, but the implementing partner will do it.” Another two said they would not invest their own funds, or that they did not know whether they would invest their own funds. Mission and IP staff favor continued Mission support for trilateral exchanges. Eleven (11) of 12 Mission and IP staff who responded to the WBS said that the Mission should continue to support trilateral exchanges. In KIIs, Mission and IP staff unanimously advocated for ongoing Mission support for trilateral exchanges in one way or another. 4.5.2 Conclusions • The literature affirms that trilateral cooperation is a legitimate complement or alternative to traditional North-South Cooperation. Within El Salvador, multilateral donor agencies 33 are actively supporting trilateral cooperation with El Salvador and in the region, each using a different operational and financial model. • There is widespread agreement about the advantages of trilateral cooperation relative to traditional North-South Cooperation. These comparative advantages, however, remain theoretical given the current lack of empirical evidence to support them. • While the evaluation findings do not allow a conclusion about the relative effectiveness of trilateral cooperation, they do allow the conclusion that, in the case of RTC, trilateral exchanges have been effective, particularly for learning how other countries in the region have addressed similar problems and for forging relationships with provider country counterparts. • Bilateral and multilateral donor/cooperation agencies working in El Salvador and the LAC region are actively involved in facilitating trilateral or bilateral South-South exchanges. While USAID/El Salvador is working on trilateral exchanges, in a number of cases there is room to increase its activity in this area, including working more closely with MINEX on identifying trilateral cooperation opportunities with other donor/cooperation agencies. • Mission and IP staff value their participation in the RTC-facilitated trilateral exchanges. This positive opinion is evidenced by their intention or willingness to participate in future RTC and non-RTC trilateral exchanges and invest their own funds in future trilateral exchanges. It is also apparent by their near unanimous advocacy for continued Mission support of trilateral cooperation. 4.6 EQ3.2: TO WHAT EXTENT HAVE THE BENEFICIARY ORGANIZATIONS BUILT NETWORKING WITH THEIR COUNTERPARTS OR PRIVATE SECTOR IN THE SSC PROVIDER COUNTRIES THAT MAY LEAD TO FUTURE COLLABORATION BEYOND THE LIFE OF THE RTC PROJECT? 4.6.1 Findings Trilateral exchange participants have maintained the relationships they formed during the trilateral exchanges for at least six months. FUSADES tracked both the number of new relationships trilateral exchange participants reported forming during the exchange and the number of these relationships that participants reported still having six months later. The results, shown in Figure 11, reveal that exchange participants formed an average of 2.5 new relationships in provider and other and maintained an average of 2.4 relationships six months later. In contrast, the same trilateral exchange participants formed an average of 2.5 new home country relationships during the exchanges and maintained an average of 1.9 of those relationships six months later. 34 Figure 11: Number of New Relationships Formed During Trilateral Exchanges and Six Months After 2.5 2.5 1.9 2.4 0 0.5 1 1.5 2 2.5 3 Home Country Other Countries Post trilateral exchange After 6 months Source: RTC MEL Plan Trilateral exchange participants are confident that they will maintain these strengthened or new cooperative relationships over time. Among the WBS respondents who reported strengthening or forming cooperative relationships through RTC trilateral exchanges, 87.5 percent said that it was either “likely” (27.1 percent), “very likely” (37.5 percent), or “certain” (22.9 percent) that they would maintain those relationships over the long term (see Figure 12). Figure 12: Likelihood of Sustaining Cooperative Relationships Strengthened or Formed as a Result of USAID-Facilitated Trilateral Exchanges (N=48) Source: WBS In the KIIs, key informants expressed confidence that the relationships established during the trilateral exchanges would be sustained over time. Of those who commented on this question, 19 said that the relationships would be sustained, two said they would not, and six were not sure. Among the latter group, one USAID IP observed, “To be in contact with someone is unlikely to continue. Everyone has their own fires to put out each day, and they have to deal with day-to-day issue and operations. Plus, officials and technical staff rotate.” Factors contributing to the expected sustainability of relationships include a high personal motivation, organizational support, the value of counterpart knowledge. Other important factors contributing to the expected sustainability of the relationships include shared objectives and productive working relationships among relationship counterparts and the delivery of tangible benefits from the relationships (see Figure 13). 35 Figure 13: Factors Contributing to the Sustainability of Strengthened or New Cooperative Relationships (N=46) 24 21 18 16 16 16 9 8 6 5 2 0 5 10 15 20 25 30 High level of personal motivation Support from my organization Counterparts possess valuable knowledge Shared objectives with counterparts Relationships provide tangible benefits Productive working relationship with counterparts Support from my immediate supervisor Availability of financing Counterparts all contribute equally Get along well with counterparts Relatively high priority Source: WBS Relationships strengthened or formed are more ad hoc than ongoing. Further probing about these relationships indicated that many of these relationships (at least half) were ad hoc or intermittent in nature and conducted via social networking platforms like Facebook or WhatsApp or through occasional emails. That is, they were not characterized by ongoing communication and collaboration. 14 In these cases, key informants viewed the relationships akin to a contingent resource on which they could draw as needed. “So far, we are not using the contacts too aggressively. We are confident that they are people we can go to if we need assistance.” – GOES “We kept those relationships for a good while. Now we hardly ever communicate, but we know we can, and that they will be responsive.” – GOES “After the visit, we did follow-up with the experts, but these are not very frequent and only as needs arise.” – GOES “At this point, not much [communication], but yes, every now and then we greet them through email or WhatsApp. We know, and they know, there is an open channel and trust.” – Provider While the ET does not necessarily see this as a criticism of such relationships—indeed the ET sees the expectation that all or most of these relationships include ongoing communication and collaboration as unrealistic—it is important to understand that a “relationship” in this context takes on a variety of forms and exists along a continuum from low to high intensity. Based on information provided by key informants, the majority of these relationships appear to exist on the less intensive end of the continuum. A number of relationships, however, were of a more intensive, ongoing nature. Nonetheless, there was evidence that several of the trilateral exchanges had either produced more intensive relationships or that participants were working to create them. For example, 14 Despite probing on this topic, the ET found it difficult at times to pin key informants down as to the precise nature of these relationships and how they were functioning. 36 participation in the Civil Service Reform trilateral exchanges (part of the Justice Sector Strengthening trilateral effort) has led to ongoing, formalized forms of cooperation building on what was achieved during the trilateral exchange. This was described by one of its GOES participants, who said, “We still have activities planned with Chile, their assistance is still ongoing, we have identified other areas of collaboration. After our visits, we produced a document with agreed upon cooperation areas.” The series of Ruta N/Roadmap trilateral exchanges (part of the Higher Education for Economic Growth and Economic Competitiveness trilateral effort) is another example of participants appearing to have established long-term collaborative relationships or making efforts to do so. According to one USAID IP: “Yes, I established good relationships, and these are two-way relationships. I still have good communication with Medellín experts, particularly those involved in innovation labs in universities and entrepreneurship labs in alcaldías for attracting investment.” Similarly, a key informant in the Mission noted, “With the group that went to Medellin, we have established a sustainable network…everyone now knows each other, we keep in touch. Yesterday, we had a meeting with the group to discuss our participation in the Innovation Week, for example.” Participants in the Ruta N/Roadmap trilateral exchanges and other local stakeholders have met on at least two occasions post-exchange to organize and coordinate innovation initiatives across local sectors and actors. Whether or not this initiative will bear fruit remains to be seen. Another example of emerging cooperation is in the area of open government where participants in the Open Government Partnership Trilateral Exchange Activity (part of the Justice Strengthening trilateral effort) have drawn on relationships established during the Open Government Conference in Argentina to improve the GOES’ open government web platform. One GOES participant in this conference said: “At the meeting, I met with Sergio Araiza, one of the pioneers of DemocracyOS, an open source platform that enables citizens not only to get informed about public issues but also to debate proposals, positions, points of views…I have phoned this ICT [Information and Communications Technology] expert, and we have established a good relationship…He is sharing with us the open government process done in Mexico, and we are frequently consulting with him…We have also organized a team with the IAIP that is defining the structure and nature of the data to be included in the open data platform. We have agreed with him that we will share with him our advancements.” The Sanitary Registration Trilateral Exchange Activity, part of the Regional Trade Facilitation trilateral effort, is another example of the exchange leading to more structured, ongoing collaboration efforts. According to a Mexican counterpart who participated in the exchange: “We left an open channel for collaboration with many of the people whom we met. We have kept close communication since the encounter, particularly with Honduras. One month after the event, a Honduran delegation from ARSA [Honduras Health Regulation Agency] came to COFEPRIS [Mexican Federal Commission for the Protection against Sanitary Risk] for three days and received further technical assistance…It was agreed that a work plan was going to be formulated for implementing COFEPRIS’ recommendations for ARSA, establish direct communications channels between both regulatory agencies, and work on a collaborative instrument to formalize and promote cooperation ties.” 37 ARSA contacted its counterpart at the Brazilian Health Regulatory Agency (ANVISA), which also participated in the Sanitary Registration trilateral exchange activity, and the two have entered into conversations about doing a joint cooperation project in 2019. The cooperation agreement entered into by the Salvadoran and Guatemalans customs agencies described in Section 4.4.1 is another of how participation in a trilateral exchange has contributed to formalized, ongoing cooperation between participant counterpart organizations. “It [Sanitary Registration Trilateral Exchange Activity] was the first time that all health registration authorities were gathered in the same room…this was extremely important because it allows to exert pressure for formulating a work plan for the project to continue.” – Provider 4.6.2 Conclusions • RTC trilateral exchanges have been an effective mechanism for strengthening and/or forming cooperative relationships that have a reasonable likelihood of being maintained over time. • While the evidence does not allow confirmation that these relationships will be maintained over the long term, trilateral exchange participants are confident they will be. • To the extent that these relationships are maintained over the long term, they appear more likely to be maintained as ad hoc relationships in the form of a contingent asset that trilateral exchange participants can draw on as needed and not ongoing, active collaborations. 5.0 RECOMMENDATIONS Drawing on the findings and conclusions presented above, the ET recommends the following to guide decision-making and planning regarding RTC and future Mission programming in trilateral cooperation: • USAID/El Salvador should continue to support trilateral cooperation at the Mission. The results from trilateral exchanges implemented under RTC have demonstrated that trilateral cooperation can be an effective instrument in the Mission’s toolbox of development programming. While the evaluation results did not allow a conclusion as to the relative effectiveness of trilateral cooperation compared to more traditional development approaches, they did allow a conclusion about its case-by-case effectiveness in terms of 1) transferring knowledge; 2) facilitating action based on transferred knowledge; and 3) strengthening and/or forming cooperative relationships with host country, provider country, or other beneficiary country counterparts. In addition to this, Mission staff and IPs overwhelmingly favor a continuation of Mission support for trilateral cooperation. • If USAID/El Salvador accepts the recommendation to continue supporting trilateral cooperation, it needs to decide which model to use. One option is to extend RTC into its option year. In light of cost-benefit considerations, however, the ET does not recommend this if extending RTC’s role would continue to be limited to logistical support. There are potentially lower-cost options to provide logistical support, such as tasking projects with the responsibility of funding and organizing trilateral exchanges, whether with their own or out-sourced staff; contracting with a local firm for logistical support; 38 extending RTC but staffing it entirely with local (or regional) staff; or creating a trilateral￾support fund administered by the Program Office (or other entity). This support fund would allow projects to apply for funding to support trilateral exchanges and other needs. The annexes to this evaluation provide information and links to different models of trilateral cooperation for the Mission’s reference. Assessing these different models was outside the evaluation’s scope, but the ET encourages the Mission to investigate these other models to see if any are relevant to the Mission’s situation. • If, on the other hand, the Mission elects to extend RTC, it should expand RTC’s role to include an increased set of tasks more likely to generate added value. The added value would compensate for its costs, with the caveat that the tasks should be compatible with RTC’s existing technical capacity and budget allocations. Alternately, the Mission could either allocate new resources or reallocate existing resources to expand RTC’s technical capacity to undertake a wider variety of tasks. • In deciding how to expand RTC’s role, the Mission should facilitate discussions between RTC and the technical teams in the Mission. Together, they should identify areas of highest demand and that best fit with RTC’s technical and budget constraints. • Two overlapping and mutually reinforcing areas where RTC might add particular value and that also appear consistent with its technical and budget constraints, are working with trilateral exchange participants and IPs to plan and facilitate actions, both pre and post￾exchange, to 1) strengthen/form relationships between exchange participants and leverage those relationships over time, and 2) implement actions/reforms drawing on the knowledge acquired during the exchanges. This recommendation addresses three concerns about the ultimate effectiveness of trilateral exchanges and RTC’s role in facilitating them. The three concerns were reflected in the EQs and voiced by key informants, they are, 1) the lack of pre and/or post-exchange planning on how to leverage the trilateral exchange to implement reforms, innovations, and other concepts in beneficiary organizations and countries; 2) strengthening or forming cooperative relationships and maintaining them over time; and, 3) increasing the cost-effectiveness of RTC’s service delivery model. • USAID/El Salvador should build closer relationships with other bilateral and multilateral donors in the LAC region already actively engaged in trilateral and South-South cooperation activities in El Salvador and the Northern Triangle. These donors include the EU, AECID, GIZ, Lux-Dev, AMEXCID, Brazilian Cooperation Agency (ABC), APCI, APC, and AGCID. Relationships with donors should be forged in addition to strengthening the Mission’s relationship with MINEX, which is responsible for coordinating GOES’ trilateral cooperation efforts within the country. The Donor Coordinator in USAID’s Program Office can take the lead in developing these relationships and can serve as the trilateral cooperation point of contact in case USAID decides not to extend RTC. In that case, he/she can further develop a listing of countries and donor organizations that have a comparative advantage and can offer collaboration opportunities to USAID projects, Mission technical offices, or the Mission/Embassy. 39 6.0 ANNEXES 40 ANNEX A: EVALUATION STATEMENT OF WORK 41 STATEMENT OF WORK (SOW) USAID/CAM Mid-Term Performance Evaluation of the USAID Regional Trilateral Cooperation Activity A. Purpose of the Evaluation The purpose of this Mid-Term Performance15 Evaluation is to inform USAID/CAM on: 1) the efficiency of resources used under this mechanism, 2) the level of leverage achieved, and partnerships built, and 3) how the transfer of knowledge and technology is being applied in the recipient countries, which are El Salvador, Honduras, Guatemala, Nicaragua, and Mexico. In addition, the evaluation will serve to provide empirical evidence on management issues and will document learning and continuous improvement. Mission will use the findings and recommendations of this evaluation to make decisions on how to exercise the option period of the activity and inform the potential design of future trilateral cooperation mechanisms. The principal audience of this internal evaluation will be USAID/CAM, particularly Mission Management, the Regional Program Office, which manages the Regional Trilateral Cooperation (RTC) activity, and both Technical Offices in the El Salvador Mission: Economic Growth and Democracy and Governance. Other audiences inside USAID interested in the evaluation results are the Program Offices of the other missions in the Central American region (mainly Donor Coordinators), the Bureau for Latin America and the Caribbean, and the Office of Donor Engagement in the Bureau for Policy, Planning, and Learning. The implementing partner, International Business & Technical Consultants Inc. (IBTCI), will carry out the remaining activity implementation based on findings and recommendations from the evaluation. Lessons learned and best practices identified in this evaluation will be disseminated to stakeholders to be considered for future trilateral interventions that could be designed in USAID/CAM or other Missions. This Mid-Term Performance Evaluation is scheduled for the end of calendar year 2018, with approximately six months left of implementation of the base period (the activity’s base period ends on July 31, 2019). The evaluation will cover the period from August 1, 2016 through September 30, 2018. B. Background Information Since August 1, 2016, USAID/CAM has implemented the RTC activity through a task order. This award is among the first to leverage the skills, knowledge, technologies, and experiences of other countries in Latin America toward the development objectives of the regional Mission. Since its inception the RTC program has changed to focus more on providing logistics support, south￾south cooperation advisory services, and, at times, agenda design guidance. RTC is building upon past successes working trilaterally to move towards increased, more systemic trilateral 15 Although RTC is an innovative approach for USAID in doing business and promoting collaboration, this support service mechanism is not implementing untested interventions whose effectiveness should be assessed; therefore, the Agency will conduct an internal performance evaluation instead of a rigorous external impact evaluation as normally required by the USAID Evaluation Policy. 42 cooperation, ensuring all actors receive the greatest benefit possible from the partnerships. RTC does this by responding to the Concept Notes generated by USAID delineating Trilateral Activity agendas, specific activities, and relevant stakeholders, counterparts, and participants. Activity Implementing Partner Award Number and Dates Funding Regional Trilateral Cooperation (RTC) Activity Prime: International Business & Technical Consultants, Inc. (IBTCI) Subcontractor: Fundación Salvadoreña para el Desarrollo Económico y Social (FUSADES) AID-596-TO-16-00006 Base period: August 1, 2016-July 31, 2019 Option period: August 1, 2019-July 31, 2020 $ 5,620,555.46 The purpose of the RTC activity is to build trilateral cooperation16 among the United States, Central American countries, and selected SSC provider countries, primarily from Latin America, or other countries as relevant. The RTC activity is supporting the transfer of technical expertise and other resources between participating partner countries in the region, specifically in areas that align with, and contribute to, achievement of the U.S. Strategy for Central America (CEN Strategy), the Regional Development Cooperation Strategy (RDCS), the Country Development Cooperation Strategy (CDCS) of the recipient countries, and the Plan of Alliance for Prosperity in the Northern Triangle. Under this mechanism, IBTCI is developing, coordinating, and managing cooperation and exchanges among U.S. Government agencies (represented by USAID), provider countries (i.e., SSC countries and private organizations), and recipient countries. The RTC mechanism focuses on the following SSC provider countries: Chile, Colombia, Peru, Argentina, Costa Rica, Panama, Uruguay, and Mexico; and five recipient countries: El Salvador, Guatemala, Honduras, Nicaragua, and Mexico. RTC activity provides expert support to ensure effective, results-oriented exchanges between partner countries. One of IBTCI’s main tasks under this activity is to provide managerial, logistical, technical, and administrative support to implement short-term and medium-term study tours and other exploratory or pilot programs that address specific needs in the receiving country. This mechanism has a demand-driven approach, responding to USAID-identified opportunities. However, in general terms, some activities performed by IBTCI may include the following: • Organize complementary (extra-curricular) activities to facilitate dialogue, coordination, and stimulate exchange of ideas in multiple settings; and prepare a proposed design and recommend additional program components to supplement an offering country’s assistance, with USAID involvement and approval. 16 Trilateral Cooperation is a collaboration in which traditional donor countries and multilateral organizations facilitate South-South initiatives through the provision of funding training, and management and technological systems as well as other forms of support. 43 • Identify and recommend trilateral intervention options to address specific development problems. • Research, develop, and recommend concepts for potential trilateral cooperation based on a provider country specific expertise in line with missions’ programmatic areas and prepare draft documents for USAID’s use in negotiations between and among participating countries. • Assess existing operational procedures and practices that are yielding high results in selected sectors in provider countries that can be piloted in El Salvador and other participating countries. • Adapt or outright develop trilateral activities that can be piloted in Central America. Based on results of the piloted activities, and as requested by USAID/CAM or other relevant missions, support the scaling up of selected activities and pilot initiatives. • Identify and deploy short-term resource persons and Subject Matter Experts in priority sectors to facilitate and implement trilateral cooperation initiatives. • Facilitate travel and event logistical activities related to implementing events C. Evaluation Questions The following three main evaluation questions, in priority order, have been identified by USAID/CAM. The Evaluation Team should answer them and clearly present them in the Final Report in terms of how they relate to the evaluation purpose. The Evaluation Team should answer the sub-questions separately in the report and use the data collection and analysis of the sub-questions to answer the main questions. In addition, the Evaluation Team should incorporate, to the extent feasible, analysis of possible differences associated with gender or social groups, particularly historically excluded groups (youth, people with disabilities, indigenous populations, etc.), and report these separately for men and women. As an example, the Evaluation Team should consider gender or other social-related barriers when analyzing the obstacles to apply new knowledge in the country. 1. To what extent are the beneficiary organizations applying the new knowledge transferred by SSC provider countries through the RTC activity? 1.1 What are the hindrances and barriers that beneficiaries face in applying this newly transferred knowledge? Context for Question 2: Other donor countries such as Japan, Spain, or Germany have implemented trilateral cooperation projects in El Salvador. USAID has had trilateral exchanges in the past, but as a component within an activity (i.e., the Fiscal Policy and Public Expenditure Management Program facilitated exchanges with Brazil, and the Promoting Food Security and Trade Integration Program facilitated trilateral cooperation with Chile). 2. In terms of the use of resources such as cost and staff time, timeliness to respond to needs, and other factors, what are the advantages and disadvantages of a centralized approach like RTC's, and how do they compare to those of activities with trilateral exchanges as a component or sub￾component of a development project? 3. To what extent has the RTC activity contributed to building partnerships among the U.S. Government, the Government of the recipient country(ies), Private Sector and Civil Society of El 44 Salvador, Guatemala, and Honduras (Northern Triangle of Central America), and those SSC provider countries? 3.1 How do the above recipient countries perceive the trilateral cooperation facilitated by the RTC activity compared with traditional North-South Cooperation? 3.2 To what extent have the beneficiary organizations built networking with their counterparts or private sector in the SSC provider countries that may lead to future collaboration beyond the life of the RTC activity? D. Evaluation Methodology, Data Collection, and Analysis For this performance evaluation, a non-experimental mixed-methods design that combines a comprehensive, rigorous analysis of existing quantitative data with customized qualitative techniques designed to elicit primary data from a wide range of counterparts, partners, beneficiaries, and stakeholders is recommended. This approach allows for triangulation of complementary data to elucidate linkages between activity inputs and outputs. The Evaluation Team should consider a range of possible methods and approaches for collecting and analyzing the data to address the evaluation questions thoroughly. The use of participatory methods and activities that will enhance collaboration and dialogue among counterparts is required. Further, data collection and analysis methods should follow applicable Institutional Review Board (IRB) guidance on data security to ensure safety and confidentiality of all individuals providing data or information for the purposes of the evaluation. The finalized evaluation methods and approaches, data collection plan and analysis will be included in the Evaluation Plan submitted to USAID for revision and approval before field visits and data collection begin (see Deliverables section below). The method(s) proposed should comply with the USAID Evaluation Policy. The data collection plan for this evaluation will include, at a minimum: a desk review of relevant documents such as Task Order and work plans; review of activity performance monitoring data and financial data; and key informant interviews and/or focus group discussions promoting equal opportunity of participation of women and men. USAID/CAM expects both qualitative and quantitative data to be collected; and the results will be coded, triangulated, and analyzed for content. The Evaluation Team is encouraged to propose additional/alternate data collection and analysis methods in the Evaluation Plan that they consider can yield stimulating, robust evidence in answering each of the evaluation questions. Data collection shall be systematic and data must comply with the five data quality standards of validity, integrity, precision, reliability, and timeliness. Specific interview, survey, and/or focus group protocols will be appended to the Evaluation Plan and finalized with approval from USAID/CAM; the questions should be used to answer the evaluation questions listed in this document and address the purpose of this evaluation. All data collected in response to the evaluation questions must have as much level of disaggregation as possible. At a minimum, and per USAID Gender Equality and Female Empowerment Policy, all data must be disaggregated and analyzed by sex, as well as analyzed for any differences between effects on men and women or male and female participation. 45 Desk review of relevant documents USAID/CAM will provide the Evaluation Team with all relevant documents, such as the contract between USAID and IBTCI and its amendment(s) including the expected results, performance reports stating the results achieved, any prior assessments if applicable, etc. The Evaluation Team must review these documents and other existing literature provided by USAID/CAM and others in preparation for the initial team planning meetings and before meeting with local stakeholders for interviews. The Evaluation Team is expected to review these, make their own contextual literature research, and create a Review Matrix to be delivered to USAID/CAM as part of the final Evaluation Report. The Review Matrix should indicate how key information extracted from reviewed documents and other methodologies is linked to each evaluation question. At a minimum, the Evaluation Team shall review the following documents relevant to the Mid￾Term Performance Evaluation. USAID/CAM will provide these documents to the Evaluation Team: • USAID/CAM RDCS 2015-2019 (Link here) • RTC activity Task Order and its amendments • USAID Regional Central America Trilateral Cooperation Assessment (Link here) • USAID Assessment of International Donor Coordination in El Salvador (Link here) • RTC activity Monitoring, Evaluation, and Learning Plan • RTC activity Annual work plans (Link here) • RTC activity Quarterly and Annual reports • RTC Trilateral Cooperation interventions tracker and Concept Notes • Satisfaction Surveys • CEN Strategy (Link here) • Plan of Alliance for Prosperity in the Northern Triangle (Link here) Review of performance and financial data IBTCI has an activity-specific Monitoring, Evaluation, and Learning Plan that contains data collected on a number of performance indicators during activity implementation. The Evaluation Team will use monitoring data on performance indicators as part of the data analysis and should report on it in the Final Report as it relates to the evaluation questions stated above and satisfies relevant data quality standards. As part of its duties, IBTCI negotiates costs and resource sharing, and oversees, disburses, manages, and maintains accountability of program and operational expenses, including per diem payments for persons traveling from participating countries. When feasible, the Evaluation Team will use the financial data that IBTCI manages to analyze the efficiency of the RTC mechanism. Context data should be analyzed and included to the maximum extent possible when answering the evaluation questions. Key informant interviews, surveys, focus group discussions The Evaluation Team will interview stakeholders, through key informant interviews, group interviews, short surveys, and/or focus groups discussions. The Evaluation Team will include both men and women in the stakeholders’ consultation processes. USAID/CAM or IBTCI will provide key informant contact information once the evaluation begins. At minimum, the Evaluation Team will interview: 46 • Key USAID/CAM staff (Program Office: Program Officer and COR. Technical Offices: Team Leaders and A/CORs) • IBTCI staff • Representative(s) of the SSC provider countries [i.e., representative(s) of Cooperation Agencies and/or from counterpart institutions from the SSC provider countries] • Representative(s) of government bodies of countries that have received assistance through RTC • Representative(s) of the private sector, academia, Civil Society Organizations, and Non￾Governmental Organizations that have participated in trilateral interventions financed under RTC • USAID Implementing Partners that have implemented bilateral or regional trilateral cooperation interventions with and without support of RTC activity • Representative(s) of the Ministry of Foreign Affairs of El Salvador The Evaluation Team is encouraged to interview other stakeholders as needed involved with similar trilateral cooperation experiences (including other international donors) and other actors that can provide an insight into USAID programmatic impacts. A sampling plan describing the selection process; such as purposeful, random, or a combination of approaches;17 for organizations and stakeholders for key informant interviews, surveys, and focus group discussions must be included in the Evaluation Plan and Final Report. Site visits and direct observation In consultation with USAID/CAM, IBTCI, and other USAID Implementing Partners, the Evaluation Team will select, if applicable, relevant site visits for direct observation in El Salvador where new knowledge transferred through trilateral cooperation is being applied. Selection should be based on the sampling plan developed for the Evaluation Plan and included in the Final Report. The Evaluation Team may conduct direct observation at events hosted or sponsored in El Salvador by the RTC activity during the fieldwork period of the evaluation. The Evaluation Team can use these events to talk with stakeholders, conduct interviews, and collect additional data as evidence to answer the evaluation questions. USAID/CAM and IBTCI will provide the Evaluation Team with a list of events once the evaluation begins, if appropriate. Although RTC activity is a regional mechanism, it is not expected that the Evaluation Team travels to other countries in the region beyond El Salvador. Team planning meetings An initial team-planning/kick off meeting will be held in El Salvador between USAID/CAM and the Evaluation Team and between IBTCI and the Evaluation Team before the submission of the Evaluation Plan so that USAID/CAM can clarify any questions from the Evaluation Team, expectations, and guidelines. The expected results of this meeting are to: • Clarify each team member's role and responsibilities 17 Some sampling approaches include quota sampling, proximity sampling, convenience sampling, theoretical sampling, typical case sampling, etc. 47 • Confirm the anticipated timeline and deliverables • Discuss data collection tools and methodologies by evaluation question to be presented in the Evaluation Plan • Identify communications logistics and how the Evaluation Team, USAID/CAM, and IBTCI will communicate with each other Additional meetings may be held as deemed necessary by USAID/CAM and/or the Evaluation Team. E. Deliverables It is estimated that no more than 70 working days of services from the starting date of the evaluation (100 calendar days) will be required to complete a high-quality evaluation as required under this SOW. During that time frame, the evaluation team shall submit the following deliverables: 1. An Evaluation Plan, in Word Gill Sans font18 size 12, to be completed by the Evaluation Team after the Team Planning Meetings, no later than 15 calendar days after the starting day of the evaluation. USAID will receive the Evaluation Plan via electronic mail and review it to provide comments no later than five (5) working days after receiving the document. The Evaluation Plan will provide details of how the various deliverables, tasks, and activities will be undertaken. It must include at least: • RTC description; • Evaluation design,19 and the explanation of why one design or mix of designs proposed is the most appropriate, its limitations, and how these limitations will be addressed; • A matrix summarizing the following information per each evaluation question: - Method(s) for data collection, data source, the explanation of why one method or mix of methods is the most appropriate, its limitations and the ways to address them; - Technique(s) for data analysis,6 the explanation of why one analysis technique or mix of techniques is the most appropriate, its limitations and the ways to address them. • Data Management Plan describing the capture of data (for example, interview notes or live recording), storage and transfer, and how all data will be handled in such a manner as to protect the identities of informants in any situations where there are comments could potentially have a negative impact on their employment or security. • Timeline and/or Milestone Plan, including tentative starting time for data collection and duration of each activity conducted under the evaluation; 18 If the Evaluation Team does not have Gill Sans family available, they must use any other approved font as per the USAID Graphics Manual and Partners Co-Branding Guide. 19 Some examples of evaluation designs for performance evaluations include snapshot design, cross-sectorial design, before-and-after design, time series design, case study design, panel design, etc. 6 Some examples of data analysis techniques include parallel, conversion, sequential, multilevel, data synthesis, content analysis, contribution analysis, etc. 48 • Drafts of data collection protocols, such as questionnaires or focus group moderator guide(s), interview scripts, consent form,20 etc.; • Evaluation Team composition, roles, and responsibilities; • Location for the evaluation and Site visit plan. If the Evaluation Plan includes key informant interviews, surveys, and/or focus group discussions, the Evaluation Plan should include the following information: 1. How the interviews/surveys will help to answer the evaluation questions 2. Who will conduct the interviews/surveys and why they are qualified to do so? 3. What the rationale and methods are for deciding the number, timing, and location of the interviews/surveys 4. How the participants will be selected and recruited 5. How the interviews/surveys will be recorded 6. How the interview/survey data will be analyzed and presented The Evaluation Plan, particularly the data collection and analysis protocols, as well as interview and focus group guides, must be approved by USAID prior to the start of data collection and the field work. All interview protocols must be submitted in English and Spanish. The Evaluation Team will have another five (5) working days to make any changes. Once the Evaluation Plan is approved, the Evaluation Team will submit to USAID an electronic copy in PDF. Any subsequent change to the Evaluation Plan must be approved by USAID. The Evaluation Team may provide USAID and the Implementing Partner with a preliminary briefing on the Evaluation Plan prior to the beginning of data collection. 1. Brief weekly bullet reports of activities submitted to the manager of this evaluation by electronic mail due every Monday by the close of business. 2. A Preliminary Findings Briefing for USAID, IBTCI, and other stakeholders that USAID/CAM consider necessary on the preliminary findings identified by the Evaluation Team immediately after finalizing the data collection phase and before starting the draft report. According to the audience, the briefing may be conducted in English or Spanish. Only the Team Leader needs to be present for this briefing; however, local/regional Evaluation Team members may also attend. The Preliminary Findings Briefing will be used by the Evaluation Team as a feedback exercise to prepare the Draft of the Final Report. 3. A Draft of the Final Report in Word, Gill Sans font size 12, submitted for review due no later than 35 calendar days after the approval of the Evaluation Plan via electronic mail. The Program Office in USAID/CAM will be responsible for coordinating the peer-review process with different offices within USAID/CAM and USAID/Washington and for distributing it to the implementing partner and other stakeholders for comments. USAID/CAM will consolidate all comments and send the draft back to the Evaluation Team within 10 working days. At a minimum, and in accordance with the USAID 20 If underage persons (less than 18 years old) will participate in this performance evaluation, the Evaluation Team must make sure to comply with all national regulations related to Child Protection. 49 Evaluation Policy and ADS 201, the Final Report and its draft versions must include the following sections: • Executive Summary of the purpose, background, evaluation questions, findings, conclusions, and recommendations (no more than five pages); • Evaluation purpose and questions; • Thorough description of the evaluation design and any challenge/limitations,21 with emphasis on the timeliness and methods for data collection and data analysis; • Relevant data analysis tables; • Findings and conclusions drawn from the analysis of the findings;22 • Action-oriented, practical, and specific recommendations with defined responsibility for the action; • A dissemination plan of findings, conclusions, and recommendations to intended users of the evaluation, and; • Appendices: - Original SOW, annotated with any changes approved by USAID - Evaluation and data collection team composition and roles, with conflict of interest disclosures for all real or perceived conflicts of interest, if necessary - Data collection protocols and instruments including questionnaires and checklists - Review matrix of documents consulted - Meeting notes - Complete schedule of evaluation activities, meetings, and interviews - List of individuals and organizations contacted, and sites visited - Tables, graphs, pictures taken during site visits, maps USAID/CAM expects to receive a high-quality Draft Report from the Evaluation Team. USAID/CAM will assess the quality of the Draft Report using the Evaluation Report and Review Template. 4. Final Report in PDF, font Gill Sans size 12, no longer than 40 pages in its body, excluding the cover page; Table of Contents; List of Acronyms; and Appendices. The approved Final Report must adhere to USAID’s Evaluation Policy and ADS 201, Criteria to Ensure the Quality of the Evaluation Report,23 and must be submitted in English and Spanish and have incorporated USAID's comments, as appropriate. The Final Report will be due to USAID/CAM 10 working days after the Evaluation Team receives comments on the draft and no later than 90 calendar days after the start of the evaluation. Three high-quality printed, bound copies in English and Spanish of the Final Report must be submitted to USAID/CAM within 10 calendar days of acceptance of the Final Report. 21 The Evaluation Team must identify (a) steps taken to mitigate limitations, and (b) how/whether the limitations affect any particular finding, conclusions, or recommendations. 22 In moving from findings to conclusions, the analysis must be clear as to how findings are synthesized through different techniques such as divergence, convergence, and amalgamation; propensity; weighting; etc. 23 See ADS mandatory reference 201maa, available at https://www.usaid.gov/sites/default/files/documents/1870/201maa.pdf 50 5. A One-Page summary of the evaluation purpose, findings, conclusions, and recommendations. The One-Page summary will be prepared in English and Spanish in PDF. 6. Any raw data (qualitative or quantitative) collected in electronic form (DVD or flash drive, in original format of Word, Excel, etc.) is due no later than 100 calendar days after the starting date of the evaluation. As per ADS 540, the Evaluation Team must submit to the Development Data Library (DDL), in a machine-readable, non-proprietary format, a copy of any datasets, and their codebooks and metadata, that are used (or of sufficient quality) to produce an Intellectual Work. 7. Other deliverables as identified during the Team Meeting and agreed to by USAID/CAM and the Evaluation Team. All reports and papers will be considered draft versions until they are approved by USAID/CAM. These draft documents must be labeled with the word “DRAFT” in watermark. Findings must be presented as analyzed facts, strong qualitative and quantitative evidence and data, and not based on anecdotes, hearsay or the compilation of people’s opinion. To ensure unbiased findings, there is no guarantee that findings will be modified based on USAID suggestions. The Evaluation Team will research, investigate, and corroborate as objective any suggestion before it is incorporated in the findings, and the change will be noted in the draft document so as to have a record of the change. All submitted reports and presentations must be thoughtful, well-researched, and well-organized documents, and objectively answer the evaluation questions. When writing the report, the Evaluation Team must remember the different audiences. The style of writing should be easy to understand and concise, while making sure to address the evaluation questions and issues with accurate and data-driven findings, justifiable conclusions and practical recommendations.24 The Evaluation Team should clearly list any biases or limitations that exist during both data collection and analysis (selection bias, recall bias, unobservable differences between comparator groups, etc.). In addition, all real or possible conflicts of interest must be disclosed by each member of the Evaluation Team in writing. When quoting an individual in any report, the Evaluation Team must always give the context or circumstances of the quote. Correcting a grammatical error in the quote may be valid, but not rewording an entire phrase. When translating quotes from one language to another, the Evaluation Team should do so in an idiomatic way and care must be taken to ensure that the tone of the translation is equivalent to the tone of the original. Quotes should be presented in their original language in report texts. All reports must comply with the USAID Graphic Standards Manual and the ADS Style and Format Guide. Once a Final Report has been approved by USAID, the Evaluation Team will make it compliant and submit it to USAID’s Development Experience Clearinghouse (DEC).25 The Evaluation Team will email USAID the DEC link for the evaluation reports. USAID may attach a 24 For additional information on the criteria to ensure the quality of Evaluation Reports, see the USAID’s How-To Note: “Preparing Evaluation Reports”, and Annex 1 of the USAID Evaluation Policy 25 Per Automated Directives System 540, documents and development assistance projects materials produced or funded by USAID must be submitted for inclusion in the DEC. 51 Statement of Differences as an Annex to any Final Report if any differences remain in the final version. F. Evaluation Management Evaluation Team This performance evaluation will use a combination of multidisciplinary international, regional, and/or local experts. The Evaluation Team must include at a minimum the following two positions: Evaluation Team Leader Minimum Qualifications Education: Master’s degree in fields such as International Relations, Political Sciences, Public Administration, Economics, or other fields related to international development is required. Ph.D. or doctorate degree or professional with Doctoral candidacy is a plus. Formal training in monitoring and evaluation is a plus. Language Proficiency: American English Level IV and Spanish Level III Work Experience: At least five years of relevant prior experience conducting rigorous external evaluations using both quantitative and qualitative methods for development objectives and monitoring projects and programs overseas, preferably in Latin America. Knowledge and detailed understanding of Central America’s economic, social, cultural, and political characteristics and its development environment is necessary. Working experience in Central America is a plus. At least eight years of project management experience in development is required. Experience with management of multidisciplinary teams is a plus. Familiarity with USAID’s objectives, approaches, operations, and policies, particularly as they relate to evaluations is a plus. Role: The Evaluation Team Leader will be responsible for overseeing and coordinating all activities related to this performance evaluation and for ensuring the production and completion of quality deliverables in a professional manner, in conformance with this SOW. International Cooperation Specialist Minimum Qualifications Education: Master’s degree in fields such as International Relations, Political Sciences, Public Administration, Economics, or other fields related to international development is required. Ph.D. or doctorate degree or professional with Doctoral candidacy is a plus. Formal training in monitoring and evaluation is a plus. Demonstrated knowledge of the theory framework of International Cooperation for Development is necessary. Language Proficiency: American English Level III and Spanish Level IV. Work Experience: At least five years of progressively responsible, professional-level experience conducting organizational capacity assessments, cost-effectiveness analyses of international cooperation projects, or other similar studies related to this performance evaluation. Experience with the application of methodologies for social science research is preferred. Familiarity with USAID’s objectives, approaches, and operations, particularly as they relate to evaluations is a plus. Work experience in Central America is preferred. 52 The Evaluation Team should have considerable experience in designing, monitoring, and evaluating development assistance programs. They must have proficiency in MS Office Suite and have excellent written, proofreading, and oral presentation skills and the ability to conceptualize and write clearly and concisely and attention to details. Understanding of the Latin American context is necessary, with a preference for personnel with work experience specifically in Central America. At least one key personnel member must have experience working in gender and social inclusion issues in a development context, transforming qualitative data, analyzing quantitative data, and producing data visualization in an easily digestible format. All Team members will be required to provide in advance to USAID a signed statement indicating any conflict of interest. The Team Leader must be someone external to USAID. No Evaluation Team members shall have been directly involved in the implementation of the RTC activity in any stage. Anyone who has been directly employed by USAID/CAM, IBTCI, or FUSADES in the last five years must not be considered as part of the Evaluation Team. Logistics The Evaluation Team will be responsible for all logistics support under this SOW, including field office administration, all travel arrangements (with required USAID/CAM clearances), team planning facilitation and appointment scheduling, coordination with all partners and stakeholders involved, administrative services (computer support, printing and copying), report editing and dissemination, and for complying with provisions set forth in this SOW. USAID/CAM will provide limited support to the Evaluation Team. This support, if needed, may include assistance in arranging high-level meetings; access to the U.S. Embassy compound in El Salvador, as necessary; and access to all reports, data, and other relevant documents created by IBTCI. USAID representatives may accompany the Evaluation Team for some or all the evaluation, especially in high level meetings. The Evaluation Team is expected to consider this when making logistical arrangements. 53 ANNEX B: BIBLIOGRAPHY 54 RTC Project Documents (N=59) Regional Trilateral Cooperation Activity, “All USAID Funded Participants,” (RTC Stakeholder List), October 16, 2018. Regional Trilateral Cooperation Activity, “Annual Work Plan – Year 1,” Submitted December 7, 2016. Regional Trilateral Cooperation Activity, “Annual Work Plan – Year 2,” Submitted August 31, 2017; Re-Submitted January 15, 2018; Resubmitted March 28, 2018. Regional Trilateral Cooperation Activity, “Branding Implementation and Marking Plan,” Submitted March 14, 2017. Regional Trilateral Cooperation Activity “Citizen Security – Tertiary Prevention.” Regional Trilateral Cooperation Activity “Civil Service Reform Merit Based Model Part I.” Regional Trilateral Cooperation Activity “Civil Service Reform Merit Based Models Part II.” Regional Trilateral Cooperation Activity “Electricity Strengthening.” Regional Trilateral Cooperation Activity, “Final Assessment Report,” Submitted April 11, 2017. Regional Trilateral Cooperation Activity, “First Annual Performance Report. August 6, 2016 – September 30, 2017,” Submitted November 15, 2017. Regional Trilateral Cooperation Activity “Fiscal Policy Cash Flow Models Part X.” Regional Trilateral Cooperation Activity, “Fiscal Policy Diagnosis of El Salvador Customs Port System Part I.” Regional Trilateral Cooperation Activity “Fiscal Policy – International Public Sector Accounting Standards Part VI.” Regional Trilateral Cooperation Activity “Fiscal Policy Result Oriented Budget Part IX.” Regional Trilateral Cooperation Activity, “Fiscal Policy – Reverse Auctions and Public Procurement Part III.” Regional Trilateral Cooperation Activity, “Fiscal Policy – Simplified Tax System Part V.” Regional Trilateral Cooperation Activity “Fiscal Policy -Single Tax Payer System Part V.” Regional Trilateral Cooperation Activity, “Fiscal Policy – Tax Collection and Anticorruption Part IV.” Regional Trilateral Cooperation Activity, “Fiscal Policy – Transfer Pricing Control System Part III.” Regional Trilateral Cooperation Activity, “Gira de Estudios Medellín-Ruta N, del 3 al 9 de Septiembre de 2017,” September 2017. Regional Trilateral Cooperation Activity “Government Integrity Project Request for Ethics Week 2018.” Regional Trilateral Cooperation Activity “High Risk Courts Models.” 55 Regional Trilateral Cooperation Activity “Innovation Roadmap Part III.” Regional Trilateral Cooperation Activity “Innovation Ruta N Part I.” Regional Trilateral Cooperation Activity “Innovation Ruta N Part II.” Regional Trilateral Cooperation Activity “IOM Regional Conference for Returning Migrants.” Regional Trilateral Cooperation Activity, “Integrity Evaluation and Control Center PNC.” Regional Trilateral Cooperation Activity, “Interventions Completed as of October 22, 2018,” October 22, 2018. Regional Trilateral Cooperation Activity “Low Emission Cattle Farming.” Regional Trilateral Cooperation (RTC), “Monitoring, Evaluation and Learning (ME&L) Plan Year 1,” Final Submitted July 10, 2017. Regional Trilateral Cooperation Activity “Natural Gas.” Regional Trilateral Cooperation Activity “Open Government Partnership.” Regional Trilateral Cooperation (RTC), “Quarterly Report, January-March 31, 2016,” Submitted May 1, 2017. Regional Trilateral Cooperation (RTC), “Quarterly Report, October 1-December 31, 2016,” Submitted January 31, 2016. Regional Trilateral Cooperation (RTC), “Quarterly Report, April 1-June 30, 2017,” Submitted August 15, 2017. Regional Trilateral Cooperation (RTC), “Quarterly Report, September-November 2017,” Submitted January 12, 2017. Regional Trilateral Cooperation (RTC), “Quarterly Report, March 2018-May 2018,” Submitted January 15, 2017. Regional Trilateral Cooperation Activity “Regional Trade Facilitation – Applying Information Technology Part X.” Regional Trilateral Cooperation Activity, “Regional Trade Facilitation – Customs Risks Management Part V.” Regional Trilateral Cooperation Activity “Regional Trade Facilitation – Customs Risk Management Strategy Finalization Part VIII.” Regional Trilateral Cooperation Activity, “Regional Trade Facilitation – Customs Union Integration Part IV.” Regional Trilateral Cooperation Activity, “Regional Trade Facilitation One Stop Shop Customs Border Crossing Part 2.” Regional Trilateral Cooperation Activity “Regional Trade Facilitation US-Mexico Customs and Border Facility Part IX.” Regional Trilateral Cooperation Activity, “Regional Trade Facilitation Public-Private Inspection Certification Part 1.” 56 Regional Trilateral Cooperation Activity “Regional Trade Facilitation – Risk Management SIECA Part VI.” Regional Trilateral Cooperation Activity “Regional Trade US Law Equivalences Part VII.” Regional Trilateral Cooperation Activity, revisions to “Annex II: Stakeholder Survey Questionnaire,” conducted by the USAID Monitoring, Evaluation, and Learning Initiative for the Mid-Term Performance Evaluation of the USAID Regional Trilateral Cooperation Activity, October 2018. Regional Trilateral Cooperation Activity “Sanitary Product Registration.” Regional Trilateral Cooperation Activity, “Second Annual Performance Report, October 1, 2017 – September 30, 2018,” submitted November 15, 2018. Regional Trilateral Cooperation Activity, “Summary Report: Regional Conference on Reintegration of Returning Migrants: San Salvador, El Salvador, May 8-9, 2018,” Submitted July 2018. Regional Trilateral Cooperation Activity, “Summary Report: Regional Conference on Integration of Returning Migrants: San Salvador, El Salvador May 8-9, 2018,” July 2018. Regional Trilateral Cooperation Activity, “Trade Facilitation Authorized Economic Operator Part III.” Regional Trilateral Cooperation Activity, “Tri-Lateral Activity Post-Trip Brief. AMEXCID IV Facilitación,” January, 2018. Regional Trilateral Cooperation Activity, “Tri-Lateral Activity Post-Trip Report. Mexican Electrical Market, March 15-16, 2018.” Regional Trilateral Cooperation Activity, “Tri-Lateral Activity Post-Trip Report. Civil Service, June 24-29, 2018. Santiago de Chile, Chile.” Regional Trilateral Cooperation Activity, “Tri-Lateral Activity Post-Trip Report. Visit od the Runa N and the Chamber of Commerce from Medellin to El Salvador. July 23-27, 2018.” Regional Trilateral Cooperation Activity, “Tri-Lateral Activity Post-Trip Report. Sanitary Workshop. Dates: August 14-15, 2018. Tegucigalpa, Honduras.” Regional Trilateral Cooperation Activity, “USAID Regional Trilateral Cooperation Activity Inventory”, October 11, 2018. Regional Trilateral Cooperation Activity “Youth Workforce.” Other Documents Reviewed (N=49) A. Abdenur, “The Strategic Triad: Form and Content in Brazil’s Triangular Cooperation Practices,” International Affairs Working Paper 1007/6. New York: New York University, 2007. AECID, ‘‘Triangular Cooperation in the Context of Aid Effectiveness: Experiences and Views of European Donors,” Ministerio de Asuntos Exteriores y de Cooperacion. Madrid: Government of Spain, 2010. http://ec.europa.eu/development/icenter/files/europa_only/ featured_triangular_cooperation_concept_note_20100309.pdf 57 Adriana Erthal Abdenur and João Moura Estevão Marques Da Fonseca, “The North’s Growing Role in South-South Cooperation: keeping the foothold, Third World Quarterly,” 2013. Alice Bancet, Instituto Universitario de Desarrollo y Cooperación; Universidad Complutense de Madrid; Universidad del País Vasco; Universidad de Murcia, “Análisis de la Cooperación Triangular: Discursos y prácticas de los países del CAD/OCDE sobre una modalidad en construcción,” Madrid, January 2012. Chery McEwan and Emma Mawdley, “Trilateral Development Cooperation: Power and Politics in Emerging Aid Relationships, Development and Change, Volume 43, Issue 36, 1185-1209, November 2012. E. Pantoja and C. Elsner (2009) “Triangular Cooperation: New Paths to Development. Summary Report of the Discussions and Experiences presented in the first International Symposium on Triangular Cooperation, Triangular Cooperation Programme, GTZ-Brazil.” Paris: OECD. http://www.oecd.org/dataoecd/63/32/43705278.pdf Economic Commission for Africa (ECA) Southern Africa Office (SRO-SA), “South-South and Triangular Cooperation. Implications for Southern African Countries,” June 2011. Federal Ministry for Economic Cooperation and Development, “Regional Fund for Triangular Cooperation in Latin America and the Caribbean,” July11, 2018. Federal Ministry for Economic Cooperation and Development, Z Future-makers, “Triangular cooperation in German development cooperation. Position paper. BMZ Strategy Paper,” May 2013. Food and Agriculture Organization of the United Nations (FAO), “FAO’s South-South and Triangular Cooperation Strategy in Action. Fostering Partnerships among the Global South,” 2016. Conference Summary. Innovation Conference. Alliance Academy-Business. FUSADES, “Cooperación Trilateral IBTCI/USAID. Adhesión de El Salvador a la Unión Aduanera Guatemala y El Salvador, 24 al 28 de julio 2017,” Informe: Pedro Argumedo. July 2017. FUSADES, Programa de Innovación Tecnológica, PROINNOVA, “Proyecto regional de USAID para la cooperación trilateral. Informe Técnico” [Ruta N], Samuel Salazar Genovez, September 2017. G. Ashoff, “Triangular Cooperation: Opportunities, Risks and Conditions for Effectiveness,” in “Special Report Development Outreach,” pp. 22–24, 2010, Washington, DC: World Bank Institute. http://siteresources.worldbank.org/WBI/Resources/213798– 1286217829056/ashoff.pdf H. Kharas, “The New Reality of Aid,” Brookings Blum Roundtable. Washington, DC: Brookings Institute, 2007. H. Rhee, “Promoting South-South Cooperation Through Knowledge Exchange,” in H. Kharas, K. Makino and W. Jung (eds) (2011) Catalysing Development: A New Vision for Aid, pp. 260–80. Washington, DC: Brookings Institute, 2011. IFAD, International Fund for Agricultural Development, “China-IFAD South-South and Triangular Cooperation Facility.” 58 “Informe Taller de Autoridades Regulatorias y Sector Privado Medicamentos y Cosméticos,” José Vicente Coto, August, 2018. Instituto de Acceso a la Información Pública, “Encuentro Regional de las Américas de la Alianza para el Gobierno Abierto. Informe al Pleno,” presentado por: Jorge Martínez, Unidad de Informática; Vicente Hernández, Unidad de Acceso a la Información Pública; Max Mirón, Comisionado Propietario. December 22, 2017. L. Cabral and J. Weinstock, “Brazil: An Emerging Aid Player,” Overseas Development Institution (ODI) Briefing Paper 64. London: ODI. McEwan and Mawdsley, Development and Change: Published on behalf of the Institute of Social Studies, The Hague, “Trilateral Development Cooperation: Power and Politics in Emerging Aid Relationships,” 2012. Ministerio de Hacienda, Dirección General de Tesorería, Gobierno de El Salvador, “Información General de la Pasantía [en Argentina],” April, 2018. Ministerio de Relaciones Exteriores, Dirección de Cooperación para el Desarrollo, Gobierno de El Salvador, “Panorama de la Cooperación para el Desarrollo. Boletín 01-julio 2018.” N. Kumar, “South-South and Triangular Cooperation in Asia-Pacific: Towards a New Paradigm in Development Cooperation.” Keynote address at the Asia-Pacific Development Cooperation Forum: Regional Workshop on Trends and Progress in Triangular and South-South Cooperation, Bangkok (21-22 October, 2018). New York: United Nations. http://www.un.org/en/ecosoc/newfunct/pdf/background%20study%20final.pdf Nadine Piefer, Technical University of Darmstadt, “Triangular Cooperation – Bridging South￾South and North-South Cooperation.” Paper prepared for the “Workshop on South-South Development Cooperation,” University of Heidelberg, September 26-27, 2014. OECD Cooperation Directorate, DAC Global Relations. The Development Assistance Committee: Enabling effective development, “Triangular Cooperation: What’s the literature telling us? Literature review,” May 2013. OECD Development Cooperation Directorate, “Dispelling the myths of triangular co-operation – Evidence from 2015 OECD survey on triangular cooperation,” September 2015. Pradeep S Mehta and Nitya Nanda, “Trilateral Development Cooperation: An Emerging Trend,” November 19, 2004. Programa Regional para la Cooperación Trilateral – USAID – RTC – IBTCI. “Desayuno Taller de Recopilación y Análisis de Iniciativas de Cooperación para la Prevención de la Violencia. Avance y Desempeño.” Proyecto regional de USAID para la Cooperación Trilateral, “Viaje de estudio – Colombia – Actividades de desarrollo laboral y empleo para jóvenes en riesgo, Bogotá-Medellín, 13-19 de noviembre, 2017.” November 24, 2017. Proyecto de USAID para la Cooperación Trilateral, “Informe Actividad El Salvador Protección a la Niñez Migrante. Presentación de Agenda de Viaje de Estudio Protección a la Niñez El Salvador. Mejores Prácticas Tijuana-San Diego,” Facilitador: Juan Francisco Rivas Figueroa, March 21, 2018. Proyecto Regional de USAID para la Cooperación Trilateral, “Guía de evaluación: organizador local + internacional.” 59 “Reporte de la Pasantía en CONEVAL, México.” January 2018. Secretaría General Iberoamericana, “Informe de la Cooperación Sur-Sur en Iberoamérica 2017. Resumen Ejecutivo,” 2017. Secretaría Técnica y de Planificación, Gobierno de El Salvador, “Memorandum – Informe de Misión Servicio Civil de Chile,” June 2018. Shunichiro Honda, JICA Research Institute, “Looking into Triangular Cooperation Mechanisms. Japan’s Triangular Cooperation Mechanism with a Focus on JICA’s Activities,” April 2014. Shunichiro Honda and Mihoko Sakai, JICA Research Institute. BMZ, Federal Ministry for Economic Cooperation and Development, JICA, UKaid, “Looking into Triangular Cooperation Mechanisms. Triangular Cooperation Mechanisms. A Comparative Study of Germany, Japan and the UK,” April 2014 T. Altenburg and J. Weikert, “Trilateral Development Cooperation with New Donors,” DIE Briefing Paper 5/2007. Bonn: Deutsche Institut fur Entwicklungspolitik. T. Chahoud, “Financing for Development Series: Southern Non-DAC Actors in Development Cooperation,” DIE Briefing Paper 13/2008. Bonn: Deutsche Institut fur Entwick-lungspolitik, 2008. T.Y. Fordelone, “Triangular Cooperation and Aid Effectiveness,” OECD-DAC. Paper prepared for Policy Dialogue on Development Cooperation, Mexico City (September 28-29, 2009). Paris: OECD. http://www.oecd.org/dataoecd/63/37/43704891.pdf United Nations Development Programme – South-South Cooperation, “Trilateral Cooperation with China. Sharing China’s Development Experience through Innovative Partnerships. Discussion Paper,” August 2016. United Nations – Meetings Coverage and Press Releases, “High-Level Committee on Review of Technical Cooperation Among Developing Countries Concludes Eleventh Session,” Press Release, June 4, 1999. United Nations Office for South-South Cooperation, “Good Practices in South-South and Triangular Cooperation for Sustainable Development,” May 2016. United Nations, “Strengthening the Global Partnership for Development in a Time of Crisis. Millennium Development Goal 8. MGD Gap Task Force Report 2009,” New York, 2009. USAID/Central America, “RTC Statement of Work.” USAID/Central America, “Statement of Work (SOW), Mid-Term Performance Evaluation of the USAID Regional Trilateral Cooperation Activity,” September 2018. USAID/El Salvador – USAID Monitoring, Evaluation, and Learning Initiative, “Assessment of International Donor Coordination in El Salvador,” October 2017. USAID/El Salvador, Monitoring, Evaluation, and Learning Initiative, “USAID Regional Trilateral Cooperation Activity Performance Evaluation Work Plan,” October 12, 2018. Viceministerio de Cooperación para el Desarrollo, Dirección General de Cooperación para el Desarrollo, Programa de Fortalecimiento de la Cooperación Sur-Sur El Salvador-Luxemburgo, 60 “Promoviendo la Cooperación Sur-Sur y Triangular en América Latina y el Caribe desde un marco de horizontalidad, reciprocidad y equidad.” World Food Program, “South-South and Triangular Cooperation for food security and nutrition. Snapshot of WFP practices in facilitating South-South and triangular cooperation to promote progress towards Zero Hunger (SDG2),” October 2016. 61 ANNEX C: LIST OF KII PARTICIPANTS 62 No Name Institution and Position Country E-mail Trilateral Exchange RTC 1 Carmen Eugenia Domínguez RTC/IBTCI – Administrative and Financial Manager El Salvador cdominguez@ibtci.com N/A 2 Karina de Lasala RTC/IBTCI – Project Manager El Salvador klasala@ibtci.com N/A 3 Aracely Margarita Villalta RTC/IBTCI – Logistics Manager El Salvador avillalta@ibtci.com N/A 4 Janina Khayali USAID/El Salvador Project – Project Manager Regional Trilateral Cooperation Project (RTC) El Salvador jkhayali@ibtci.com N/A 5 Rick Clark RTC, Former Project Manager El Salvador rlclarkwork@yahoo.com N/A 6 Kirk Dahlgren RTC Consultant El Salvador kirkmdahlgren@gmail.com N/A 7 Gloribel Hernández Ruíz Non-Governmental Organization (NGO)-FUSADES￾Special Projects/Monitoring and Evaluation-Cooperation Management El Salvador ghernandez@fusades.org Innovation Ruta N Part I 8 Margarita Sanfeliu NGO-FUSADES – Director Center for Research and Statistics El Salvador msanfeliu@fusades.org Innovation Ruta N Part I USAID/El Salvador 9 Joshua Mike USAID/El Salvador – Acting Economic Growth Office Director El Salvador jmike@usaid.gov Sanitary Product Registration 10 Martin Schulz USAID/El Salvador – Contracting Officer Representative (COR)- Domestic Revenue Mobilization Project El Salvador mschultz@usaid.gov Fiscal Policy: International Public-Sector Accounting Standards Part VI 11 Lorena Aceto USAID/El Salvador – COR-New Trade Facilitation Activity El Salvador laceto@usaid.gov Sanitary Product Registration/Food Safety 12 María Antonieta Zelaya USAID/El Salvador – COR￾Government Integrity Project El Salvador mazelaya@usaid.gov Civil Service Reform/Merit￾Based Models Part I 13 Gracia López USAID/El Salvador – COR￾Justice Sector Strengthening Project El Salvador glopez@usaid.gov Integrity Evaluation and Control Center (PNC) 14 Julliette Grier￾Villatte USAID/El Salvador – Agreement Officer Representative (AOR) for Whole Child International El Salvador jgriervillatte@usaid.gov 63 No Name Institution and Position Country E-mail Trilateral Exchange implementing the Protection and Quality of Care for Children Project 15 Christopher Moore USAID/El Salvador – Leader of Private Sector Team (former COR of Bridges to Employment) El Salvador chrmoore@usaid.com Innovation Ruta N Part I 16 Annie G. de Valencia Annie G. de Valencia USAID - COR-Regional Trilateral Cooperation Project (RTC) El Salvador avalencia@usaid.gov Innovation Ruta N Part I 17 Carlos Arce USAID/El Salvador – COR￾Economic Competitiveness Project El Salvador carce@usaid.gov Innovation Ruta N Part I 18 Sandra Duarte USAID/El Salvador – COR￾Higher Education for Economic Growth Activity El Salvador sduarte@usaid.gov Innovation Ruta N 19 Gregory Swarin USAID/El Salvador (Regional Program Office Director) El Salvador gswarin@usaid.gov N/A USAID IPs 20 Enrique Giraldo DAI (Chief of Party (COP)- Domestic Revenue Mobilization Project) El Salvador enrique_giraldo@dai.com Fiscal Policy: Results Oriented Budget Part IX 21 Iris Palma USAID/El Salvador Project - Innovation and Technology Manager for the USAID Economic Competitiveness Project El Salvador iris.palma@thepalladiumgroup.com Innovation: RUTA N Part II/ Innovation: Roadmap Part III 22 Reina Durán Higher Education for Economic Growth El Salvador rduran@es.rti.org Innovation Ruta N 23 Javier Agosto Former COP – Justice Sector Strengthening Project El Salvador jagosto@checchiconsulting.com Integrity Evaluation and Control Center (PNC) 24 Paola Barragán COP USAID/El Salvador Government Integrity Project El Salvador paola.barragan@pro-integridad.com Civil Service Reform/Merit￾Based Models Part I; Civil Service Reform Part II 25 Caterina Valero COP - Bridges to Employment El Salvador cvalero@puentesempleo.com Planning a trilateral exchange with Colombian experts from SENA (Colombia’s National Learning Service). 64 No Name Institution and Position Country E-mail Trilateral Exchange Trilateral Exchange Beneficiaries: GOES 26 Mario Menéndez GOES – Ministry of Finance – Tax Debt Collection Director El Salvador mario.menendez@mh.gob.sv Fiscal Policy: Tax Collection and Anti- Corruption (Part IV) 27 German Lisandro Garay GOES – Ministry of Finance – Tax Debt Collection Deputy Director El Salvador german.garay@mh.gob.sv Fiscal Policy: Tax Collection and Anti- Corruption (Part IV 28 Jeremías Antonio Aguilar GOES – Ministry of Finance – Tax Debt Collection Deputy Director El Salvador jeremias.aguilar@mh.gob.sv Fiscal Policy: Tax Collection and Anti- Corruption Part IV 29 Carlos Eduardo Alvarenga GOES – Ministry of Finance - Deputy Director Investigative Intelligence Unit-Transparency and Anticorruption Unit El Salvador carlos.alvarenga@mh.gob.sv Fiscal Policy: Tax Collection and Anti-Corruption Part IV 30 Ernesto Zúniga GOES/Technical and Planning Secretariat of the Presidency – Director of Civil Service Professionalization for the Executive El Salvador ezuniga@presidencia.gob.sv Civil Service Reform/Merit￾Based Models Part I 31 Liz Aguirre GOES-Ministry of Public Works – Access to Public Information Officer El Salvador liz.aguirre@mfusades.orgop.gob.sv Open Government Partnership 32 Martha Cea GOES - National Directorate of Medicines-Head of Imports, Exports and Donations El Salvador marta.cea@medicamentos.gob.sv Sanitary Product Registration/Food Safety 33 Jorge Martínez GOES - Access to Public Information Institute - Head of IT Unit El Salvador jemartinez@iaip.gov Open Government Partnership 34 Vicente Hernández GOES - Access to Public Information Institute-Vicente Orlando Hernández Melara – Information and Accountability Officer El Salvador vhernandez@iaip.gov Open Government Partnership 35 Luis Alberto Martínez Ramírez GOES – Ministry of Finance (Auditor) El Salvador alberto.martinez@mh.gob.sv Transfer Pricing Control System Part III 36 Alexander Palma GOES – Ministry of Finance El Salvador alexander.palma@mh.gob.sv US-Mexico Customs and Border Facility Part IX 65 No Name Institution and Position Country E-mail Trilateral Exchange 37 Edgar Alejandro Huezo Saavedra GOES – MINEX – Head of South-South and Triangular Cooperation Projects at the General Directorate of Cooperation for Development, Vice Ministry of Cooperation for Development El Salvador eahuezo@rree.gob.sv IOM Regional Conference for Reintegration of Returning Migrants Trilateral Exchange Beneficiaries: CSOs 38 Tony Carbonero Private – Entrepreneurship￾Innovation – Executive Director of “Inbox” El Salvador tony.carbonero@innbox.sv Innovation Ruta N Part I 39 Alfredo Atanacio Cáder Private –-Innovation￾Entrepreneurship- Creator of UASSIST.ME El Salvador alfredoa@uassist.me Innovation Ruta N Part 1 Trilateral Exchange Providers 40 Julián Taborda Araque Private –-Innovation￾Entrepreneurship Senior Consultant for ROADMAP Colombia julian.taborda@roadmapinnovacion.com Innovation Ruta N Part 1; Innovation: Roadmap Part III 41 Adriana Martínez COFEPRIS (Federal Commission for the Protection against Sanitary Risks-Mexico) Pharmaceutics and Medicines Deputy Director￾Mexico dmartinez@cofepris.gob.mx Sanitary Product Registration/Food Safety 42 Fernando Masumura Perú Procurement, Head of Public Procurement Office Perú fernando.masumura@perucompras.gob.pe Implementation of a modern Public Procurement System Part VIII 43 Varley Diaz Souza ANVISA - Brazilian Health Regulatory Agency Brazil Varley.Sousa@anvisa.gov.br Sanitary Product Registration/Food Safety 44 Christian Cesar Chocano Davis OSCE – Perú’s State Contracting Supervising Organization Perú cchocano@osce.gob.pe Reverse Auctions and Public Procurement Part II 45 William García SIECA – Economic Integration and Trade Facilitation Director SIECA wgarcia@sieca.int US-Mexico Customs and Border Facility Part IX Donor/Cooperation Agencies 46 Lisseth Lipa Cano Management and International Negotiation Directorate - Perú llipa@apci.gob.pe Implementation of a Modern Public Procurement System Part VIII 66 No Name Institution and Position Country E-mail Trilateral Exchange Peruvian Agency for International Cooperation 47 Denis Pigot Luxembourg Development Cooperation Agency (LuxDev) – El Salvador Project Coordinator Luxembourg denis.pigot@luxdev.lu N/A 48 José Luis Cabezas Sañudo AECID – Spain’s International Cooperation Agency for Development - Responsible of Governance and Gender Programs – Technical Cooperation Office Spain jose.cabezas@aecid.es N/A 49 Natalie Bartelt Director of GIZ’s Regional Fund for Triangular Cooperation in Latin American and the Caribbean Germany natalie.bartelt@giz.de N/A 50 Pierre-Yves Baulain Head of Cooperation – European Union Delegation to El Salvador European Union pierre-yves.baulain@eeas.europa.eu N/A 51 Pedro Andres Amo Senior Private Sector Specialist - IFC/World Bank Group IFC/EB pandresamo@ifc.org Sanitary Product Registration/Food Safety 67 ANNEX D: KII DISCUSSION GUIDES 68 DISCUSSION GUIDE FOR RECIPIENTS Name: Organization: Position: Telephone: Email: trilateral exchanges: INTRODUCTION • We are evaluating the USAID Regional Trilateral Cooperation Activity (or RTC) implemented by IBTCI. We would like to discuss your experience with and perceptions of the USAID-facilitated trilateral cooperation activities in which you participated as part of the RCT Activity. • For the purpose of this interview, a trilateral cooperation activity is taken to be an activity aimed at building relationships and facilitating cooperation between 1) one or more organizations (e.g., donor organizations, government ministries, private enterprises, civil society organizations) with 2) other organizations in other countries to facilitate a sharing of knowledge and expertise or otherwise promote development across two or more countries. • Your participation is entirely voluntary and whatever information that you provide will be kept strictly confidential. • Do you have any questions? • May we begin? EQ1 ⇒ Please describe the trilateral cooperation activities in which you participated? o What were the objectives of these activities? o What was your personal role in these in these activities? o Who else participated and how? ⇒ Why did you participate in these activities? What did you hope to achieve? ⇒ To what extent did you achieve what you had hoped to achieve? ⇒ What new knowledge or expertise did you acquire as a result of these trilateral activities? ⇒ How are you applying the new knowledge or expertise acquired as a result of these trilateral cooperation activities in your job or in your organization? ⇒ How has applying this new knowledge or expertise affected you and your organization? 69 ⇒ What other benefits did you receive from participating in these trilateral cooperation activities? ⇒ In the absence of these trilateral cooperation activities, would you have acquired this knowledge or expertise on your own? Why or why not? EQ1.1 ⇒ What are the primary hinderances or barriers you face in applying the new knowledge or expertise you acquired in these trilateral cooperation activities? ⇒ What additional assistance do you need in applying this new knowledge and expertise? EQ2 ⇒ How well were the USAID trilateral cooperation activities managed? What was done well? What was done less well? How could it be improved? In answering this question, please consider the following: o Time commitment o Timeliness or relevance of topics covered to you, your organization, and your country o Expertise of experts provided knowledge or expertise o Usefulness of knowledge or expertise provided o Overall management and organization ⇒ Have you participated in cooperation activities similar to this in the past? These may include either traditional North-South cooperation activities or regional cooperation activities that are sponsored by USAID, Deutsche Gesellschaft für Internationale Zusammenarbeit GmbH (GIZ), Japan International Cooperation Agency (JICA), or other donors or development organizations. If Yes, please describe them? (A North-South Cooperation activity is one in which countries from the Latin America and Caribbean Region partner or otherwise cooperate with the United States to receive knowledge or expertise about important topics. A regional relationship/cooperation activity is one in which countries from the Latin America and Caribbean Region partner or otherwise cooperate with each other countries in the region to share knowledge or expertise about important topics.) ⇒ How did these cooperative partnership activities compare to the USAID trilateral cooperation activities in terms of the following? o Time commitment o Timeliness or relevance of topics covered to you, your organization, and your country o Expertise of experts provided knowledge or expertise o Usefulness of knowledge or expertise provided o Overall management and organization 70 EQ3 ⇒ To what extent have the USAID trilateral cooperative activities contributed to building cooperative relationships between you and other persons, either in your country, another country in the Latin America and Caribbean region, or the United States? Please describe these cooperative relationships. [A cooperative relationship is a working relationship between two or more people or organizations in which they share information or expertise with each other on topics of mutual interest. A cooperative relationship need not be formalized with a Memorandum of Understanding (MOU) or any other formal agreement and may be, but need not be, entirely informal in nature.] ⇒ To what extent would these cooperative relationships would have occurred in the absence of the USAID trilateral cooperation activities? ⇒ How well do these cooperative relationships function? What benefits are they providing you? Your organization? Other relationship partners? ⇒ What are the primary hinderances or barriers you face in forming such cooperative relationships? ⇒ Overall, how effective were the USAID trilateral cooperation activities in fostering cooperative relationships between activity participants? Why? How could the activities be improved to better foster cooperative relationships across countries? EQ3.1 ⇒ How do your cooperative relationships developed through USAID facilitation compare to other types of cooperative relationships you may have formed through other North-South or regional cooperation activities? EQ3.2 ⇒ To what extent are the cooperative relationships developed as a result of the USAID trilateral cooperation activities likely to be sustained and continue delivering benefits over the long term? Why? ⇒ To what extent will your organization continue to form cooperative relationships, such as those formed through the USAID trilateral cooperation activities? ⇒ What are the primary hinderances or barriers you face in sustaining such cooperative relationships? In answering this question, please consider: o How well they function o The benefits they provide you and other relationship partners o Challenges forming and maintaining them Gender ⇒ What, if any, do you see as important gender issues related to: 1) the topics covered during the such cooperation activities; 2) participants in the cooperation activities; and 3) application of knowledge and expertise acquired from cooperation activities? 71 ⇒ To what extent did the USAID trilateral cooperation activities address these or other gender issues? 72 DISCUSSION GUIDE FOR PROVIDERS Name: Organization: Position: Telephone: Email: trilateral exchanges: INTRODUCTION • We are evaluating the USAID Regional Trilateral Cooperation Activity (or RTC) implemented by IBTCI. We would like to discuss your experience with and perceptions of the USAID-facilitated trilateral cooperation activities in which you participated as part of the RCT activity. • For the purpose of this interview, a trilateral cooperation activity is taken to be an activity aimed at building relationships and facilitating cooperation between 1) one or more organizations (e.g., donor organizations, government ministries, private enterprises, civil society organizations) with 2) other organizations in other countries to facilitate a sharing of knowledge and expertise or otherwise promote development across two or more countries. • Your participation is entirely voluntary and whatever information that you provide will be kept strictly confidential. • Do you have any questions? • May we begin? EQ1 ⇒ Please describe the trilateral cooperation activities in which you participated? o What were the objectives of these activities? o What was your personal role in these in these activities? o Who else participated and how? ⇒ Why did you participate in these activities? What did you hope to achieve? ⇒ To what extent did you achieve what you had hoped to achieve? ⇒ What benefits did you receive from participating in these trilateral cooperation activities? ⇒ Overall, how effective do you think the trilateral cooperation activities have been or will be in transferring important knowledge or expertise to participants? 73 EQ1.1 ⇒ What do you think are the primary hinderances or barriers participants face in applying the new knowledge or expertise acquired from the USAID trilateral cooperation activities? EQ2 ⇒ How well were the USAID trilateral cooperation activities managed? What was done well? What was done less well? How could it be improved? In answering this question, please consider the following: o Time commitment o Timeliness or relevance of topics covered o Usefulness of knowledge or expertise provided o Overall management and organization ⇒ Have you participated in cooperation activities similar to this in the past? These may include either traditional North-South Cooperation activities or regional cooperation activities that are sponsored by USAID, GIZ, JICA, or other donors or development organizations. If Yes, please describe them? (A North-South Cooperation activity is one in which countries from the Latin America and Caribbean Region partner or otherwise cooperate with the United States to receive knowledge or expertise about important topics. A regional relationship/cooperation activity is one in which countries from the Latin America and Caribbean Region partner or otherwise cooperate with each other countries in the region to share knowledge or expertise about important topics.) ⇒ How did these cooperation activities compare to the USAID trilateral cooperation activities in terms of the following? o Time commitment o Timeliness or relevance of topics covered o Usefulness of knowledge or expertise provided o Overall management and organization EQ3 ⇒ To what extent have the USAID trilateral cooperation activities contributed to building cooperative relationships between you and other persons either in your country, another country in the Latin America and Caribbean region, or the United States? Please describe these cooperative relationships. (A cooperative relationship is a working relationship between two or more people or organizations in which they share information or expertise with each other on topics of mutual interest. A cooperative relationship need not be formalized with a MOU or any other formal agreement and may be, but need not be, entirely informal in nature.) ⇒ To what extent would these cooperative relationships would have occurred in the absence of the USAID trilateral cooperation activities? ⇒ How well do these cooperative relationships function? What benefits are they providing you? Your organization? Other relationship partners? 74 ⇒ What are the primary hinderances or barriers you face in forming such cooperative relationships? ⇒ Overall, how effective were the USAID trilateral cooperation activities in fostering cooperative relationships between activity participants? Why? How could the activities be improved to better foster cooperative relationships across countries? EQ3.1 ⇒ How do your cooperative relationships developed through USAID facilitation compare to other types of cooperative relationships you may have formed through other North-South or regional cooperation activities? EQ3.2 ⇒ To what extent are the cooperative relationships developed as a result of the USAID trilateral cooperation activities likely to be sustained and continue delivering benefits over the long term? Why? ⇒ To what extent will your organization continue to form cooperative relationships, such as those formed through the USAID trilateral cooperation activities? ⇒ What are the primary hinderances or barriers you face in sustaining such cooperative relationships? In answering this question, please consider: o How well they function o The benefits they provide you and other relationship partners o Challenges forming and maintaining them Gender ⇒ What, if any, do you see as important gender issues related to: 1) the topics covered during the such cooperation activities, 2) participants in the cooperation activities, and 3) application of knowledge and expertise acquired from cooperation activities? ⇒ To what extent did the USAID trilateral cooperation activities address these or other gender issues? 75 DISCUSSION GUIDE FOR SUPPORTERS Name: Organization: Position: Telephone: Email: trilateral exchanges: INTRODUCTION • We are evaluating the USAID Regional Trilateral Cooperation Activity (or RTC) implemented by IBTCI. We would like to discuss your experience with and perceptions of the USAID-facilitated trilateral cooperation activities in which you participated as part of the RCT activity. • For the purpose of this interview, a trilateral cooperation activity is taken to be an activity aimed at building relationships and facilitating cooperation between 1) one or more organizations (e.g., donor organizations, government ministries, private enterprises, civil society organizations) with 2) other organizations in other countries to facilitate a sharing of knowledge and expertise or otherwise promote development across two or more countries. • Your participation is entirely voluntary and whatever information that you provide will be kept strictly confidential. • Do you have any questions? • May we begin? EQ1 ⇒ Please describe the trilateral cooperation activities in which you participated? o What were the objectives of these activities? o What was your personal role in these in these activities? o Who else participated and how? ⇒ Why did you support these activities? What did you hope to achieve? ⇒ To what extent did the activities achieve what you had hoped to achieve? ⇒ Overall, how effective do you think the trilateral cooperation activities have been or will be in transferring important knowledge or expertise to participants? 76 EQ1.1 ⇒ What do you think are the primary hinderances or barriers activity participants face in applying the new knowledge or expertise acquired from the USAID trilateral cooperation activities? EQ2 ⇒ How well were the USAID trilateral cooperation activities managed? What was done well? What was done less well? How could it be improved? In answering this question, please consider the following: o Use of resources o Time commitment of participants o Timeliness or relevance of topics covered o Expertise of experts providing knowledge o Usefulness of knowledge or expertise provided o Overall management and organization ⇒ Have you sponsored or participated in cooperation activities similar to this in the past? These may include either traditional North-South Cooperation activities or regional cooperation activities that are sponsored by USAID, GIZ, JICA, or other donors or development organizations. If Yes, please describe them? (A North-South Cooperation activity is one in which countries from the Latin America and Caribbean Region partner or otherwise cooperate with the United States to receive knowledge or expertise about important topics. A regional relationship/cooperation activity is one in which countries from the Latin America and Caribbean Region partner or otherwise cooperate with each other countries in the region to share knowledge or expertise about important topics.) Probe to determine whether these other cooperation activities were a centralized approach like RTC or an approach in which cooperation activities were either a component or sub-component of a development project. ⇒ How did these cooperative partnerships activities compare to the USAID trilateral cooperation activities in terms of the following? o Use of resources o Time commitment of participants o Timeliness or relevance of topics covered o Expertise of experts providing knowledge o Usefulness of knowledge or expertise provided o Overall management and organization ⇒ Trilateral or other cooperation activities may be designed in a few ways. One way is a centralized approach, such as that used by USAID/RTC. Another way is to include the 77 cooperation activities as a component or sub-component of a larger development project. What do you see as the advantages and disadvantages of the different approaches? Why? ⇒ If you were going to design a follow-up activity to the RTC, which approach would you take? Why? EQ3 (Note: Supporters may or may not be able to answer questions related to EQ3, EQ3.1, and EQ3.2, but they are worth asking to determine whether they have knowledge or opinions on these questions.) ⇒ To what extent do you think the USAID trilateral cooperation activities have contributed to building cooperative relationships between participants. (A cooperative relationship is a working relationship between two or more people or organizations in which they share information or expertise with each other on topics of mutual interest. A cooperative relationship need not be formalized with a MOU or any other formal agreement and may be, but need not be, entirely informal in nature.) ⇒ To what extent do you think would these cooperative relationships would have occurred in the absence of the USAID trilateral cooperation activities? ⇒ What in your opinion are the primary hinderances or barriers persons or organizations face in forming such cooperative relationships? ⇒ Overall, how effective in your opinion were the USAID trilateral cooperation activities in fostering cooperative relationships between activity participants? Why? How could the activities be improved to better foster cooperative relationships across countries? EQ3.1 ⇒ How do you think the cooperative relationships developed through USAID facilitation compare to other types of cooperative relationships that may have been formed through other North-South or regional cooperation activities? EQ3.2 ⇒ To what extent do you think the cooperative relationships developed as a result of the USAID trilateral cooperation activities are likely to be sustained and continue delivering benefits over the long term once the RTC project ends? Why? ⇒ What do you think are the primary hinderances or barriers people or organizations face in sustaining such cooperative relationships? In answering this question, please consider: o How well they function o The benefits they provide you and other relationship partners o Challenges forming and maintaining them Gender ⇒ What, if any, do you see as important gender issues related to: 1) the topics covered during the such cooperation activities, 2) participants in the cooperation activities, and 3) application of knowledge and expertise acquired from cooperation activities? 78 ⇒ To what extent did the USAID trilateral cooperation activities address these or other gender issues? ⇒ What are opportunities or challenges in incorporating gender considerations into trilateral cooperation activities or similar cooperation activities? 79 DISCUSSION GUIDE FOR COUNTERPART BI-LATERAL COOPERATION AGENCIES Name: Organization: Position: Telephone: Email: trilateral exchanges: INTRODUCTION • We are evaluating the USAID Regional Trilateral Cooperation Activity (or RTC) implemented by IBTCI. We would like to discuss your experience with and perceptions of the USAID-facilitated trilateral cooperation activities in which you participated as part of the RCT activity. • For the purpose of this interview, a trilateral cooperation activity is taken to be an activity aimed at building relationships and facilitating cooperation between 1) one or more organizations (e.g., donor organizations, government ministries, private enterprises, civil society organizations) with 2) other organizations in other countries to facilitate a sharing of knowledge and expertise or otherwise promote development across two or more countries. • Your participation is entirely voluntary and whatever information that you provide will be kept strictly confidential. • Do you have any questions? • May we begin? ⇒ Confirm that they have participated in a USAID (RTC) trilateral exchange. ⇒ Determine in which USAID (RTC) trilateral activities they have participated and what their role was. Determine further what and much resources, if any, both cash and in-kind, they contributed to the RTC trilateral activity. How did they come to contribute these resources? If they did not contribute resources, why not? ⇒ What relationships did they establish as a result of their participation in the RTC trilateral exchanges? How do these relationships operate? Are they continuing and how? How sustainable are they? ⇒ Are they planning to participate in more RTC trilateral exchanges? If no, why not? If yes, how? 80 ⇒ Ascertain whether they support trilateral (or triangular) cooperation outside of RTC. We are particularly (although not solely) interested in South-South Cooperation, which is cooperation two or more other countries from Latin America & Caribbean (LAC) facilitated/supported by the relevant cooperation agency. ⇒ Ask for some examples of trilateral South-South Cooperation they have supported. ⇒ Ask about the institutional model they use to support triangular South-South Cooperation. Probe to find out how they (1) administer the cooperation (e.g., what administrative/management structure do they have in place to operate the triangular South￾South cooperation) and (2) finance the cooperation. Why did they choose the model they use and what are its pros and cons? Probe to get as in-depth information about their model as possible. ⇒ What are their plans for supporting trilateral cooperation (particularly South-South) in the future, including their plans to invest resources (including money) to support it? Why? ⇒ What recommendations do they have for improving the USAID trilateral exchanges in the future? 81 DISCUSSION GUIDE FOR OTHER DONOR AGENCIES Name: Organization: Position: Telephone: Email: INTRODUCTION • We are evaluating the USAID Regional Trilateral Cooperation Activity (or RTC) implemented by IBTCI. We would like to discuss your experience with and perceptions of the USAID-facilitated trilateral cooperation activities in which you participated as part of the RCT activity. • For the purpose of this interview, a trilateral cooperation activity is taken to be an activity aimed at building relationships and facilitating cooperation between 1) one or more organizations (e.g., donor organizations, government ministries, private enterprises, civil society organizations) with 2) other organizations in other countries to facilitate a sharing of knowledge and expertise or otherwise promote development across two or more countries. • Your participation is entirely voluntary and whatever information that you provide will be kept strictly confidential. • Do you have any questions? • May we begin? ⇒ Confirm that they support trilateral (or triangular) cooperation. We are particularly (although not solely) interested in South-South Cooperation, which is cooperation two or more other countries from Latin America & Caribbean (LAC) facilitated/supported by the relevant cooperation agency. ⇒ Have they ever cooperated with USAID in supporting trilateral cooperation? If no, why not? If yes, how? ⇒ Ask for some examples of trilateral (South-South) cooperation they have supported. We are interested generally in their trilateral cooperation activities but particularly those involving El Salvador. ⇒ Ask about the institutional model they use to support triangular South-South Cooperation. Probe to find out how they 1) administer the cooperation (e.g., what administrative/ management structure do they have in place to operate the triangular South-South Cooperation) and 2) finance the cooperation. Why did they choose the model they use and what are its pros and cons? Probe to get as in-depth information about their model as possible. 82 ⇒ What are their plans for supporting trilateral cooperation (particularly South-South) in the future, including their plans to invest resources (including money) to support it? Why? 83 ANNEX E: WBS QUESTIONNAIRE 84 85 86 87 88 89 90 91 92 93 94 95 ANNEX F: WBS RESPONSES 96 2. In which type of USAID trilateral cooperation activity did you participate? (N=72) Type of Activity Number Percentage Study tour 25 34.7% Expert delegation 11 15.3% Expert visit 12 16.7% Gave or attended presentation 9 12.5% Workshop 33 45.8% Training 22 30.6% Conference 23 31.9% Other 1 1.4% 3. To what extent do you agree with the following statements about the USAID trilateral activity? (N=72) Statement Strongly Disagree Disagree Neither Agree nor Disagree Agree Strongly Agree Do Not Know Time required to participate was appropriate 4 2 6 26 29 5 Activities addressed timely and important topics 4 1 3 8 49 7 Activities adequately addressed participants’ needs 4 2 4 20 37 5 Persons who provided experience and knowledge were experts in their fields 4 2 1 17 39 9 The experience and knowledge provided was useful 5 2 0 14 41 10 The logistics were well organized and managed 4 3 6 18 33 8 The activities strengthened existing relationships with foreign counterparts 4 5 5 16 35 7 The activities facilitated the formation of new relationships with foreign counterparts 3 5 5 21 33 5 The activities produced practical benefits for participants 4 4 4 20 32 8 Trilateral cooperation is an effective way to address problems for organizations or countries 5 2 2 13 44 6 4. What do you think are the most important benefits to participating in trilateral cooperation activities? (N=72) Benefits 1st Most Important 2nd Most Important 3rd Most Important Total Learning how other countries have solved similar problems 41 10 6 57 Learn about new methods to solve problems 9 24 12 45 Establish cooperative relationships with experts in other countries 6 13 21 40 Learn about contextually appropriate solutions from other countries 5 12 11 28 Establish cooperative relationships with organizations in other countries 6 4 15 25 97 Benefits 1st Most Important 2nd Most Important 3rd Most Important Total Identify low cost solutions to problems 5 9 7 21 5. In general, how satisfied were you with the USAID trilateral cooperation activities? (N=72) Level of Satisfaction Number Percentage Significantly below expectations 0 0.0% Below expectations 3 4.2% Satisfied expectations 37 52.1% Exceeded expectations 20 28.2% Significantly exceeded expectations 9 12.7% Don’t know 2 2.8% Significantly below expectations 0 0.0% Below expectations 3 4.2% 6. What do you think are the most important challenges to participating in USAID trilateral cooperation activities? (N=72) Challenges 1st Most Important 2nd Most Important 3rd Most Important Total Lack of information about relevant opportunities 22 13 11 46 High costs/lack of financing 20 16 3 39 Difficulty coordinating with counterparts in other countries 8 15 12 35 Difficulty identifying counterparts with relevant experience or knowledge 12 11 9 32 Difficulty identifying counterparts interested in cooperating 5 5 10 20 Lack of capacity 3 5 5 13 Lack of organizational interest 0 4 9 13 Don't know 1 0 9 10 Other 1 1 0 2 Lack of personal motivation 0 0 0 0 7. Would you have participated in these trilateral cooperation activities without USAID? (N=72) Level of Satisfaction Number Percentage No 13 18.1% Probably no 34 47.2% Probably yes 15 20.8% Yes 7 9.7% Don't know 3 4.2% 8. Would you have participated in other types of trilateral cooperation activities without USAID? (N=72) Level of Satisfaction Number Percentage No 7 9.7% Probably no 28 38.9% Probably yes 20 27.8% Yes 12 16.7% Don’t know 5 6.9% 98 9. Have you participated in other trilateral cooperation activities in the past? (N=72) Level of Satisfaction Number Percentage Yes 41 56.9% No 25 34.7% Don’t know 6 8.3% 10. How do you compare these other trilateral cooperation activities to those facilitated by USAID? (N=40) Quality Criteria Much Worse Worse Equal Better Much Better Do Not Know Cost to participate 1 5 8 11 6 3 Time to participate 0 1 17 12 7 1 Relevance/importance of themes 0 1 17 9 11 2 Addressed the needs of participants 0 0 16 9 12 2 Expertise/experience of providers 0 2 15 8 13 1 Usefulness of experience and knowledge provided 0 5 11 12 9 3 Organization and logistics 0 5 12 11 9 3 Facilitation of new relationships 0 4 10 11 11 4 Creation of tangible benefits 2 4 11 9 11 3 11. Do you plan to participate in other USAID trilateral cooperation activities in the future? (N=72) Likelihood Number Percentage No 0 0.0% Not likely 1 1.4% Likely 16 22.2% Probably 11 15.3% Yes 36 50.0% Don’t know 8 11.1% 12. Do you plan to participate in other types of trilateral cooperation activities in the future? (N=72) Likelihood Number Percentage No 0 0.0% Not likely 2 2.8% Likely 23 31.9% Probably 19 26.4% Yes 13 18.1% Don’t know 15 20.8% 13. What role did you play in the USAID trilateral cooperation activity? (N=72) 99 Role Number Percentage Receiver 51 70.8% Provider 21 29.2% Organizer 11 15.3% Moderator 5 6.9% Observer 14 19.4% Authorizer 2 2.8% Other 1 1.4% Don’t know 0 0.0% 14. As a result of your participation in the USAID trilateral cooperation activities, have you or your organization taken any of the following actions? Action No & Will Not in Future No But May in Future No But Plan To Yes Do Not Know Not Applicable Shared experience or knowledge with people in my organization 0 0 2 43 5 0 Shared experience or knowledge with people outside my organization 0 10 5 30 4 1 Implemented the experience or knowledge in my job 0 1 5 39 4 1 Promoted innovations or changes within my organization 1 4 8 28 5 4 Promoted political changes in my country 1 12 8 17 2 10 Maintained contact with people I met 0 8 6 33 3 0 Solicited help from people I met 1 9 5 28 5 2 Provided help to people I met 0 14 7 19 4 6 15. What are the principle challenges you or your organization face in applying the experience or knowledge you acquired during the USAID trilateral cooperation activity? (N=50) Challenges 1st Most Important 2nd Most Important 3rd Most Important Total Lack of financing 23 10 5 38 Requires technologies or capacities too advanced 10 8 5 23 Lack of political support 6 4 12 22 Experience or knowledge acquired not relevant to country 3 8 6 17 Lack of time 4 6 5 15 Not sure how/need more capacity development 2 3 5 10 Lack of institutional support 1 2 2 5 Lack of support from immediate supervisor 0 1 1 2 Experience or knowledge acquired not relevant to job 0 1 0 1 Lack of personal motivation 0 0 0 0 Experience or knowledge acquired not relevant to organization 0 0 0 0 100 16. As a result of your participation in the USAID trilateral cooperation activity, have you strengthened existing cooperative relationships or formed new cooperative relationships with counterparts from your country? (N=62) Strengthened or Formed Relationship Number Percentage Yes 47 75.8% No 12 19.4% Don’t know 3 4.8% 17. As a result of your participation in the USAID trilateral cooperation activity, have you strengthened existing cooperative relationships or formed new cooperative relationships with counterparts from other countries? (N=62) Strengthened or Formed Relationship Number Percentage Yes 43 69.4% No 16 25.8% Don’t know 3 4.8% 18. Would you have strengthened or formed these same relationships without participating in the USAID trilateral cooperation activities? (N=49) Strengthened or Formed Relationship Number Percentage No 7 14.3% Probably no 31 63.3% Probably yes 10 20.4% Yes 1 2.0% 19. Would you have strengthened or formed similar relationships without participating in the USAID trilateral cooperation activities? (N=49) Strengthened or Formed Relationship Number Percentage No 6 12.2% Probably no 30 61.2% Probably yes 12 24.5% Yes 1 2.0% 20. Have you or your organization taken advantage of these strengthened or new cooperative relationships formed during USAID trilateral cooperation activities? (N=49) Strengthened or Formed Relationship Number Percentage Yes 38 77.6% No 4 8.2% Don’t know 7 14.3% 21. How have you or your organization taken advantage of these strengthened or new cooperative relationships formed during USAID trilateral cooperation activities? (N=49) Establish Relationships and Commitments (9) • Keeping contact data for when authorities finally bet on public policy reforms 101 • The greatest benefit has been establishing a better communication with universities allowing them to become aware of the knowledge that future professionals should be exposed to in order to assure better contributions to the productive sector. • Establish commitments between countries for the development of projects. • These activities have encouraged important conversations regarding changes and necessary improvements in Salvadoran public institutions, generating commitment of the parties for advancing in this sense. • We have strengthened relationships for working together; in 2019 we will start a project with them in our country. • Keeping in touch with visited entities in the other countries to continue looking for mutual collaboration opportunities. • Increasing the number of connections/contacts with the government and other institutions allowing us to work together for the same purpose. • Working relations with CATIE's technical team on sustainable low-carbon livestock issues have been strengthened. • A relationship of cooperation and exchange of good practices was established. Update Laws and Regulations (8) • Knowledge acquisition through experiences that have helped me improve and update laws in my country and establish valuable contacts and cooperation. • Comparison of regulatory frameworks with SNE Panama (Panama’s Secretariat of Energy) and CNE El Salvador (El Salvador’s National Center for Energy) on issues relevant to natural gas and prices and regulations in the hydrocarbons sector. Moreover, in the case of Guatemala, information on renewable energy power generation. • Structured TA for Strengthening the Civil Service Professionalization System in El Salvador’s Executive Branch. • Acquiring knowledge for updating national regulations including updated health/sanitary requirements. Strengthening the technical justification within our organization for the acquisition of resources that will help improve food inspection and control to ensure its safety. • Institutional actions for legislation reforms and training of staff • Legitimize recommendations for public policy reform by learning from the activities of other institutions. • Has made me become aware about procedures required for standardization and legislation. I have gained better understanding of the purpose and ways of working of the organization in which I work. • Different meetings were held with the institution’s liaison, the executive director and myself to learn more about the implementation and development of an open data institutional policy and about data publication. These meetings were held with an expert from Mexico, Sergio Araiza. Strengthen Institutional Capacity (5) • Acquiring new knowledge and challenges to improve the performance of my work. Obtaining better results in the social action of the organization. • Consultancies, promoting initiatives within the institution to which I belong to and continuing a cooperative relationship with technical staff of USAID projects. 102 • I have acquired new technical cooperation support sources. We have identified important synergies between organizations, sharing experiences, best practices, etc. • For research. • Improving organizational climate in addition to strengthening public, private and academic links/connections to promote innovation and plans and programs of applied research. Strengthen Partner Capacity (5) • Technical assistance and training for associated producers. • In the design of new programs for the benefit of counterparts. • Engagement of “players” in the market in Belize. • Producers have been trained in nutrition and management issues. • Knowledge transfer by experts from other countries to partners to achieve project goals and ensure their sustainability. Share Information (5) • Through sharing information and experiences. • Through the internationalization of students and teachers who access higher education studies in institutions abroad. • Exchanging knowledge and experiences. Exchange of experiences and approaches to real cases in the region. • Better knowledge of technology and implementation experience of innovative projects in similar neighboring countries. • Bringing expert consultants giving continuity to what has been learned in Medellin. Implement Innovations (3) • Established links with organizations that support the development and establishment of methodologies for innovation routes; this has already facilitated the conduction of diagnoses in some universities. Additionally, knowing about the TECNOVA model has allowed us to design s similar scheme of work with universities working with the Project. • Implemented innovative projects. • Took good advantage from getting to know about the experience of local and international partners for building a higher education national policy proposal for El Salvador, between 2017 and 2018, taking into consideration the diverse positions of the stakeholders involved in Medellin’s particular experience were city development was possible through a partnership for common good…the know how about state and private enterprises, higher education institutions and civil society agree to working together enabling local development. 22. How likely is it that these strengthened or new cooperative relationships will be maintained over the long-term? (N=48) Likelihood Number Percentage Not at all likely 0 0.0% Not likely 2 4.2% Likely 13 27.1% Very likely 18 37.5% Yes, certainly 11 22.9% 103 Likelihood Number Percentage Don’t know 4 8.3% 23. Why do you think that these relationships will not be sustained over the long-term? (N=2) Reason Number Lack of financing 2 Lack of institutional support 1 Relatively low priority 1 Counterparts rotate to different assignments 1 Doesn’t generate sufficient benefits to maintain 1 24. Why do you think that these relationships will be sustained over the long-term? (N=46) Reason Number Percentage High level of personal motivation 24 52.2% Support from my organization 21 45.7% Counterparts possess valuable knowledge 18 39.1% Shared objectives with counterparts 16 34.8% Relationships provide tangible benefits 16 34.8% Productive working relationship with counterparts 16 34.8% Support from my immediate supervisor 9 19.6% Availability of financing 8 17.4% Counterparts all contribute equally 6 13.0% Get along well with counterparts 5 10.9% Relatively high priority 2 4.3% 25. How would you compare the USAID trilateral cooperation activities to other trilateral or bilateral South-South Cooperation activities in which you have participated? (N=41) Comparison Number Percentage Much worse 0 0.0% Worse 3 7.3% Same 10 24.4% Better 16 39.0% Much better 9 22.0% Don’t know 3 7.3% 26. From which country are you? (N=71) Country Number Percentage Belize 1 1.4% Colombia 2 2.8% Costa Rica 2 2.8% El Salvador 36 50.7% Guatemala 9 12.7% Honduras 13 18.3% Mexico 1 1.4% Panama 2 2.8% Peru 3 4.2% United States 1 1.4% Uruguay 1 1.4% 27. What is your sex? 104 Sex Number Percentage Female 44 61.1% Male 28 38.9% 28. How would you describe your position within your organizations? (N=72) Position Number Percentage Executive 22 30.6% Supervisor 15 20.8% Middle Management 16 22.2% Technical or administrative 16 22.2% Other 3 4.2% 29. How would you describe your organization? (N=72) Type of Organization Number Percentage USAID 5 6.9% USAID Project 7 9.7% Donor Agency 1 1.3% Government Ministry 34 47.2% Private Sector Organization 13 18.0% Civil Society Organization 4 5.5% Academic Institution 7 9.7% Other 1 1.3% 30. Which of the following statements best represent your opinion about RTC? (N=15) Statement Number USAID Number IP Total RTC’s cost is too high, regardless of its benefits 3 0 3 RTC’s costs are too high in relation to its benefits 1 1 2 RTC’s costs are appropriate regardless of its benefits 0 0 0 RTC’s costs are appropriate in relation to its benefits 1 3 4 Don’t have knowledge about RTC’s costs 0 3 3 No opinion 3 0 3 31. To what extent do you agree with the following statement: RTC should offer technical services in addition to logistical support services? (N=15) Statement Number USAID Number IP Total Strongly disagree 0 0 0 Disagree 1 0 1 Neither agree nor disagree 0 1 1 Agree 4 1 5 Strongly agree 2 5 7 Don’t know 1 0 1 32. If RTC were to end tomorrow, would you continue investing in trilateral cooperation activities out of your own resources? (N=12) Answer Number USAID Number IP Total Yes 4 7 11 No 0 0 0 Don’t know 1 0 1 105 33. Do you think that the Mission should continue to support trilateral cooperation activities? (N=12) Answer Number USAID Number IP Total Yes 1 0 1 No 4 2 6 Don’t know 0 5 5 34. Would you recommend continuing RTC after its end? (N=12) Answer Number USAID Number IP Total Yes 1 0 1 No 4 2 6 Don’t know 0 5 5 106 ANNEX G: NEW KNOWLEDGE APPLIED AND RELATIONSHIPS FORMED BY TLA PARTICIPANTS AND RTC’S ROLE IN TRILATERAL ACTIVITIES 107 Sponsoring Entity Trilateral Effort Trilateral Exchange Trilateral Exchange Results Relationships Strengthened/Formed RTC Role in Trilateral Exchange USAID/EG Office Regional Trade Facilitation One Stop Shop Customs Border Crossings Part II Relevant issues are now included in the agenda of the customs directors at the participant Central American customs agencies. Central American customs directors acknowledged the importance of implementing the integral risk management processes, Single Window system, Authorized Economic Operator (AEO) program, the importance of private sector participation, incorporating IT and taking steps to develop a mutual recognition program. Central American Risk Management Group held its first meeting. Met with SIECA and AMEXCID representatives to follow up in the preparation of agenda for capacity exchange activities. Contacted all experts to explain the capacity building activities to be carried out. Established and maintained communication SIECE and AMEXCID to ensure their collaboration. Created the opportunity for customs directors to communicate and get to know each other personally and talk about the problems they faced and how others have dealt with them. USAID/EG Office Regional Trade Facilitation Authorized Economic Operators Part III USAID/EG Office Regional Trade Facilitation Customs Union Integration Part IV USAID/EG Office Regional Trade Facilitation Customs Risk Management Part V USAID/EG Office Regional Trade Facilitation Risk Management Strategy Finalization Part VIII USAID/EG Office Regional Trade Facilitation U.S. Law Equivalence Part VII USAID/EG Office Regional Trade Facilitation Applying Information Technology Part X USAID/EG Office Regional Trade Facilitation Public-Private Inspection Certification Part I USAID/EG Office Regional Trade Facilitation US-Mexico Customs and Border Facility Part IX (KII)26 Relevant issues are now included in the agenda of the customs directors at the participant Central American customs agencies. Central American customs directors acknowledged the importance of implementing the Central American Risk Management Group held its first meeting. Customs Directorates in El Salvador and Guatemalan formed close cooperative relationship that includes agreement on detailed plan Created communication spaces according to each level (e.g., executives or technicians) so as to facilitate the development of relationships between counterparts. USAID/EG Office Regional Trade Facilitation Risk Management SIECA Part VI (KII) 26 The KII in parentheses (KII) indicates that the ET interviewed participant(s) in the relevant trilateral exchange. 108 Sponsoring Entity Trilateral Effort Trilateral Exchange Trilateral Exchange Results Relationships Strengthened/Formed RTC Role in Trilateral Exchange integral risk management processes. of cooperation between the two agencies. GOES reports that Guatemalan customs counterparts have established presence within El Salvador territory as part of customs cooperation. Coordinated activities to carry out in the event, making sure the agenda was ready for the distribution to the participants prior to the event and that all the topics were included. Made sure that all experts were willing to travel and know what their role would be in each activity. USAID/EG Office Regional Trade Facilitation Sanitary Product Registration/Food Safety (KII) Contributed to the development of regional information technology system based in the Central American Integration System (SIECA) model that harmonizes the food and beverage commercialization processes so that products having health registration in one country may enter other regional countries. System automatically provides online documentation for registration to be valid in other countries. Established and maintaining (e.g., via ongoing or ad hoc communication) with 4-5 counterparts in other countries, including private sector. El Salvador, in coordination with the Pan American Health Organization and the World Health Organization, hosted a two-day encounter of the Pan American Network for the Harmonization of Pharmaceutical Regulation. Representatives of 35 countries, including the Central American authorities who attended the RTC trilateral exchange in Tegucigalpa, came together to discuss homogenization of the application of regulations Held or coordinated several meetings with IPs and different counterparts (e.g., USAID, World Bank, and AMEXCID) to obtain participants’ information, develop the agenda, and plan the trilateral exchange. Contacted the experts and explain to them the activities to carried out. Supported the experts in the distribution of the participants during the discussion tables so as to facilitate more productive discussions. Ensured that the distribution of the working tables was according to the needs requested by the implementers. 109 Sponsoring Entity Trilateral Effort Trilateral Exchange Trilateral Exchange Results Relationships Strengthened/Formed RTC Role in Trilateral Exchange and requirements for medicines, cosmetics, and hygiene products; interpretation of regulations; risk management and registration of cosmetic products; synergies and agreements with the private sector (manufacturers and suppliers), among other topics. ARSA (Honduras) agreed to implement COFEPRIS recommendations related to Sanitary Registration, and the two agreed to establish direct communication channels and to work on a collaborative instrument to formalize and promote cooperation ties. ARSA and ANVISA (Brazil) agree on a joint collaboration project for 2019. Fiscal Policy and Expenditure Management Program (DAI) Fiscal Policy Diagnosis of the Salvador Customs/Port System Part I (KII) MOF and USAID Domestic Resource Mobilization Project developed a custom code proposal. The proposal remains at the MOF and is waiting to be sent to the Legislative Assembly. Information not available (N/A) Established communication with the IP to plan capacity building activities for the exchange. Liaised with the Colombian expert to explain expectations for capacity building activities and 110 Sponsoring Entity Trilateral Effort Trilateral Exchange Trilateral Exchange Results Relationships Strengthened/Formed RTC Role in Trilateral Exchange shared the agenda for all planned activities. Fiscal Policy and Expenditure Management Program (DAI) Fiscal Policy Transfer Pricing Control System Part III (KII) MOF made adjustments and improved its auditing processes for transfer pricing. MOF now reviews pricing studies in a more in-depth manner, with another analysis perspective. MOF adopted OECD standards for transfer pricing; however, GOES decreed the standard to be unconstitutional. N/A Held meetings with IPs to obtain participants’ information, develop the agenda, and plan the trilateral exchange. Fiscal Policy and Expenditure Management Program (DAI) Fiscal Policy Tax Collection and Anti-Corruption Part IV (KII) MOF revised its proposal for a new National Tax Strategy, which is now awaiting Congressional approval. MOF (Investigative Intelligence Unit & Transparency and Anti￾Corruption Unit) refined the investigations/research and verification processes, with better procedural guarantees for investigating growth of capital/assets by public servants. MOF improved risk management focused on what public officials do and established more assertive penalties for their criminal/corruptive behavior. MOF redefined and reclassified actions that may be subjected to penalization. MOF is working with its Communications Unit to establish an electronic MOF shared documents and exchanged emails post-trilateral exchange with counterparts in Colombia on issues related to tax collection call centers. Held meetings with IPs to obtain participants’ information, develop the agenda, and plan the trilateral exchange. Made sure all participants were ready to travel and knew the purpose and objectives of trilateral exchange. 111 Sponsoring Entity Trilateral Effort Trilateral Exchange Trilateral Exchange Results Relationships Strengthened/Formed RTC Role in Trilateral Exchange mechanism for submitting complaints through a system that also protects complainants’ integrity. MOF developed a more efficient tax collection processes based on a platform shared by Colombia. Domestic Resource Mobilization Project (DAI) Fiscal Policy Single Taxpayer System Part VII (KII) MOF developed a proposal for a simplified tax system for micro and small taxpayers, which is under discussion at the MOF and the Comisión Nacional de la Micro y Pequeña Empresa (CONAMYPE). N/A Held meetings with IPs to obtain participants’ information, develop the agenda, and plan the trilateral exchange. Acted as liaison between the IP and non-MOF participants. Held meeting to give participants the opportunity to meet each other before the trip Domestic Resource Mobilization Project (DAI) Fiscal Policy Implementation of a Modern Public Procurement System Part VIII (KII) MOF developed the Salvadoran electronic procurement system COMPRASAL II. MOF formulated the Public Procurement System Reform (draft bill) by its Public Procurement and Contracting Unit. N/A Coordinated with counterparts to develop the agenda, plan the trilateral exchange, and obtain participants’ information. Contacted experts to explain the activity in El Salvador. Domestic Resource Mobilization Project (DAI) Fiscal Policy Results Oriented Budget Part IX (KII) MOF developed a M&E Budget conceptual framework to be implemented beginning in January 2020 as part of its results￾oriented budgeting initiative. N/A Coordinated with counterparts to develop the agenda, plan the trilateral exchange, and obtain participants’ information. 112 Sponsoring Entity Trilateral Effort Trilateral Exchange Trilateral Exchange Results Relationships Strengthened/Formed RTC Role in Trilateral Exchange Contacted experts to explain the activity in El Salvador. USAID/EG Office Fiscal Policy Cash Flow Models Part X (KII) MOF developed better cash projection models within its Integrated Financial Management System (SAFI II) N/A Coordinated with counterparts to develop the agenda, plan the trilateral exchange, and obtain participants’ information. Contacted experts to explain the activity in El Salvador. Fiscal Policy and Expenditure Management Program (DAI) Fiscal Policy Simplified Tax System Part V (KII) MOF and the USAID Domestic Resource Mobilization Project developed a Simplified Tax System Law Proposal. The proposal is currently under discussion at the MOF and CONAMYPE. N/A Coordinated with counterparts to develop the agenda, plan the trilateral exchange, and obtain participants’ information. Contacted experts to explain the activity in El Salvador. Domestic Resource Mobilization Project (DAI) Fiscal Policy International Public Sector Accounting Standards Part VI (KII) The experience was used to develop the General Directorate Government Accounting organizational reform proposal and implementation plan. The organizational reform is required in order to guarantee a proper implementation of the International Public Sector Accounting Standards. N/A Coordinated with counterparts to develop the agenda, plan the trilateral exchange, and obtain participants’ information. Contacted experts to explain the activity in El Salvador. Fiscal Policy and Expenditure Management Program (DAI) Fiscal Policy Reverse Auctions and Public Procurement Part II (KII) MOF and USAID Domestic Resource Mobilization Project developed a Public Procurement Law reform proposal. The proposal remains at the MOF N/A Coordinated with counterparts to develop the agenda, plan the trilateral exchange, and obtain participants’ information. 113 Sponsoring Entity Trilateral Effort Trilateral Exchange Trilateral Exchange Results Relationships Strengthened/Formed RTC Role in Trilateral Exchange and is waiting to be sent to the Legislative Assembly. Contacted experts to explain the activity in El Salvador. Higher Education for Economic Growth Activity (HEA) and Economic Competitiveness Activity (ECA) Higher Education and Economic Competitiveness (Ruta N) Innovation: RUTA N Part I (KII) ECA Annual Work Plan (Year 2) includes activities addressing the following initiatives identified in Ruta N: 1) innovation labs, 2) Innovation for All Program, 3) digital business centers, and 4) entrepreneurship development centers. ASI and GOES (MINED) launch joint innovation initiative involving academia, government, and the private sector. Catholic University of El Salvador will implement an innovation lab and has included it in its work plan. ASI redesigned its USAID Industria Joven project based on principle learned during trilateral exchange HEA set up the innovation cluster. Inbox (CSO participant in trilateral exchange) is working with Industria Joven to consolidate a digital manufacturing model, to develop a production engineering/innovation management course, and set up a display at ASI’s Industrial Fair demonstrating the digital lab manufacturing initiative. HEA and ECA increased collaboration on issues of shared interest. HEA and ECA are in discussions for HEA to take over implementation of 5 of 21 applied research cases and bring them to market with a pilot. HEA and ECA formed ongoing relationships with Colombia counterparts in the public, private, and CSO sectors. ECA strengthened relationships with technology cluster members: Chamber of Commerce GOES ministries (MINED, Ministry of Economy, Ministry of Science and Technology), entrepreneurs, and ASI. HEA establish connections with the innovation organization Tecnova in Colombia and is working to establish relationships with four universities linked to specific clusters. ECA expects to collaborate with the Met with IP, Palladium and RTI, COPs to explain SSC and the goals of building relationships between regional counterparts to ensure the continued exchange of capacity after the project ends. Assisted them in thinking through the participant list to ensure appropriate relationships are created and encouraged them to design capacity building activities to incorporate the development of those relationships. Helped the COPs think through the strategy of what they are trying to get from the trilateral exchange and how to achieve it by putting in place relationships between the Colombians, Chileans, and Salvadorians towards creating an innovation eco￾system in El Salvador like those existing in the providing countries. Led meetings between the USAID and APC to obtain Higher Education for Economic Growth (RTI) and Economic Competitiveness Activities (Palladium) Higher Education and Economic Competitiveness (Ruta N) Innovation: RUTA N Part II (KII) Higher Education for Economic Growth (RTI) and Economic Competitiveness Activities (Palladium) Higher Education and Economic Competitiveness (Ruta N) Innovation: Ruta N Roadmap Part III (KII) 114 Sponsoring Entity Trilateral Effort Trilateral Exchange Trilateral Exchange Results Relationships Strengthened/Formed RTC Role in Trilat eral Exchange Chamber of Commerce, FUSADES (via the innovation and technology program Proinnova), and other stakeholders to implement digital business centers and entrepreneurship development centers. ASI and GOES (MINED) beginning to collaborate despite historical antagonism. For example, ASI invited MINED to speak at the “Road to 4.0: Toward Industrial Digitalization” inauguration event announcing GOES’ investment in technology and innovation to enhance the market. Innovation stakeholders (e.g., ECA, HEA, ASI, Banco Agricola, Chamber of Commerce, FUSADES, Inter -American Development Bank, etc.) have held follow -up meetings in attempt to organize and collaborate development of an innovation ecosystem. A group of stakeholders have agreed on development of a Route Map, which will be sponsored by the Inter - their cooperation in the trilateral exchange. Coordinated meetings between USAID and IPs involved in planning the trilateral exchange. Ensured that all relevant topics were included in the agenda. Hired external consultant to research the experience of RUTA N. Identified Colombian organizations and explained to them the trilateral exchange’s purpose to obtain their cooperation to receive the trilateral exchange participants and help the participants make connections with relevant Colombian entities for future collaboration. Facilitated the identification of experts than could come to El Salvador to follow -up on the trip to Colombia. Coordinated a wrap -up activity at the end of the trilateral exchange to make sure the exchange’s objectives were accomplished. 115 Sponsoring Entity Trilateral Effort Trilateral Exchange Trilateral Exchange Results Relationships Strengthened/Formed RTC Role in Trilateral Exchange American Development Bank. ASI and La Asociación Nacional de Empresarios de Colombia established an open line of communication laying the groundwork for future collaboration. USAID Government Integrity Project (TetraTech) Justice Sector Strengthening Open Government Partnership (KII) Ministry of Public Works (MOP) Access to Public Information Office created and launched a well-structured and functional open government portal with endorsement of Minister. The MOP is now managing and delivering information that meets the Declaration’s open data requirements. The MOP would have done portal in any case, but as result of the trilateral exchange, it was able to improve the design features. IAIP is formulating regulatory framework for publishing open data that will allow the use of “DemocracyOS.” IAIP established relationships with Mexican expert and pioneers of DemocracyOS with whom working to develop regulatory framework for publishing open data. IAIP and MOP established and maintained relationships with open government experts in other countries and with local counterparts in El Salvador. IAIP provides TA to Peruvian counterpart agency Tribunal of Transparency on agency implementation. Held meetings with trilateral exchange counterparts to create the space for them to create relationships. Established relationships with Argentinian experts and Pioneers of Democracy, with whom it is working, to develop a regulatory framework for publishing open data. Supported the establishment and maintenance of relationships with open government experts in other countries and their counterparts in El Salvador. Justice Sector Strengthening Project (Checchi) Justice Sector Strengthening Integrity Evaluation and Control Center (PNC) (KII) GOES revised draft PNC reform bill and submitted it to President’s Office. The bill is currently waiting to be submitted to National Assembly. N/A Coordinated meeting between IP and other counterparts to provide opportunity for them to create relationships prior to the travel portion of the trilateral exchange. 116 Sponsoring Entity Trilateral Effort Trilateral Exchange Trilateral Exchange Results Relationships Strengthened/Formed RTC Role in Trilateral Exchange Held a meeting to give IPs and counterparts the opportunity to develop a better understanding of what was required to create the Evaluation Center, the cost and human resources involved, and the requirements to sustain this type of project. USAID Government Integrity Project (TetraTech) Justice Sector Strengthening Civil Service Reform/Merit￾Based Models Part I (KII) GOES formulated new Civil Service Law (bill) submitted to the Legislative Assembly for approval. GOES incorporated psychologists from institutions of the executive branch to participate in the validation of the work evaluation tools prior to finalizing. GOES reduced estimated cost of relevant civil service reforms originally estimated at $278,020 to $ 44,901. GOES updated work organization system and salary policy for the executive branch. GOES revised national government management policies as follows: 1) updated work organization system, 2) reformed teacher certification process, 3) reformed career development management systems, 4) reformed executive development program, 5) GOES reached agreement with Chile to reformulate the TA project, to reduce costs and incorporate the areas in which Chile can provide its support. GOES reached agreement with Chile to assist with relevant civil service reforms and to establish a communication mechanism between the two countries through the realization of video conferences. Chilean counterparts to conduct technical visit to El Salvador to work with El Salvador Direction of Professionalization of the Public Function and to speak at the VI forum of the public function in October 2018, organized by RTC. South-South Cooperation relationships were created and strengthened between Contacted AGCID to obtain its cooperation with the GOES Technical Secretariat. Held several meetings with the IP to obtain the information related to the trilateral exchange and get to know the objectives and the purpose of its participation in order to guide the IP to identify the right participants to enable capacity building of the Salvadorian counterpart. Encouraged the IP to design an agenda that created spaces for these relationships to develop by staging specific set-aside conversations and other informal discussions that facilitate the development of relationships. Facilitated the relationships between counterparts by having the local government USAID/DG Office Justice Sector Strengthening Civil Service Reform Part II (KII) 117 Sponsoring Entity Trilateral Effort Trilateral Exchange Trilateral Exchange Results Relationships Strengthened/Formed RTC Role in Trilateral Exchange reformed performance management system. the Chilean government’s human resource subject matter expert and senior leadership in SETEPLAN in order to build the capacity of the Salvadorians to further develop their National Strategy for Civil Service Performance Management. authorities travel to Chile to meet with their counterparts to learn from their experiences. USAID Office of the Mission Director and Office of the US Ambassador Justice Sector Strengthening IOM Regional Conference for Reintegration of Returning Migrants (KII) Identified migratory trends and common and specific aspects between countries that would make possible the strengthening of legal frameworks, institutions/services, and partnerships with the private sector, among other actors. N/A Met IOM leadership on several occasions to assist it in designing the trilateral exchange strategy, including their main goals for the event and, thus, who participants should be and who should be paired with whom in relationship creation. Created spaces for those identified groups (e.g., those focused on the labor aspect of immigration or others focused on the finance or education aspect) to meet. Created a space for high level diplomatic talks to take place between USAID Deputy Assistant Administrator for LAC, GOES Minister of Justice, USAID Mission Director, and other high-level officials to strengthen conversations on the topic of returnees. During the trilateral exchange, pulled aside 118 Sponsoring Entity Trilateral Effort Trilateral Exchange Trilateral Exchange Results Relationships Strengthened/Formed RTC Role in Trilateral Exchange participants to make introductions between them and their regional counterparts as identified by IOM as priority relationships. Created informal dialogue spaces for those conversations and encouraged folks to exchange contact information. Held several meetings with the IOM, USAID, and facilitators to ensure the success of the trilateral exchange. Coordinated activities related to this event, such as agendas, and identified a consultant to facilitate exchanges during the conference. Coordinated and facilitated a series of South-South dialogue sessions on the experiences of programs and projects related to returned emigrants and reintegration in countries of origin. Worked on the final deliverable report to USAID. Bridges to Employment (DAI) Higher Education and Economic Competitiveness Youth Workforce (KII) IP and its Salvadoran counterparts explored the workforce development Head of Col Café, largest coffee producer in Colombia, will, with Held meetings with counterparts to design the 119 Sponsoring Entity Trilateral Effort Trilateral Exchange Trilateral Exchange Results Relationships Strengthened/Formed RTC Role in Trilateral Exchange activities taking place in Colombia through institutions like SENA and Fundación Corona and identified opportunities to replicate successful strategies to economically empower at-risk youth in El Salvador, adapted to the local environment. support of the Colombian Cooperation Agency, come to El Salvador to help implement its model of inclusion/disabilities. Col Café, for example, had developed its own sign language for deaf and mute workers unique to the coffee industry. agenda and related capacity building activities. Coordinated a meeting between the IP and APC to obtain their collaboration in this trilateral exchange. Contacted SENA and other Colombian experts to get their cooperation in receiving the group that was traveling to Colombia to share the expertise in the matter. Worked with SENA to prepare the capacity building topics to be included in the Agenda. Promoted a consensus between SENA and the IP to developing capacity building activities to be included in the trip agenda, ensuring that critical capacity exchange relationships were created between the Colombians and the Salvadorians. Helped establish a relationship between SENA and its counterpart in El Salvador, INSAFORP, for joining dialogue spaces RTC created for this purpose. Facilitated a wrap up meeting at the end of the trilateral exchange to get to find out how participants 120 Sponsoring Entity Trilateral Effort Trilateral Exchange Trilateral Exchange Results Relationships Strengthened/Formed RTC Role in Trilateral Exchange benefited from the exchange and to establish next steps. USAID/EG Office of Environment Environment Low Emissions Cattle Farming Supported the Northern Triangle countries on more productive and environmentally friendly cattle practices and shared the achievements obtained through the Low Emission Cattle Farming Cattle producers and other sector organizations within Northern Triangle have increased information sharing and productive collaboration on technical, scientific and institutional issues. Coordinated meetings with U.S. Department of Agriculture representatives to plan the capacity building activities included in the trilateral exchange agenda. Hired a consultant in Costa Rica to work with the local organizations to include the participation of the Northern Triangle countries. USAID/EG Office of Environment Environment Electricity System Strengthening Supported studies on infrastructure design and the Mexico SIEPAC market to be enriched. N/A Coordinated and held meetings between RTC, AMEXCID, and the U.S. Department of State to plan the trilateral exchange. Contacted the experts and invited them to participate in the trilateral exchange. Supported the preparation of the agenda for the experts. USAID/EG Office of Environment Environment Natural Gas Support the development of national natural gas markets and a commercial natural gas project in Central America. Increased the reliance on natural gas in order to reduce dependence on imported oil and to lower the cost of electricity. N/A Coordinated and held meetings with RTC, AMEXCID, the U.S. Department of State representatives to plan the capacity building activities included in the trilateral exchange agenda. 121 Sponsoring Entity Trilateral Effort Trilateral Exchange Trilateral Exchange Results Relationships Strengthened/Formed RTC Role in Trilateral Exchange Contacted the experts and invited them to participate in the trilateral exchange. 122 ANNEX H: SUMMARY DESCRIPTION OF OTHER TRILATERAL COOPERATION INITIATIVES 123 Description: The India, Brazil, and South Africa (IBSA) Facility for Poverty and Hunger Alleviation is an SSC fund that facilitates the execution of human development projects to advance the fight against poverty and hunger in developing countries. Northern Country(ies): N/A Provider (Pivotal) Country(ies): India, Brazil, South Africa Beneficiary Country(ies): Guinea Bissau, Cape Verde, Sierra Leone, Burundi, Cambodia, Haiti, Palestine, Lao PDR, and Vietnam Years of Operation: 2006-Present Operational Model: The IBSA Fund operates through a demand driven approach. Governments requesting support by this fund initiate discussions with focal points appointed among IBSA countries’ officers around the world. These focal points submit proposals to the IBSA Board of Directors for review. If a proposal receives a favorable review, the UNDP’s for South-South Cooperation, which acts as the fund manager and board of directors’ secretariat, initiates contact with a potential executing agency to advance a project formulation and to facilitate the project’s implementation. Cooperation projects should be need-driven, as well as locally owned and managed, to ensure sustainability and continuity of development outcomes on the ground. The IBSA Board of Directors meets quarterly to review and approve new projects as well as monitor the implementation of projects and provide strategic direction to ongoing activities. The SU-SSC acts as the fund manager and the Board of Directors’ Secretariat and initiates contact with potential executing agencies to advance the formulation of IBSA projects as to facilitate and support implementation. Financial Model: IBSA projects are executed through partnerships with the UNDP, national institutions, or local governments. Important concerns of IBSA partners in the design of their projects include capacity building among project beneficiaries, built-in project sustainability, and knowledge sharing among Southern experts and institutions. IBSA partners have committed to a minimum contribution of US$ 1,000,000 per year to finance fund activities. Contributions are made to UNDP, and once a project has been approved by the Board of Directors and opened, budgeting takes place through UNDP’s Atlas system Links: http://tcdc2.undp.org/IBSA/finance/finances.htm http://www.ibsa-trilateral.org/about-ibsa/ibsa-fund Description: The objective of the Public Sector Training and Development Project in Post‐Conflict Countries facility is to improve public service capability through institutional capacity development and training. Northern Country(ies): Canada (CIDA) Provider (Pivotal) Country(ies): South Africa (PALAMA) Beneficiary Country(ies): Burundi, Rwanda, Southern Sudan Years of Operation: 2008-2013 Operational Model: The training methodology is developed by PALAMA and the partner institutions in the three post-conflict countries and covers four areas: project management, financial management, human resource planning, and M&E. There is an ownership and mutual responsibility, which includes: equal participants in the planning and development, in-country political and government support, regional network sustained through the regional office, ownership and mutual accountability built in, impact of changing structures and leadership, procurement and control, and extent of equality and control in relation to implementing agency. Financial Model: Five-year capacity building project funded by CIDA (CAN$ 10.5 million or ZAR 70 million) Links:http://www.africaplatform.org/sites/default/files/the_regional_capacity_building_project_for_public_servic e.ppt http://www.oecd.org/dac/46387212.pdf Description: Political commitment to strengthen the justice sector and to build capacity and professionalize the Rwanda National Police (RNP) Northern Country(ies): Sweden (Swedish National Police Board) Provider (Pivotal) Country(ies): South Africa (South Africa Police Service) Beneficiary Country(ies): Rwanda Years of Operation: 2009-2013 Operational Model: The joint project management team forms the center of the tripartite cooperation. It is chaired by the RNP project manager and also includes the Swedish and South African counterparts. Financial Model: The tripartite collaboration is based on a cost sharing principle, which is specified in agreements between Sweden and Rwanda and between South Africa and Rwanda. The three parties finance their own salaries. 124 Links: https://www.sida.se/contentassets/4468c79dd4dd430ebdb25709d1ee4260/14961.pdf Description: The Trilateral Cooperation Secretariat (TCS) aims to serve as a hub for trilateral cooperation that encompasses a broad spectrum of sectors and stakeholders. With a view to solidifying the cooperative ties among Japan, Republic of Korea, and China. Northern Country(ies): Japan, Republic of Korea Provider (Pivotal) Country(ies): China Beneficiary Country(ies): N/A Years of Operation: 2015-Present Operational Model: The board, the executive decision-making body of the TCS, is comprised of a Secretary￾General and two Deputy Secretary-Generals, who are appointed by each country on a two-year rotational basis Financial Model: On the basis of equal participation, each country shares one-third of the total operational budget Links: http://www.tcs-asia.org/data/file/publication/977998899_PvnfpSy4_2016-2017_TCS_Annual_Report_- _EN.pdf Description: Third Country Training Program (TCTP) Northern Country(ies): Japan (JICA) Provider (Pivotal) Country(ies): Kenya, Senegal, Egypt, South Africa, Morocco, Tunisia, Tanzania, Uganda Beneficiary Country(ies): 45 Sub Saharan countries Years of Operation: 1974-Present Operational Model: With assistance from donor countries and aid organizations, a developing country accepts trainees from other developing countries with shared characteristics in order to transfer development expertise and skills. JICA generally implements this scheme through an organization in a developing country to which it provided assistance in the past, thus transferring Japanese skills and expertise to trainees in other developing countries in a manner that has been adapted to meet local conditions. TCTP training is given to either groups or individuals. TCTP group training is conducted by bringing together personnel engaged in the subject field from neighboring developing countries, whereas individual training is given primarily to the counterparts of JICA’s technical cooperation projects or expert dispatch implemented in other developing countries. Financial Model: JICA provides support for SSC by making use of the UNDP’s Human Resources Development Fund and holding international SSC conferences; Japan has been the largest financial contributor to UNDP’s various SSC support activities by establishing the Japanese Human Resources Development Fund in 1986 and appropriating a substantial portion of it for SSC. Japan and other donors (countries or organizations) combine their respective personnel, technological, and financial resources to mutually complement and cooperate with each other to resolve the development issues of a developing country or region. In relation to SSC, other donors are the developing countries. Japanese cooperation is targeted at end recipient countries, and a prerequisite is that this cooperation must conform to Japan’s priority areas for assistance to end recipient countries. In the case of the standard triangular training program, pivotal countries are expected to provide appropriate venues and facilities, develop and prepare a curriculum, select participants, and organize the actual training. The proportion of required contributions from the southern partners is determined on a case-by-case basis through dialogue, taking into account their financial capacity and other factors. JICA then provides its partners with various resources, including advisory support to develop curriculum and select participants, and the complementary financial contributions such as the travel and accommodation expenses of the participants, and honoraria for guest instructors from abroad. Links:http://gwweb.jica.go.jp/km/FSubject2101.nsf/03a114c1448e2ca449256f2b003e6f57/a86789dc7894cdd0492 579d4002b9963/$FILE/JICA_TG_South-SouthCooperation_En_2005.pdf 125 Description: Third Country Expert Dispatch Northern Country(ies): Japan (JICA) Provider (Pivotal) Country(ies): Philippines, Indonesia, Brazil, Kenya, Mali Beneficiary Country(ies): Sub Saharan Africa Years of Operation: 1995-Present Operational Model: With support from donor nations and aid organizations, experts from developing countries are dispatched to other developing countries in order to transfer their expertise and skills. There are two types of third-country expert dispatch. One is called the “complementary/support type,” which dispatches experts from developing countries to other developing countries as support staff for Japanese experts when the knowledge and skills of the third-country experts are deemed valuable for the effective implementation of technical cooperation by the Japanese experts. The other is known as the “dissemination type,” which dispatches the counterpart experts of Japanese technical cooperation when their knowledge and skills are deemed especially useful for the effective dissemination of the output of Japanese technical cooperation Financial Model: JICA provides support for SSC by making use of the UNDP’s Human Resources Development Fund and holding international SSC conferences; Japan has been the largest financial contributor to UNDP’s various SSC support activities by establishing the Japanese Human Resources Development Fund in 1986 and appropriating a substantial portion of it for SSC. Japan and other donors (countries or organizations) combine their respective personnel, technological, and financial resources to mutually complement and cooperate with each other to resolve the development issues of a developing country or region. In relation to SSC, other donors are the developing countries. Japanese cooperation is targeted at end recipient countries, and a prerequisite is that this cooperation must conform to Japan’s priority areas for assistance to end recipient countries. In the case of the standard triangular training program, pivotal countries are expected to provide appropriate venues and facilities, develop and prepare a curriculum, select participants, and organize the actual training. The proportion of required contributions from the southern partners is determined on a case-by-case basis through dialogue, taking into account their financial capacity and other factors. JICA then provides its partners with various resources, including advisory support to develop curriculum and select participants, and the complementary financial contributions such as the travel and accommodation expenses of the participants, and honoraria for guest instructors from abroad. Links:http://gwweb.jica.go.jp/km/FSubject2101.nsf/03a114c1448e2ca449256f2b003e6f57/a86789dc7894cdd0492 579d4002b9963/$FILE/JICA_TG_South-SouthCooperation_En_2005.pdf Description: Partnership Program Northern Country(ies): Japan (JICA) Provider (Pivotal) Country(ies): Asian countries: Singapore, Thailand, Philippine and Indonesia; Latin American countries: Chile, Brazil, Argentina, and Mexico; Middle-East and North African countries: Egypt, Tunisia, Morocco, and Jordan Beneficiary Country(ies): Other developing countries Years of Operation: 2002-Present Operational Model: This is a comprehensive framework through which the Japanese government and other developing country partner governments agree to jointly support the efforts of other developing countries and regions. Under this framework, JICA works jointly with its counterpart organizations in partner countries to implement Financial Model: JICA provides support for SSC by making use of the UNDP’s Human Resources Development Fund and holding international SSC conferences; Japan has been the largest financial contributor to UNDP’s various SSC support activities by establishing the Japanese Human Resources Development Fund in 1986 and appropriating a substantial portion of it for SSC. Japan and other donors (countries or organizations) combine their respective personnel, technological, and financial resources to mutually complement and cooperate with each other to resolve the development issues of a developing country or region. In relation to SSC, other donors are the developing countries. Japanese cooperation is targeted at end recipient countries, and a prerequisite is that this cooperation must conform to Japan’s priority areas for assistance to end recipient countries. In the case of the standard triangular training program, pivotal countries are expected to provide appropriate venues and facilities, develop and prepare a curriculum, select participants, and organize the actual training. The proportion of required contributions from the southern partners is determined on a case-by-case basis through dialogue, 126 taking into account their financial capacity and other factors. JICA then provides its partners with various resources, including advisory support to develop curriculum and select participants, and the complementary financial contributions such as the travel and accommodation expenses of the participants, and honoraria for guest instructors from abroad. Links:http://gwweb.jica.go.jp/km/FSubject2101.nsf/03a114c1448e2ca449256f2b003e6f57/a86789dc7894cdd0492 579d4002b9963/$FILE/JICA_TG_South-SouthCooperation_En_2005.pdf Description: UNDP has since 2011 worked with China’s Ministry of Commerce (MOFCOM) and Cambodia’s Ministry of Agriculture to help small processors and exporters of cassava move up the value chain, increase their revenue and cassava exports, and create jobs in the cassava sector. Northern Country(ies): UNDP Provider (Pivotal) Country(ies): China Beneficiary Country(ies): Cambodia Years of Operation: 2011-2015 Operational Model: Supports the technology transfer of cassava through transfers of tools such as targeted needs assessments of Cambodian cassava sector, tailored training, incubation. The first phase ran from December 27, 2011 to January 16, 2012 and consisted of a training program for 30 Cambodian agricultural practitioners. Based on the success of the initial phase, the Chinese government agreed to a second phase for which it provided US$ 400,000, the first time that the Chinese government ever provided grant support for a trilateral project with any partner. Phase II of the project was launched in May 2013 and completed in March 2015 Financial Model: Jointly funded: MOFCOM gave US$ 605,000, and UNDP China gave US$ 95,000 Links: http://www.cn.undp.org/content/china/en/home/library/south-south-cooperation/-.html http://www.cn.undp.org/content/china/en/home/operations/projects/south-south-cooperation/overview￾trilateral-cooperation-projects/china-cambodia-undp-trilateral-cooperation-on-cassava.html Description: In 2016, the Government of China, UNDP, and the Government of Malawi, through its Department of Disaster Management Affairs, agreed to implement the Disaster Risk Reduction Small Grants Scheme to contribute to disaster risk reduction and recovery among communities Northern Country(ies): UNDP Provider (Pivotal) Country(ies): China Beneficiary Country(ies): Malawi Years of Operation: 2016-2017 Operational Model: Throughout 2016, the scheme funded small grants projects submitted by NGOs and community-based groups to enhance the environmental and social sustainability of disaster management solutions in rural Malawi. Financial Model: Jointly funded: Government of China gave $900,000, and the UNDP gave $400,000. Links: http://www.cn.undp.org/content/china/en/home/library/south-south-cooperation/-.html http://www.cn.undp.org/content/china/en/home/operations/projects/south-south-cooperation/overview￾trilateral-cooperation-projects/china-cambodia-undp-trilateral-cooperation-on-cassava.html Description: Phase II Community Based Disaster Management in Asia Northern Country(ies): United Kingdom, UNDP Provider (Pivotal) Country(ies): China Beneficiary Country(ies): Bangladesh and Nepal Years of Operation: 2011-2016 Operational Model: The initiative consists of a global research and policy network with 18 research members and 16 government departments from six disaster-prone countries in Asia with peer-to-peer learnings to exchange experiences and knowledge. Financial Model: The model incorporates a fourth partner, often a traditional donor. In this case, an external donor provides finances to pilot the projects but also brings know how and experience. Denmark provides financing to the program. Denmark is increasingly focused on SSC so as to enable better and more coherent cooperation between China and countries in Africa, in particular around the promotion of the UN’s Sustainable Energy for All initiative. 127 Links: https://www.un.org/sustainabledevelopment/wp-content/uploads/2016/09/Southern-Alliance-of-Climate￾Friendly-Communities-0921.pdf http://www.cn.undp.org/content/china/en/home/library/south-south￾cooperation/-.html Description: Ghana-Zambia-China UNDP Trilateral Cooperation Northern Country(ies): Denmark, UNDP Provider (Pivotal) Country(ies): China Beneficiary Country(ies): Ghana and Zambia Years of Operation: 2014-2018 Operational Model: The response builds firmly on a trilateral setup where all parties play an active role and draw learning from the cooperation. China has concrete experience and technical knowledge in the renewable energy sector, while it needs to build project management capabilities and approach to SSC. Ghana and Zambia both have a gap in electrification, especially in rural areas, while these countries have unique insight into local context, including productive uses of energy and experience with past electrification projects that can be shared between the two countries. UNDP plays a facilitating, coordinating, and oversight role between and for the implementing agencies and contributes with its project management and learning process tools. Financial Model: It worked with a model that incorporates a fourth partner, often a traditional donor. In this case, an external donor provides finances to pilot the projects but also brings know how and experience. Denmark provides financing to the program. Denmark is increasingly focused on SSC so as to enable better and more coherent cooperation between China and countries in Africa, in particular around the promotion of the UN’s Sustainable Energy for All initiative. Links: http://www.cn.undp.org/content/china/en/home/library/south-south-cooperation/-. html http://www.cn.undp.org/content/china/en/home/operations/projects/south-south-cooperation/overview￾trilateral-cooperation-projects/china-cambodia-undp-trilateral-cooperation-on-cassava.html Description: The main purpose of the Alliance for Climate Friendly Communities is to bring together potential SSC partners to learn good practice approaches to ensuring sustainable development from each other. Northern Country(ies): UNDP Provider (Pivotal) Country(ies): China Beneficiary Country(ies): Bangladesh, Ecuador, Guinea, Jamaica, Mongolia, Sri Lanka, Seychelles, Tanzania, and Zimbabwe Years of Operation: 2016-Present Operational Model: Facilitation of a match-making process between developing countries to identify interventions that qualify for financial, technical, and equipment support from China’s energy program. Financial Model: UNDP offices support a component of a larger Chinese development program. Similar to the “10 – 100 – 1000” initiative by China to meet climate change targets set by COP21, China cooperates with developing countries to set up 10 model areas for low-carbon initiatives, establish 100 carbon reduction projects, and provide training to 1,000 people. It is also part of the USD $3.1 billion South-South Cooperation Climate Fund. Links: https://www.un.org/sustainabledevelopment/wp-content/uploads/2016/09/Southern-Alliance-of-Climate￾Friendly-Communities-0921.pdf http://www.cn.undp.org/content/china/en/home/library/south-south￾cooperation/-.html Description: Regional Fund for Triangular Cooperation in Latin America and the Caribbean Northern Country(ies): Development Assistance Committee (OECD), Germany (GIZ) Provider (Pivotal) Country(ies): LAC countries Beneficiary Country(ies): 21 LAC countries Years of Operation: 2010-2018 Operational Model: The fund consists of three action areas: 1) promotion and implementation of triangular cooperation projects, 2) capacity development and training (Red de Capacitación online platform), and 3) regional dialogue and conferences (policy level). Regional Conferences. Fund holds two calls for proposals per year in May and November. To data, the fund has supported more than 45 projects in cooperation with 21 countries. 128 Financial Model: Triangular cooperation projects are planned, financed, and implemented jointly by an established Development Assistance Committee (DAC) provider and an emerging economy that can be a South provider or beneficiary, and a third country as a beneficiary. The egalitarian nature of the partnerships is also reflected by the balanced financial contributions of the Latin American South providers to the triangular projects. The contribution of the Latin American provider is at least as large as the German contribution. Links: https://www.cooperacionsursur.org/images/Actividades_del_programa_2016/2016-05-18_BMZ￾GIZ_Regional_Fund_TrC_in_LAC.pdf https://www.giz.de/en/worldwide/12942.html Description: The aim of Triangular Cooperation Germany-Mexico is to pass in the experience gained from bilateral German-Mexican development cooperation so as to help develop other countries and to help Mexico develop its role as a donor in the region and make use of its existing capacities. Northern Country(ies): Germany (GIZ) and Federal Ministry of Economic Cooperation and Development (BMZ) Provider (Pivotal) Country(ies): Mexico (AMEXCID) Beneficiary Country(ies): LAC countries Years of Operation: Operational Model: The countries brought eight projects to a successful conclusion and have another eight ongoing. AMEXCID has taken a lead role in the initiation, support, and strategic orientation of triangular cooperation; the country’s respective specialist agencies are charged with consultancy and implementation. Financial Model: The technical and financial support are provided by AMEXCID and other by technical Mexican counterparts, and the other part by bilateral cooperation programs from GIZ. This model allows cooperation from a third party (a country or a regional organism) with technical support. Links: https://www.giz.de/en/worldwide/35928.html Description: Cooperation with Arab Donors in Middle East and North Africa (MENA) Northern Country(ies): Germany (GIZ, BMZ) Provider (Pivotal) Country(ies): Arab donors such as, AGFUND, Islamic Development Bank (IsDB), Silatech, and the OPEC Fund for International Development (OFID) Beneficiary Country(ies): Countries from Middle East and North Africa (e.g., Yemen, Jordan, Morocco, Egypt) Years of Operation: 2009-2018 Operational Model: Through a grant, OFID is supports the Arab-German master studies program of Integrated Water Resources Management that allows Arab students to join an academic program at specialized universities in Jordan and Germany. OFID also joined GIZ in supporting Arabterm.org, a technical online dictionary. Furthermore CAD, through the Open Regional Fund, and OFID joined forces in Jordan through the Madrasati Project in order to improve the learning environment in public schools with large numbers of Syrian refugees and will soon start funding two Village Development Councils in rural areas of Jordan in cooperation with IsDB. Financial Model: Through a third party, BMZ and GIZ manage an open regional fund, which supports projects in the MENA region. Each project is planned, financed, and implemented in close cooperation with at least one Arab donor organization. Links: http://www.oecd.org/dac/dac-global-relations/Cooperation-with-arab-donors-in-MENA-flyer-June￾2016.pdf Description: Brazilian-German Trilateral Cooperation Northern Country(ies): Germany (GIZ) Provider (Pivotal) Country(ies): Brazil (Brazilian Cooperation Agency, ABC) Beneficiary Country(ies): Other development countries (e.g., Mozambique, Peru) Years of Operation: 2010-Present Operational Model: The technical trilateral cooperation carried out by the governments of Brazil and Germany to benefit other countries shares the following principles: ownership of the general process by the third country; focus on recognizing local knowledge and experience; joint execution (planning, implementation, monitoring, and evaluation) by the three parties and a horizontal approach in relationships; outcome-guided management; and demand-driven approach by the beneficiary country, guaranteeing the alignment with that country’s goals and systems. The combination of the complementary technical contributions and the application of specific knowledge 129 and experience of the three countries involved are requirements for the joint execution of trilateral technical cooperation projects by Brazil and Germany. Furthermore, this technical cooperation is the result of a demand originated by the third country and/or by a complementation opportunity identified in the bilateral cooperation by Brazil or Germany with the third country. The management structure of the Brazilian-German Trilateral Cooperation is made up of three levels: Joint Coordination Committee (made up of representatives of the third country and their deputies, ABC and GIZ, assigned to the direction or coordination level), Management Unit (made up of a representative of the executing institution in the third country, a project analyst of ABC, and a technical assistant or consultant with GIZ), Executing Unit (made up of technicians from the third country’s executing institutions engaged in the implementation of the activities described in the annual operational plan, technicians of the Brazilian executing institutions, and technical assistants or consultants from GIZ). Financial Model: On the Brazilian side, the projects are funded through ABC’s budget, and the technical hours dedicated by the professionals of the Brazilian governmental institutions who are responsible for the contribution of technical knowledge. The administration costs are not accounted for, neither the ones related to the team responsible for the project management. On the German side, the projects are funded, preferentially, by a Regional Fund for Trilateral Projects with Latin America and the Caribbean of the BMZ. The Regional Fund allows a contribution of up to 300 thousand euros from the German side per project approved. The funding can also be complemented by the bilateral programs and projects of Germany with third countries and/or by the Brazilian￾German Trilateral Cooperation Program. The funding sum includes the administration costs and the ones related to teams responsible for the project management. Brazil and Germany fund their respective technical contributions in trilateral projects. Whenever possible, is equivalence in the amounts spent by both countries in each project; the contribution of each partner must be between 30 percent and 70 percent. The counterpart of the benefited institutions in the third countries must be, for example, through a team dedicated to the managerial and technical implementation of the whole project cycle, the coordination with national agents, equipment and infrastructure, and logistic support to the execution of the activities foreseen in the project. Links: https://www.giz.de/en/downloads/giz2015-pt-en-sp-brazil-german-trilateral-cooperation-manual.pdf Description: During 2014-2015, the United Nations International Children’s Emergency Fund (UNICEF) facilitated 142 SSC initiatives between Brazil and other developing countries as part of its global partnership with the Government of Brazil. UNICEF promoted horizontal exchanges by connecting supply and demand for knowledge, expertise, and development innovations. It also provided technical input and operational support and facilitated outreach for potential partners to mobilize resources for SSC. Together with the Brazilian government, UNICEF has advocated for children’s rights in global policy processes. Northern Country(ies): UNICEF Provider (Pivotal) Country(ies): Brazil Beneficiary Country(ies): Other developing countries Years of Operation: 2014-Present Operational Model: SSC between Brazil, other developing countries, and UNICEF takes place in two key modalities: study tours and projects. Study tours involve missions comprised of policy-makers, experts, officials, and other representatives of governments and NGOs from partner countries who visit Brazil to engage in policy dialogue and share lessons and practices with Brazilian counterparts. SSC projects promote capacity development at the partner country level to formulate and implement policies in priority areas to advance children’s and women’s rights. Projects are designed in accordance to SSC principles in general, and the country program agreed between UNICEF and national authorities at the partner country level. Financial Model: The Brazil-UNICEF trilateral initiative has been financed through the following sources: • Brazilian support in the form of in-kind contributions (experts, technical and material resources, logistic support) and, under specific conditions, financial support. • Partner governments and non-state actors in the private sector or civil society of the partner country, through in-kind contributions in the form of access to office and training space, sundries, logistical support, transportation, accommodation, cost of visas, etc. This can also include the sharing of costs through financial contributions, UNICEF’s financial and technical contributions, and financial support from ABC. • Direct grants from a fourth party, such as other multilateral organizations, traditional donors, or other developing partners to budgeted activities. The expenditures of the SSC unit of UNICEF Brazil throughout the period of 2014-15, including two staff (P4 and GS6), amounted to approximately $960,000. 130 Links: https://drive.google.com/file/d/0B-buqyoV0jpSSTZrODh0N1lZd2M/view Description: The India-UN Development Partnership Fund supports Southern-owned and led, demand-driven, and transformational sustainable development projects across the developing world, with a focus on least developed countries and small island developing states. United Nations agencies implement the fund’s projects in close collaboration with partnering governments. Northern Country(ies): United Nations Provider (Pivotal) Country(ies): India Beneficiary Country(ies): Antigua and Barbuda, Belize, Benin, Burkina Faso, Chad, Dominica, Grenada, Liberia, Mali, Mauritania, Palau, Papua New Guinea, Saint Vincent and the Grenadines, Sao Tome and Principe, Seven Pacific Island Countries, Tonga, Tuvalu, Uruguay, and Vanuatu Years of Operation: 2017-Present Operational Model: The India-UN Development Partnership Fund supports specific projects agreed to with the Government of the Republic of India, in accordance with UNDP policies for subsidiary trust fund arrangements with specific donors and development partners. Proposals may be presented by the governments of developing countries, in particular small island developing states and least developed countries where India only has concurrent diplomatic accreditation. The Board will confirm the implementing agency. The Secretariat, the partner country, or the Board can suggest a suitable implementing agency. Project documents are approved by the Board of Directors at their annual meeting or during extraordinary or ad hoc meetings. The implementing agency monitors project activities in line with its policies and procedures. Additional monitoring is conducted by partnering governments, Indian representatives accredited in/for the partner country/ies and, where applicable, the United Nations Office for SSC as fund manager and secretariat of the Board of Directors Financial Model: India has committed $100 million to the Fund over the next decade. Links: https://www.unsouthsouth.org/partner-with-us/india-un-fund/#updates https://info.undp.org/docs/pdc/Documents/TUV/India-UN%20Programme%20Guidelines%20(Distribution).pdf Description: In these initiatives, the United Kingdom (UK) Department for International Development (DFID) mainly draws on technical assistance to build the capacity of partners in India and other developing countries and facilitate their relationships. The overarching objective is to maximize the development impact of Indian partners in their varying roles of generating research. India-UK cooperation is operationally relatively disconnected from their respective bilateral programs with third countries Northern Country(ies): UK (DFID) Provider (Pivotal) Country(ies): India Beneficiary Country(ies): Developing countries in South Asia and Africa. Years of Operation: 2011-Present Operational Model: Some projects are completely designed as triangular partnerships, others include triangular cooperation as a sub-component. Other projects focus mainly on cooperation in India but have a component for cooperation in third countries. An important feature of triangular cooperation in the India-UK partnership is the role played by DFID/India (with its “Global Partnerships Team”) in building partnerships. Projects also vary with regard to types of partners, including think tanks, international organizations and CSOs; other programs focus especially on the Indian private sector. Compared to other examples of triangular cooperation, India-UK cooperation is characterized by a weak government-to government dimension. The India-UK practice of triangular cooperation can be seen as spearheading this trend by focusing especially on non-state actors as partners. Financial Model: On both sides, constraints on how to use funds contribute to this separation (e.g., UK aid cannot directly fund governmental bodies in India anymore; Indian development finance is often tied to Indian implementation partners). While the UK’s bilateral program with India can fund non-state partners, funds for partners in third countries do not come from the UK’s or India’s bilateral portfolios for these countries but from a separate UK funding source (initially from the centrally managed Global Partnerships Program, then from regional departments). All the projects are jointly funded. Links: https://www.orfonline.org/wp-content/uploads/2018/03/ORF_WorkingPaper_Triangular_Cooperation.pdf 131 Description: Facility exchanges expertise, knowledge, and resources, and promotes business-to-business links and investments across developing countries with the broad aim of enhancing agricultural productivity and the livelihoods of poor people living in rural areas Northern Country(ies): United Nations International Fund for Agriculture and Development (IFAD) Provider (Pivotal) Country(ies): China Beneficiary Country(ies): Developing Countries Years of Operation: 2018-Present Operational Model: IFAD’s main counterpart for the facility within the Government of China is the Ministry of Finance. The work of the Facility is supported by an Advisory Panel, consisting of representatives of China’s Ministry of Finance and IFAD. The role of the panel is to: 1) review and endorse the annual Strategic Guidance Note, 2) periodically monitor the implementation of SSTC activities undertaken by the Facility and suggest future priorities, and 3) review individual project proposals to be financed through the Facility and recommend their approval by the Ministry of Finance. Financial Model: Initial funding: US$ 10 million (until the end of 2021) Links: https://www.ifad.org/web/knowledge/publication/asset/40691635 Description: FAO has worked with close to 100 partners to support the mutual sharing and exchange of knowledge, good practices, and policies and technology between and among countries in the global South. The partners play various roles, whether involving countries sending their experts abroad, countries hosting experts from abroad, or third-party countries providing financial and/or technical resources to facilitate SSC exchanges. The SSC and Resource Mobilization Division at FAO’s headquarters and SSC Regional Officers collaborate closely with a broad network of country and regional offices, to promote the progress of a wide range of initiatives, as well as to provide international visibility for each cooperating partner. Northern Country(ies): FAO - African Development Bank, European Union, IsDB, Japan, France, Monaco, Mexico, Netherlands, Oman, Republic of Korea, Saudi Arabia, Spain, Switzerland, United Kingdom, Venezuela, West African Economic and Monetary Union, and World Bank Provider (Pivotal) Country(ies): Algeria, Bangladesh, Brazil, Cambodia, Chile, China, Cuba, Egypt, India, Indonesia, Iran, Jordan, Lao People’s Democratic Republic, Malaysia, Morocco, Myanmar, Pakistan, Philippines, Thailand, Turkey, and Vietnam Beneficiary Country(ies): Countries in Global South Years of Operation: 2012-Present Operational Model: FAO set up an SSC unit at headquarters and placed SSC officers in each region. The overarching strategy builds on FAO’s successful track record and sets out renewed direction, with focus on: • Facilitating exchanges of development solutions at the grassroots level. Exchanges can occur through multiple means, including through the deployment of experts, policy dialogues, technology exchanges, study tours, and learning programs. FAO provides practical guidance and support to ensure the quality of the knowledge￾sharing. • Promoting SSC knowledge networks and platforms at the institutional level. • Providing upstream policy support. There are increasing opportunities to share policy, lessons learned, and best practices in one country that can be leveraged to enhance policy-making capacities in another. Among the methods of making this happen are high-level platforms and events, including workshops and other effective channels for policy dialogue, including parliamentarian forums. • Fostering an enabling environment for effective SSC. This includes mainstreaming SSC as a means to deliver on FAO’s strategic objectives, build broader partnerships and strategic alliances, increase visibility, and mobilize resources. FAO also fosters the formulation of projects and agreements, provides technical backstopping as well as instruments and program guidance to support cooperation. Financial Model: Extrabudgetary resources are sought in the context within which SSC activities are implemented (at global, regional, and country level). In addition, the prospect of obtaining in-kind contributions is being perused (including human resources, such as through secondment and UN volunteers or interns). In keeping with the theme of cooperation, FAO anticipates that each partner contributes financially to ensure the successful ownership and implementation of SSC initiatives. To this end, FAO supports outreach to potential resource partners to mobilize resources for SSC. FAO secures resources through multiple channels. Beneficiary country governments contribute financially to the cost of implementing SSC in country by establishing Unilateral Trust Funds. Provider countries may also provide financial resources through the creation of government cooperative 132 projects and trust funds and through in-kind contributions, including paying the home salaries of the experts and language training prior to deployment. Traditional donors, emerging countries, and multilateral organizations facilitate SSC through the provision of resources to allow the exchange to happen. Some triangular cooperation partners also provide technical assistance where necessary. To date, FAO’s main SSC providers are Brazil and China, whose main focus is on Latin America and Africa, respectively. Other SSC providers include Mexico, Morocco, and Venezuela, as well as triangular partners such as Japan and the Republic of Korea. Links: http://www.fao.org/3/a-i6249e.pdf 133 ANNEX I: TRILATERAL COOPERATION ACTIVITIES IN EL SALVADOR BY SELECTED MULTILATERAL DONOR AGENCIES 134 GIZ operates the Regional Fund for Cooperation in the LAC region. The fund consists of three action areas: 1) promotion and implementation of trilateral projects; 2) capacity development and training; and 3) regional dialogue and conference (policy level). The fund currently has 60 projects in the LAC region with contributions from Germany and from a South provider country. The contribution of the Latin American provider is at least as large as the German contribution. Projects are jointly designed, planned, implemented, and evaluated under the coordination of entities that manage cooperation in the different countries (e.g., the MINEX in El Salvador). The EU does not have triangular cooperation as a direct modality. Its cooperation responds mostly to bilateral TA contracts/agreements with El Salvador’s MOF with funds that may include TA exchanges with other countries. The EU has a TA agreement with GOES’ Social and Economic Plan, and funds allocated for this plan may also be used for bringing and/or sending technicians and experts from and/to other countries. At the regional level, triangular cooperation is more systematic. From Brussels, a fund close to €15 million is strategically distributed among members of a consortium of regional countries that request or need TA exchanges on different themes, especially dialogue and public policy implementation. This cooperation is managed directly from Brussels. The leader of the consortium signs a subsidy contract with EU. The EU announces a call for proposals, and countries interested in receiving/providing cooperation submit their proposals. Two years ago, 6-7 proposals with the best design, clear objectives, expectations, and well￾founded budget were selected and are now being implemented. In 2011/2012, AECID signed a trilateral SSC memorandum with El Salvador as a provider country. Under this agreement, AECID works directly with the Vice Ministry of Cooperation for Development (in the MINEX) and with El Salvador’s counterpart funds. During 2014-2017, for example, El Salvador’s Micro and Small Enterprise (MSME) Development Centers that work under the National Commission for Micro and Small Enterprises (CONAMYPE), in alliance and coordination with AECID, provided assistance to the Ministry of Industry, Commerce and MSME of the Dominican Republic for the development of nine MSME development centers. AECID channeled the cooperation agreement between the two governments and together identified the key nexuses of the executing institutions. AECID also collaborates with El Salvador on South-South cooperation as a recipient country. In the past four years, AECID has executed approximately 10-12 projects through its South-South trilateral cooperation strategy. For example, AECID and Argentina jointly worked with the El Salvador National Youth Institute (INJUVE) to strengthen its capacity to implement an employment and employability program (Youth with Everything) and a participation strategy, through the exchange of knowledge and experience between counterpart. There is no specific trilateral cooperation fund; they fall under the Salvadoran Fund for South￾South and Triangular Cooperation of El Salvador’s Vice Ministry of Cooperation for Development (VMCD). Initiatives are financed in three different ways: 1) initiatives of El Salvador as a provider and capacity building via the Counterpart Fund for Institutional Strengthening for Development Spain-El Salvador, with direct management by Direct management by the VMCD; 2) initiatives of El Salvador as recipient with Mexico, Chile and Colombia through the respective trilateral cooperation programs, managed by the development agencies of these countries; and 3) initiatives 135 of El Salvador as recipient with Peru, Ecuador, and Argentina through AECID funds specific to the Counterpart Fund, with direct management by the VMCD and the sectoral entity involved. 136 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523