EVALUATION FINAL PERFORMANCE EVALUATION REGIONAL INTEGRATED TRADE AND FOOD SECURITY STRATEGY 2011-2017 August, 2017 This publication was produced at the request of the United States Agency for International Development (USAID). It was prepared independently by Mendez England & Associates under contract AID-OAA-I-15-00024 / AID-519-TO-16-00002. REGIONAL INTEGRATED TRADE AND FOOD SECURITY 2011-2017 FINAL PERFORMANCE EVALUATION FINAL REPORT AID-OAA-I-15-00024 / AID-519-TO-16-00002 August, 2017 Cover image: A Mayan cocoa producer participating of the RUTA project Petén Guatemala. Credit: Joke Vuurmans. This report was prepared by Dr. Luis Bernal, Evaluation Team Leader; Amy Angel and Rafael Cornejo, Trade Experts; and Fernando Hernandez and Joke Vuurmans, Agriculture Experts. DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ABSTRACT The purpose of the final performance evaluation of the Regional Integrated Trade and Food Security Strategy 2017-2017 (the Strategy), a USAID Central America (USAID/CAM) initiative, was to determine if the design and implementation of the four activities under the Strategy were effective in achieving the Strategy’s expected results, and to inform on future programming in trade and food security based on that analysis. The evaluation covered implementation of the Strategy from its inception in September 2011 through September 30, 2016. It was based on evaluation questions and evaluation criteria, and included extensive document reviews, key informant interviews, site visits and an online survey. Its field work included interviews in El Salvador, Guatemala, Honduras, Costa Rica and the U.S. The evaluation found that strengthening of national governments’ and regional entities’ capacity for analysis and policy formulation related to trade facilitation and food security was one of the major accomplishments of the activities. The Strategy activities have contributed to producer organizations capacity building, but the impact on their income remains to be seen. There were no significant instances of collaboration among the four Strategy’s activities, nor were they mandated or monitored. Same can be said about gender integration activities or interventions. Activities did contribute to the dissemination and use of science, technology and innovation for agricultural practices among small farmers, mostly through training and technical assistance. CONTENT EXECUTIVE SUMMARY ____________________________________________________________________ i 1. INTRODUCTION _______________________________________________________________________ 1 1.1. EVALUATION PURPOSE_____________________________________________________ 1 1.2. EVALUATION AUDIENCE ___________________________________________________ 1 1.3. EVALUATION QUESTIONS __________________________________________________ 1 1.4. EVALUATION CRITERIA ____________________________________________________ 2 1.5. A NOTE ON VOCABULARY AND ACRONYMS ___________________________________ 3 2. STRATEGY BACKGROUND _____________________________________________________________ 3 3. EVALUATION METHODS AND LIMITATIONS ____________________________________________ 5 3.1. METHODOLOGY___________________________________________________________ 5 3.2. EVALUATION ACTIVITIES __________________________________________________ 6 3.3. EVALUATION LIMITATIONS _________________________________________________ 8 4. EVALUATION FINDINGS _______________________________________________________________ 8 4.1. FINDINGS ON THE EVALUATION QUESTIONS __________________________________ 8 4.1.1. To what extent did the activities strengthen national governments’ and regional entities’ capacity for analysis and policy formulation related to trade facilitation and food security? __ 8 4.1.2. To what extent did technical assistance provided to SIECA resulted in improved organizational capacity to support the Central American economic integration process? ___10 4.1.3. To what extent did the grants program and technical assistance build the capacity of agriculture local partners and small producer organizations? _________________________11 4.1.4. To what extent have activities contributed to market alliances and mechanisms that will likely last beyond USAID activity implementation? ____________________________________12 4.1.5. How has collaboration between the four activities affected Strategy and activity results?___13 4.1.6. To what extent have the activities contributed to the dissemination and use of science, technology and innovation for sustainable agricultural practices among small farmers? ____14 4.1.7. To what extent have the activities contributed to the dissemination and use of science, technology and innovation for trade facilitation? __________________________________15 4.1.8. To what extent have the activities contributed to the dissemination and use of science, technology and innovation for a reduction in times and costs to cross borders? __________16 4.1.9. How did each activity promote gender integration and social inclusion?________________16 4.1.10. Gender___________________________________________________________________16 4.1.11. Social Inclusion ____________________________________________________________17 4.1.12. (IFC) How has the Regional System of Mutual Recognition of Sanitary Registries facilitated, or not, trade of food and beverages in Central America? ____________________________17 4.1.13. How did RUTA’s activity change the capacity of communities of practice to perform as networks disseminating best practices? _________________________________________18 4.1.14. (RTMA) To what extent have the activities under the RTMA supported producers of selected agricultural value chains to improve access to regional and export markets?_____________19 4.2. FINDINGS ON THE EVALUATION CRITERIA 20 4.2.1. Relevance ________________________________________________________________21 4.2.2. Effectiveness______________________________________________________________24 4.2.3. Efficiency ________________________________________________________________30 4.2.4. Sustainability _____________________________________________________________31 4.2.5. Cross Cutting Issues ________________________________________________________32 5. CONCLUSIONS _______________________________________________________________________32 6. RECOMMENDATIONS_________________________________________________________________ 38 ANNEX 1. Annotated Statement of Work _________________________________________________________42 ANNEX 2. Original Statement of Work___________________________________________________________43 ANNEX 3. Detailed Evaluation Methodology ______________________________________________________63 ANNEX 4. Evaluation Matrices ________________________________________________________________67 ANNEX 5. Evaluation Timeline _______________________________________________________________140 ANNEX 6. Persons Interviewed________________________________________________________________147 ANNEX 7. Evaluation Team ROLES and Conflict of Interest Statements _______________________________158 ANNEX 8. Data Collection Protocols and Instruments ______________________________________________165 ANNEX 9. Documents Reviewed ______________________________________________________________172 ANNEX 10. Focus Group Discussions on the Effect of the RTMA Activity _____________________________181 ACRONYMS ACOPASCA San Carlos Cooperative Association (Asociación Cooperativa San Carlos)- El Salvador ACPACI Cooperative Association of Cocoa Agricultural Production Cocoa (Asociación Cooperativa de Producción Agropecuaria Cacao) El Salvador ADS Automated Directives System ADECHI Farmers Association for the Integral Development of Concepción Chiquirichapa. (Asociación de Agricultores para el Desarrollo Integral de Concepción Chiquirichapa) Guatemala ADIBA Association for the Integral Development of Buenos Aires. (Asociación para el Desarrollo Integral Buenos Aires) Guatemala AEO Authorized Economic Operator AGCID Chilean International Development Cooperation Agency (Agencia Chilena de Cooperación Internacional para el Desarrollo) AGEXPORT Guatemalan Exporters Association (Asociación Guatemalteca de Exportadores) AMCHAM American Chamber of Commerce AMS USDA / Agricultural Marketing Service ANACAFE Coffee National Association of Guatemala (Asociación Nacional del Café Guatemala) AOR Agreement Officer's Representative APROCACAHO Cocoa Farmers´ Association of Honduras (Asociación de Productores de Cacao de Honduras) APROCEL Producers Association of Celaque. (Asociación de Productores de Celaque) Honduras ARSAGRO Regional Association of Livestock and Agriculture Services of the East (Asociación Regional de Servicios Agropecuarios de Oriente) Honduras ASADIT Farmers Association for the Integral Development of Tejutla (Asociación de Agricultores de Desarrollo Integral de Tejutla) Guatemala ASOPRANO Agriculture and Livestock Producers Association of the North of Olancho (Asociación de Productores Agropecuarios del Norte de Olancho) Honduras ATESCATEL Integral Farming Cooperative ATESCATEL (Cooperativa Agrícola Integral ATESCATEL) Guatemala BANADESA Honduras – National Bank for Development of Agriculture and Livestock. (Banco Nacional de Desarrollo Agropecuario) BFS Bureau of Food Security (USAID) CAC Central American Council of Agriculture (Consejo Agropecuario Centroamericano) CAFEL Ecological Fraternity Coffee-farming Cooperative (Cooperativa Cafetalera Fraternidad Ecológica) CAFTA-DR United States, Central America and Dominican Republic Free Trade Agreement CAMAGRO Agriculture, Livestock and Agroindustry Chamber of El Salvador (Cámara Agropecuaria y Agroindustrial de El Salvador) CATIE Tropical Agricultural Research and Higher Education Center - CATIE (Centro Agronómico Tropical de Investigación y Enseñanza) CATRANSCA Central American Chamber of Transport (Cámara Centroamericana del Transporte) CCIE Consultative Committee for Economic Integration (Comité Consultivo de la Integración Económica) CENSALUD Center of Research and Development of Health, (Centro de Investigación y Desarrollo en Salud) CENTA National Center of Agricultural and Forestry Technology, (Centro Nacional de Tecnología Agropecuaria y Forestal “Enrique Álvarez Córdova”) CFS Certificate of Free Scale CIAT International Center for Tropical Agriculture (Centro Internacional de Agricultura Tropical) COEXPORT El Salvador Corporation of Exporters (Corporación de Exportadores de El Salvador) COMESCA Trading Company Meseta de los Cuchumatanes COMIECO Central American Economic Integration Ministers’ Council (Consejo de Ministros de Integración Económica Centroamericana) CONADEA National Council of Agricultural Development, Guatemala (Consejo Nacional de Desarrollo Agropecuario) CONASAN National Council of Food and Nutrition Security, El Salvador. (Consejo Nacional de Seguridad Alimentaria y Nutricional) COP Chief of Party COPIASURO Southeast Apiculture Production Integral Cooperative (Cooperativa de Producción Integral de Apicultores del Sur Occidente) Guatemala COR Contract Officer's Representative CTPT Plan Trifinio Tri-national Commission (Comisión Trinacional del Plan Trifinio) DO Development Objective ECARAI Intibuca Farmer Agroindustrial Company of the Agrarian Reform (Empresa Campesina Agroindustrial de la Reforma Agraria de Intibucá) ECO Cooperation Studies Association (Asociación Estudios de Cooperación – ECO) Guatemala ET Evaluation Team FAST Food Safety Agricultural Sustainability and Traceability Program FECATRANS Central American Federation of Transport (Federación Centroamericana del Transporte) FEDECOAG Coffee Producer Cooperatives Federation, Guatemala (Federación de Cooperativas Agrícolas de Productores de Café) FGD Focus Group Discussions FICCI Ixcan Cooperatives Commercialization Integral Federation (Federación Integral de Comercialización de Cooperativas de Ixcán) Guatemala FHIA Honduran Foundation for Agricultural Research (Fundación Hondureña de Investigación Agrícola) FECAICA Federation of Industrial Chambers and Associations of Central America and the Dominican Republic. (Federación de Cámaras y Asociaciones Industriales de Centroamérica y República Dominicana) FUNDER Foundation for Rural Development (Fundación para el Desarrollo Empresarial Rural) FUSADES Salvadorian Foundation for Social and Economic Development (Fundación Salvadoreña para el Desarrollo Económico Social) FSMA Food Safety Modernization Act FTF Feed the Future HORTISA Lepaterique Horticultural Company (Empresa Hortícola de Lepaterique) IDB or IADB Inter-American Development Bank IFC World Bank Group’s International Finance Corporation IFPRI International Food Policy Research Institute IHCAFE Coffee Institute of Honduras (Instituto Hondureño del Café) IICA Inter American Institute for Agricultural Cooperation (Instituto Interamericano de Cooperación para la Agricultura) IP Implementing Partner IPM Integrated Pest Management KII Key Informant Interviews MAG Ministry of Agriculture MCT Coffee Roundtable of the Trifinio Region (Mesa del Café de la Región de Trifinio) M&E Monitoring and Evaluation MOU Memorandum of Understanding NGO Non-Governmental Organization OIRSA Regional International Phyto and Zoo-Sanitary Organism (Organismo Internacional Regional de Sanidad Agropecuaria) OECD Organisation for Economic Co-operation and Development PAPA Participating Agency Program Agreement PAG Global Village Project (Proyecto Aldea Global) Honduras PRESANCA Regional Program of Food Security and Nutrition (Programa Regional para Seguridad Alimentaria y Nutricional para Centroamérica) PRESISAN Regional Program of Information Systems on Food Security and Nutrition (Programa Regional de Sistemas de Información en Seguridad Alimentaria y Nutritional para Centroamérica) PROESA El Salvador Export and Investment Promoting Organism (Organismo Promotor de Exportaciones e Inversiones de El Salvador) PROMECAFE Cooperative Regional Program for Technological Development and Modernization of Coffee Growing (Programa Cooperativo Regional para el Desarrollo Tecnológico y la Modernización de la Caficultura) PROVIASA Vegetable Producers of Izopo and Azacualpa (Productores de Vegetales de Izopo y Azacualpa) Honduras RDCS Regional Development Cooperation Strategy REDMUCH Women's Network Producing Cacao and Chocolate in Honduras (Red de Mujeres Cacaoteras y Chocolateras de Honduras) RTMA Regional Trade and Market Alliances Project RUTA Regional Unit for Technical Assistance (Unidad Regional de Asistencia Técnica) SAG Honduras - Secretary of Agriculture and Livestock (Secretaría de Agricultura y Ganadería) SEC Central America Trade Statistics System (Sistema de Estadísticas de Comercio Centroamericano) SECAC Secretariat of the Central American Council of Agriculture (Secretaría Ejecutiva del Consejo Agropecuario Centroamericano) SEDIS Development and Social Inclusion Secretary Honduras (Secretaría de Desarrollo e Inclusión Social) SENASA Honduras - National Agricultural Health Service (Servicio Nacional de Sanidad e Inocuidad Agroalimentaria) SEPSA Costa Rica- Executive Secretariat of Sectorial Agricultural Planning (Secretaría Ejecutiva de Planificación Sectorial Agropecuaria) SESAN Food Security and Nutrition Secretary (Secretaría de Seguridad Alimentaria y Nutricional) Guatemala SICA Central American Integration System (Sistema de Integración Centroamericana) SIECA Central American Secretariat for Economic Integration (Secretaría de Integración Económica Centroamericana) SIMPAH Honduras – Agriculture Products Market Information System (Sistema de Información de Mercados de Productos Agrícolas) SOW Scope of Work SPS Sanitary and Phytosanitary Standards UNAH National University of Honduras (Universidad Nacional de Honduras) UNOPS United Nations Office for Project Services UPEG Planning and Evaluation Unit (Unidad de Planificación, Evaluación y Gestión) Honduras USA United States of America USAC University of San Carlos (Universidad de San Carlos) USAID/CAM United States Agency for International Development/Central America and Mexico USDA U.S. Department of Agriculture USG United States Goverment UTSAN Technical Unit of Food and Nutritional Security, Honduras (Unidad Técnica de Sanidad Alimentaria y Nutricional) WBG World Bank Group i EXECUTIVE SUMMARY EVALUATION PURPOSE AND QUESTIONS The Regional Integrated Trade and Food Security Strategy 2011-2017 (the Strategy) is a USAID Central America and Mexico (USAID/CAM) initiative made up by four activities: 1. The Promoting Food Security and Trade Integration through Sanitary and Phytosanitary Standards and other Ag-Related Capacity Building (PAPA) 2. The Regional Sustainable Agriculture and Food Security Program (RUTA) 3. The Regional Trade and Market Alliances (RTMA) 4. The Central America Agribusiness Trade Logistics (IFC) As per the Scope of Work (SOW), the Evaluation Team (ET) was tasked with the evaluation of the Strategy and not of its four activities; however, the understanding of and evaluation of the activities were considered essential in order to evaluate the overall performance of the Strategy. The final performance evaluation covered implementation of the Strategy from its inception in September 2011 through September 30, 2016. The evaluation was conducted by a team of five members: a team leader, two experts on trade, and two experts in agriculture. The evaluation comprised field work in El Salvador, Guatemala, Honduras, Costa Rica and the U.S., and was carried out between December 2016 and April 2017. The main purpose of this final performance evaluation was to assess if the design and implementation of the four activities under the Strategy were effective in achieving the Strategy’s expected results, and to inform on future programming in trade and food security based on that assessment. The evaluation was based on evaluation questions and evaluation criteria provided in the SOW. The evaluation questions, asked by USAID/CAM, referred to: 1) the activities and their contribution to strengthening of local institutions; 2) collaboration among them to obtain the Strategy results; 3) their contribution to the dissemination and use of science, technology and innovation; 4) promotion of gender integration and social inclusion; and 5) contribution to changes in income, consumption patterns and living standards of small farmers, and their influence on immigration. Besides answering those questions, the performance of the activities and the Strategy were evaluated under the following criteria: relevance, effectiveness, efficiency, and sustainability. The principal audience of this evaluation is USAID/CAM, USAID/ Bureau for Food Security (BFS), USAID/Bureau for Latin America and the Caribbean, and other USAID offices that are expected to use the evaluation to provide a retrospective analysis of activity effectiveness to achieve the results individually and combined under the Strategy. EVALUATION METHODOLOGY The evaluation adopted a non-experimental design. It focused not only on the performance of the Strategy per se but also on its activities’ performance, and whether both the Strategy and its activities have made any impact resulting from a before-after comparisons. As a performance evaluation, it was designed to analyze the Strategy to identify results and accomplishments; recognize performance issues and constraints in its implementation; and determine how the Strategy is perceived and valued. ii The evaluation was conducted in four successive stages: 1) review and analysis of secondary information and activity-related information, including identification of stakeholders and sampling them for field work; 2) primary information data capture; 3) data analysis; and 4) evaluation reporting. The ET reviewed 189 documents; had 127 key informant interviews (KIIs) with 192 persons; convened 9 focus group discussions (FGDs) with 88 participants, including 33 women; visited 9 agriculture and/or manufacturing sites; and conducted one online survey sent to 24 people and responded by 12 recipients. Sampling of the stakeholders was made using the following criteria: 1) major priority was given to those stakeholders prioritized or requested by USAID/CAM in the evaluation SOW; 2) the next level of priority was given to beneficiaries (direct and indirect) of activities´ programs, including representatives of the value chains intervened and small farmers and their organizations; and 3) third level included other stakeholders with interest in the activities´ results. Constraints of time, cost, distance were also considered to define final lists of informants. MAIN FINDINGS • Strengthening of national governments’ and regional entities’ capacity for analysis and policy formulation related to trade facilitation and food security was one of the major accomplishments of the activities, mostly through training and technical assistance. Interventions ranged from creating information systems, to training officials in food security and nutrition policy-making, to internal re￾organizations. Assistance provided to Central American Secretariat for Economic Integration (SIECA) was the most outstanding example. • RTMA’s grants program contributed to producer organizations capacity building in the administrative, accounting, and financial areas, as well as their decision-making capacities. Activities under the RTMA-supported producers did not bring greater access to regional and export markets. • RTMA’s contribution to changes in income consumption patterns and living standards of small farmers were not found. • There were no significant instances of collaboration among the four Strategy activities, nor they were mandated by it, or otherwise monitored or evaluated. • Activities did contribute to the dissemination and use of science, technology and innovation for agricultural practices among small farmers, mostly through training and technical assistance. PAPA on the food safety area, and RUTA on agricultural productivity, were the most successful in this regard. The creation of a “regional platform for research and innovation of sustainable agriculture practices” however, as envisioned by the Strategy, did not happen. • Due to IFC activity, the Regional System of Mutual Recognition of Sanitary Registries just started facilitating trade of food and beverages in Central America. There were several instances of activities contribution to the dissemination and use of science, technology and innovation for trade facilitation both regarding government procedures related to trade, and support to farmer organizations for greater market access. With the improvements to the physical and administrative processes for four border crossings in the region, indicators from the RTMA activity point to reduced times. Likewise, the process time for recognition of the registration of a food product in a country that has been previously registered in another Central American country, has fallen. iii • Gender integration activities or interventions were not included as part of the Strategy; however, by USAID/CAM’s request, implementing partners (IPs) counted female participation in training. • The ethnic makeup of targeted farming populations/organizations automatically favored social inclusion. • A relationship between the Strategy activities and migration could not be found. Opinions on the issue vary, with some interviewees attributing migration mostly to violence, and others saying that “international migration has declined because young people can stay with their families in coffee and horticulture areas.” However, no concrete statistics are available. • Relevance of the activities was much greater than the relevance of the Strategy. The Strategy was by far most relevant for USAID/CAM and regional institutions than for small farmers. • Regarding effectiveness, the ET did not find evidence that the top-level development objectives of the Strategy were achieved. The Strategy’s stated focus on food security and nutrition got lost on decisions about budget, programming, and activity decisions and implementation. At activity level, some of their interventions did meet their objectives, whereas the effectiveness of several key interventions (even activities) ended up being indeterminable due to lack of information. • The sustainability of the Strategy, as it is known, is very low although USAID/CAM, as well as bilateral Missions, are interested in the continuum of trade facilitation activities. Some interventions might remain beyond USAID’s support thanks to the interest of communities and other donors. Contribution of activities to market alliances that will last beyond USAID implementation is positive regarding formal written agreements made; however, their effect in actual sales (or in exports) remains to be seen. MAIN CONCLUSIONS • On institutional strengthening, the activities of the Strategy accomplished several relevant initiatives that brought new capacities to national institutions, such as national governments and universities. With respect to the creation of regional capacities for policy formulation on food security and trade, except for the well-recognized support to SIECA, not much progress was obtained. Strategy activities contributed very little to create or implement actual market alliances that increased producer organizations’ actual sales or exports. • The ET did not find evidence that RTMA has contributed in increasing producers’ access to regional markets or changing their income consumption patterns. • Collaboration among the four activities was a lost opportunity. • Activities performed remarkably well on dissemination of technologies on agriculture and trade facilitation but not that well on science and innovation. • The regional system for mutual recognition of sanitary registries has just begun working at the time of this evaluation and still needs to develop its full potential, including greater commitment by national governments. An average of the five countries shows that the time for mutual recognition has decreased in two days thanks to the system. The private sector, however, although interested, has found the system not to be particularly user-friendly. iv • None of the activities complied with Automated Directives System (ADS) 205 regarding the integration of the USAID Gender Equality and Female Empowerment Policy throughout the project cycle. RUTA, that showed measures to promote women´s participation, seemed to do it of their own accord and not as per any formulated strategy. • Social inclusion, rather unintendedly, was favored by the ethnic makeup of the target farmers groups. • The relevance of the Strategy was significantly greater for USAID/CAM and national governments, than for small farmers and poor rural households. • Determining the effectiveness of the Strategy and its activities is very difficult due to its lack of proper monitoring and evaluation (M&E) systems and practices. • Although the ET found that top-level USAID officials, both in DC and in the region, consider the Strategy a worthwhile mechanism that potentially gives direction, promotes collaboration among activities and stakeholders, maximizes resources, and should be greater than the sum of its parts, it did not find evidence of the Strategy playing that role, that the IPs had the Strategy or the other activities in mind, or that there was a continuous, decision-making oriented M&E of the activities to ensure they actively contributed to achieve the objectives of the Strategy. 1 1. INTRODUCTION 1.1. EVALUATION PURPOSE The Regional Integrated Trade and Food Security Strategy 2011-2017 (RITFS) is a USAID Central America and Mexico (CAM) initiative composed of four activities: 1. The Promoting Food Security and Trade Integration through Sanitary and Phytosanitary Standards and other Ag-Related Capacity Building (PAPA) 2. The Regional Sustainable Agriculture and Food Security Program (RUTA) 3. The Regional Trade and Market Alliances (RTMA) 4. The Central America Agribusiness Trade Logistics (IFC) This report will refer to the above activities by the name most people know them: PAPA, RUTA RTMA, and IFC. The purpose of the evaluation was to assess if the design and implementation of the four activities under RIFTS were effective in achieving its expected results and, based on that assessment, inform USAID/CAM on future programming in trade and food security. In addition, the evaluation was tasked to: 1) provide empirical evidence to support learning and continuous improvement for USAID/CAM’s RITFS; 2) identify the outcomes achieved by each activity toward their expected objectives, especially on such issues as capacity building and sustainability beyond USAID support; 3) provide information and recommendations surrounding the issue of how to address ongoing challenges to improve trade and reduce trade barriers in the region, and directly or indirectly impact food security; and 4) provide feedback from supported producer groups on the results of activity implementation on their income, consumption and nutrition. The evaluation covered implementation from the start of RITFS in September 2011 through September 30, 2016. It was conducted by a team of five: a team leader, two experts on trade issues, and two experts on agriculture. Findings, conclusions, and recommendations of the evaluation may be used by USAID/CAM to make decisions on new directions for the remaining period of the Strategy implementation and beyond, as well as to inform future programming priorities in trade and food security under the Development Objective (DO) 1 of the Regional Development Country Strategy (RDCS), which is “Regional economic integration increased.” 1.2. EVALUATION AUDIENCE The principal audience of this evaluation will be: USAID/CAM program, particularly the Economic Growth Office, the Regional Program Office, and the Mission Management; USAID/ Bureau for Food Security (BFS); USAID/Bureau for Latin America and the Caribbean (LAC); USAID/Guatemala and USAID/Honduras (Guatemala and Honduras are focus countries of the U.S. Government’s Global Hunger and Food Security Initiative “Feed the Future”); and other USAID offices that are expected to use the evaluation to provide a retrospective analysis of activity effectiveness to achieve the results individually and combined under the Strategy. 1.3. EVALUATION QUESTIONS The initial evaluation questions, as proposed by the SOW are: 2 1. To what extent have the activities individually and collectively led to the strengthening of local institutions?  To what extent did the activities strengthen national governments’ and regional entities’ capacity for analysis and policy formulation related to trade facilitation and food security?  To what extent did technical assistance provided to SIECA result in improved organizational capacity to support the Central American economic integration process?  To what extent did the grants program and technical assistance build the capacity of agriculture local partners and small producer organizations?  To what extent have activities contributed to market alliances and mechanisms that will likely last beyond USAID activity implementation? 2. How has collaboration between the four activities affected strategy and activity results? 3. To what extent have the activities contributed to the dissemination and use of science, technology and innovation for:  Sustainable agricultural practices among small farmers  Trade facilitation  A reduction in times and costs to cross borders 4. How did each activity promote gender integration and social inclusion? 5. Activity Specific Questions  IFC: How has the Regional System of Mutual Recognition of Sanitary Registries facilitated, or not, trade of food and beverages in Central America?  RUTA: How did RUTA’s activity change the capacity of communities of practice to perform as networks disseminating best practices?  RTMA: To what extent have the activities under the RTMA supported producers of selected agricultural value chains to improve access to regional and export markets? 6. How did the RTMA activity contribute to changes in income, consumption patterns and living standards of small farmers? Additionally, the following question was added, per request by USAID: 7. Have the activities had any impact on migration? 1.4. EVALUATION CRITERIA In accordance to the USAID Evaluation Policy (USAID, 2016), this evaluation adopted the following evaluation criteria:  Relevance, defined as “The extent to which the objectives of a development intervention are consistent with beneficiaries’ requirements, country needs, global priorities, and partners’ and donors’ policies.”  Effectiveness, defined as “The extent to which the development intervention’s objectives were achieved, or are expected to be achieved, taking into account their relative importance.”  Efficiency, defined as “A measure of how economic resources/inputs (funds, expertise, time, etc.) are converted to results.”  Sustainability, defined as “The continuation of benefits from a development intervention after major development assistance has been completed. The probability of continued long-term benefits. The resilience to risk of the net benefit flows over time.” 3 Furthermore, due its emphasis in USAID planning for activities, the evaluation added the cross-cutting issue of “Environmental Considerations.” 1.5. A NOTE ON VOCABULARY AND ACRONYMS This document will use the following key vocabulary: • “The Strategy” (capitalized) refers to the USAID/CAM RITFS. • “Activity,” per USAID usage, refers to each of the four projects designed and implemented under the Strategy. • “Component” refers to specialized areas of focus within an activity. • “Intervention” refers to each and all actions designed and implemented under each activity/component. These are the tasks to be implemented to achieve development objectives, which usually consist of any or several of the following: training, technical assistance, information use, technology use, acquisitions, or funding. 2. STRATEGY BACKGROUND The Strategy, approved on February 23, 2012, encompasses USAID/CAM’s program interventions in regional food security, trade, and market access for critical food security agriculture value chains; improves capacity for policy analysis and formulation among Central American governments; and fosters sustainable agriculture practices through the validation and dissemination of innovative technologies. The Strategy’s main goal is to help Central American countries reach a higher path of inclusive economic growth to eradicate hunger and poverty, and enable expansion of intraregional trade and exports. Its theory of change is that “if barriers to trade are reduced and more opportunities are provided to small farmers to gain greater access to regional and international markets, so as to increase their incomes, then regional food security and nutrition will improve.” The Strategy’s development hypothesis states that: “If Central America’s food security is supported through targeted value chain trade development, a regional framework of effective, efficient, and sustainable policies, and the dissemination of innovative sustainable agriculture best practices, then the region will have more inclusive economic growth to eradicate hunger and poverty, and expand intraregional trade.” The Strategy has two expected major results: • Consolidated regional value chains and export market access for food and nutrition critical products through increased trade facilitation, harmonization, and institutional capacity. • Increased effectiveness of regional food security policy and programs through the increased capacity of key decision makers in the government and private sector with the use of analytical tools, establishment of a monitoring and evaluation (M&E) system, and the promotion of best practices for sustainable agriculture. The Strategy is comprised of four activities: The Promoting Food Security and Trade Integration through Sanitary and Phytosanitary Standards and Other Ag-Related Capacity Building. The activity is a Participating Agency Program Agreement (PAPA) between USAID/CAM and the U.S. Department of Agriculture (USDA) Foreign Agricultural Service’s Office of Capacity Building and Development (FAS/OCBD). The activity will be implemented from May 4, 2011, to July 31, 2019, with a total estimated cost of $10,233,171. 4 The overarching goal of the activity is to increase incomes of the rural poor and small farmers through agriculture-led market opportunities and trade facilitation by strengthening the capacity of regional governments’ agricultural institutions and private sectors with targeted technical assistance, exchanges, and training activities. The scope of PAPA emphasizes technical assistance and training to: 1) build public and private sector sanitary and phytosanitary (SPS) capacity; 2) improve access to regional and international markets, including market information; 3) improve statistical systems to better inform policy makers on matters of food security; and 4) address other agricultural products under the Dominican Republic-Central America-Unites States Free Trade Agreement (CAFTA-DR). Technical assistance provided to individual cooperating countries under PAPA has addressed country￾specific technical issues, which were identified as key barriers to trade, economic growth, or food security between USDA and USAID. Assistance is provided through Government-to-Government training and technical advice programs, and includes private industry and smallholder producer participation, as appropriate. This assistance looks to strengthen plant health, animal health, and food safety inspections services, as well as assist the respective country governments and the private sector in improving quality and productivity to meet international standards. Regional efforts aim at improving the enabling environment for regional trade. The Regional Sustainable Agriculture and Food Security Program is a Public International Organization grant to the United Nations Office for Project Services (UNOPS), implemented through the technical execution of the CAC’s Regional Unit for Technical Assistance (RUTA from September 14, 2012, to March 31, 2017 (original end date was September 13, 2017), with a total estimated cost of $5,000,000. The purpose of this program was to promote inclusive economic growth, improve food security, and facilitate trade in the region. Its first component focused on developing the capacity of the Central American countries’ national governments to support analysis, policy formulation, monitoring, evaluation, and learning. This component was later modified to focus solely on the organization of special training courses on food security policy in El Salvador, Honduras and Guatemala for public and private sectors and civil society. The second component sought to promote sustainable agricultural technologies in Central America and the Dominican Republic, and develop a regional platform for research and innovation of sustainable agricultural practices. The expected results of this component were: 1) at least six Memoranda of Understanding (MOUs) signed by CAC/RUTA/UNOPS with regional agriculture research centers for identification, validation, and dissemination of sustainable agriculture practices relevant to food security and applicable to small farmers; and 2) at least six new sustainable agriculture technologies adapted for small farmers, and disseminated through government extension services and relevant food security small farmers’ technical assistance programs. The Regional Trade and Market Alliances (RTMA) activity is a contract implemented by Nathan Associates, Inc. The initial base period was from April 22, 2013, to April 21, 2016; the contract was extended to April 21, 2018, with a total estimated cost of $18,954,951. The purpose of this activity is to promote inclusive economic growth, improve food security, and facilitate trade in the region. The activity has two major components: 1) market access for critical food security value chains; and 2) increased trade facilitation, and improved trade and institutional capacity. The first component supports critical agriculture value chains for food security, linking small and medium producers to regional and export markets (to and from Central America, and also to other export markets). The initial expected results of this component were: 1) at least 15 percent increase in exports sales value of participating producer groups and buyers; 2) at least 1,500 small producers units (up to 1.05 hectares associated in small producer groups, associations or cooperatives) participating in regional and export value 5 chains; 3) at least 3 MOUs with regional or national institutions signed to support implementation of regional value chain development; 4) at least 4 special regional trade mechanisms implemented; and 5) at least 20 regional buyer-seller alliances established to link small producers and small enterprises through a value-chain development approach with at least five resulting in sustainable market driven contract relationships. The second component focuses on reducing obstacles and costs of trade in the region, as well as on providing continued support for the Central American Economic Integration Subsystem to improve trade facilitation, harmonization of trade and investment laws, and regulations and procedures. The expected results of this component are: 1) the average time to trade goods across borders in the region reduced to two days; 2) trade facilitation monitoring system implemented and adopted by a regional private sector organization or national organizations; 3) a trade and customs specialist certification program implemented and adopted through partnership(s) with academic center(s); 4) one regional or five national MOUs signed with private sector organizations to promote the Central American Trade Facilitation Agenda; and 5) at least four policies, regulations and administrative procedures consulted among stakeholders and presented to the Central American Economic Integration Ministers’ Council (COMIECO). The Central America Agribusiness Trade Logistics activity was under the global USAID - IFC grant agreement to be implemented from September 30, 2013, to April 30, 2016 (original end date was December 30, 2016), with a total estimated cost of $1,500,000. The activity´s objective was to streamline, harmonize, and automate procedures for mutual recognition of sanitary registration for processed food and beverages in Costa Rica, El Salvador, Guatemala and Honduras. The activity has three components: 1) regulatory reforms for process redesigning and harmonization; 2) technological reforms in Information, Communication, and Technology (ICT) systematization; and 3) capacity building, which includes training for public sector officials and private sector users on new procedures/systems, as well as their implementation in four countries. Expected results at the end of the activity were: 1) average number of days to register a processed food product for the first time reduced from 80 days to 55 days; 2) average number of days to register a processed food product for mutual recognition reduced from 8 days to 1 day; 3) average cost to register a processed food product for the first time reduced from $400 to $300; 4) average cost to register a processed food product for mutual recognition reduced from $145 to $120; 5) number of documents required to trade for mutual recognition registration reduced from 4 to 2; and 6) number of product registries mutually recognized increased from 4,633 to 5,096. 3. EVALUATION METHODS AND LIMITATIONS 3.1. METHODOLOGY A more complete description of the evaluation´s methodology can be found in Annex 3. The evaluation adopted a non-experimental design. It focused on the performance of the activities, and whether they have made any difference, based on before and after comparisons. As a performance evaluation, it was designed to analyze the activities to identify results and accomplishments; recognize performance issues and constraints in their implementation; and determine how the activities are perceived and valued. The purpose was to present relevant findings, make conclusions and lessons to be learned, and present recommendations for informing future USAID programming on trade and food security. 6 The evaluation utilized a mixed-methods design, combining a comprehensive analysis of existing quantitative data with qualitative techniques designed to capture data from a wide range of Strategy and activity stakeholders. This approach allowed for triangulation of complementary data to determine linkages between activity inputs, outputs, and outcomes. As explained later, the evaluation considered a variety of methods for collecting and analyzing the data to address the evaluation questions thoroughly. 3.2. EVALUATION ACTIVITIES The evaluation was conducted in four successive stages: • Review and analysis of secondary information and activity-related information (January 9-20, 2017) • Primary information data capture (field work) - El Salvador: January 23-February 3, 2017 - Honduras: February 6-17, 2017 - Washington DC and Costa Rica: January 30-February 3, 2017 - Guatemala: February 20-March 3, 2017 • Data analysis (March 6-11, 2017) • Evaluation reporting (March 12-31, 2017) A more detailed work calendar is presented in Annex 5. The review and analysis of secondary information and activity-related information included the review of 189 documents (Annex 9) obtained through requests to USAID/CAM, the activities’ Contracting/Agreement Officer Representatives (COR/AOR) in USAID, their Chiefs of Party (COPs), and through internet searches. The stage of primary data capture, or field work, included key informant interviews (KIIs), focus group discussions (FGDs), site visits, and one online survey (see Figure 1 for the geographic scope of field work). Figure 1: Geographic Scope of Evaluation Activities To select the key informants that would be interviewed, participate in FGDs, or be surveyed online, the evaluation adopted the following criteria: • Major priority was given to those stakeholders prioritized by USAID in the evaluation SOW. • The next level of priority was given to beneficiaries (direct and indirect) of activities´ programs. 7 • Next were other stakeholders with interest in the activities´ results. Constraints of time, cost, and distance were also considered to determine the final lists of informants. The evaluation conducted 127 KIIs with 192 key informants (who are listed in Annex 6). KIIs utilized the key evaluation questions and sub-questions, which are included in the evaluation matrices for each activity (see Annex 4). A KII protocol is presented in Annex 8. The ET convened 9 focus group discussions related to RUTA or RTMA activities in 3 countries, with 88 participants, including 33 women. Focus groups’ participants included: 1) those specifically requested in the evaluation SOW as in the case of RTMA; 2) representatives of the product or value chain; and 3) small farmers and their organizations. Geographic proximity and logistical convenience was also a factor in selecting FGD participants. FGDs conducted included:  RUTA - National Coffee Working Group, Santa Tecla, El Salvador  RTMA - Plantain packing and dried fruit processing, San Carlos Cooperative Association (ACOPASCA), El Paisnal, Aguilares, El Salvador (SOW requirement)  RUTA - Cocoa producers assisted by FundaSistemas, Cobán, Guatemala  RUTA - Cocoa producers and processors, assisted by Fundación Pro-Petén, Guatemala  RTMA - Honey producers and processors, Integral Commercialization Apiculture Cooperative Atitlán R.L, San Pedro La Laguna, Guatemala  RUTA - Women's Network Producing Cacao and Chocolate in Honduras (REDMUCH), in La Lima, Honduras  RUTA - Food security policy course participants, Universidad Nacional de Honduras (UNAH), Tegucigalpa, Honduras  RTMA - Potato packing, Intibuca Farmer Agroindustrial Company of the Agrarian Reform ECARAI La Esperanza, Intibucá, Honduras  RTMA - Bean packing, Regional Association of Livestock and Agriculture Services of the East (ARSAGRO) Danlí, Honduras (SOW requirement)  RTMA - Lepaterique Horticultural Company (HORTISA), Lepaterique, Honduras The general protocol for FGDs is presented in Annex 8, along with an example of the protocol for the discussion held at El Paisnal with members of ACOPASCA. Annex 10 presents the report for all three of the RTMA focus groups required by the SOW. RTMA FGDs and RUTA FGD in ProPetén were complemented by site visits to the packing and/or processing facilities of the respective organizations. The ET made three additional site visits to gain first￾hand knowledge of the physical context of producers and their organizations involved with the activities, for a total of eight site visits. The additional site visits were: • RUTA - Ecological Fraternity Coffee-farming Cooperative (CAFEL), Copan Ruinas, Honduras • RUTA – Cooperative Association of Cocoa Agricultural Production (ACPACI), Los Izalcos, El Salvador • RTMA - Border crossing facilities at El Amatillo, between El Salvador and Honduras The ET conducted one online perception survey for high-level implementing stakeholders who had direct knowledge of or involvement in the Strategy, including USAID CORs, COPs of the activities, and other people involved with the design and/or implementation of the Strategy and its activities. The purpose of the survey was to obtain their assessment of the Strategy under the evaluation criteria adopted by this evaluation. The survey had 12 respondents, out of 24 recipients. 8 3.3. EVALUATION LIMITATIONS During the evaluation, the ET encountered the following limitations:  The Strategy was designed as regional, covering all Central America, whereas the evaluation covered only El Salvador, Guatemala, and Honduras. As the evaluation is focused in three countries, findings could be deemed not as comprehensive as expected. The evaluation addressed this potential issue by including stakeholders in regional institutions, and involving USAID/CAM and Washington DC-based USAID and U.S. Department of Agriculture (USDA) staff.  Several activities or components of activities concluded before the evaluation began. RUTA is in the process of closing, but its evaluation was actively supported by its AOR and COP, so that the ET had sufficient opportunities for interviews and focus groups. In the case of RTMA’s value chain component, the FGDs requested by the SOW have provided a complementary perspective.  Major staff changes have occurred at USAID offices, and among strategy implementing partners (IPs), which reduced the availability of some primary information. The ET asked IPs for help in contacting former staff members, including the former AOR/COR for three out of the four activities. In the case of RTMA, the former COP was also interviewed.  Security, distance, and availability limited some interviews, focus groups or site visits. In these cases, a “second best” option was selected that could be visited in a reasonable amount of time and cost or with greater security.  Significant delays or refusal to provide key documentation regarding RTMA, as well as lack of M&E data of the IFC activity, limited the ability to reach conclusions about the effectiveness and efficiency of this activity.  A lack of mid-term evaluations for the activities and the Strategy, and generally weak M&E systems in activities as well as the absence of one for the Strategy, created difficulties in obtaining information on the progress of activities´ outputs and outcomes, and whether corrective actions had been taken if targets were not being met. 4. EVALUATION FINDINGS 4.1. FINDINGS ON THE EVALUATION QUESTIONS 4.1.1. To what extent did the activities strengthen national governments’ and regional entities’ capacity for analysis and policy formulation related to trade facilitation and food security? The Strategy and its activities include a series of actions to strengthen national and regional public institutions. The PAPA activity undertook several interventions to improve market information systems as well as animal and plant health systems. From interviews with representatives of the Ministries of Agriculture, Health, and Economy, the ET obtained the following examples: • In El Salvador and Honduras, improvements have been made to the platforms for market data through activity support, including consultants for software development. The activity also brought a USDA consultant to teach USDA Agricultural Marketing Service methodology for price reporting. Based on this training, all three countries have modified their methodology for market price monitoring (using the mode instead of the mean), and market reporters were trained 9 • on price collection techniques and methods. With the Market Information Organization of the Americas, sponsored by USDA, and assistance from specialists from Texas Tech University, a Wiki site was created to document common characteristics of different fruits and vegetables and grains in each of the Central American markets. • With PAPA’s assistance, Ministries of Agriculture have newly formed government divisions for pest risk analysis and the formalization of a quarantine pest list in Guatemala, for pest monitoring and vigilance in Honduras, as well as for animal health monitoring and for food safety in El Salvador. According to interviewees, several of these functions were non-existent before PAPA, or were assigned to unqualified officers who did what they could, took a very long time, or did nothing at all. • Over 1,400 government officials have received training activities on such topics as integrated pest management, pest risk analysis, good agricultural practices, good manufacturing practices, post￾harvest management, traceability, and other requirements by the U.S. Food Safety Modernization Act. RUTA activity organized food security and nutrition policy courses in the three countries evaluated, in coordination with universities in each country, FAO in El Salvador, and the International Food Policy Research Institute (IFPRI). A total of 88 people, mostly government officials, received the specialized training. According to interviews with training participants, they were highly satisfied with the course. The RTMA activity provided technical support and funding for the organizational modernization of the Central American Secretariat for Economic Integration (SIECA), and the creation of new or improved systems, including the Trade Facilitation Monitoring System. These actions are described in more detail in Section 4.1.2 of this report. In the IFC activity, SIECA has agreed to absorb the regional platform for sanitary registries created by the activity, and has received appropriate equipment and training for this responsibility. The Ministries of Health of the three countries, which are tasked with the sanitary registry for food products, have attained differing levels of institutional improvements. El Salvador´s already digitized system was made compatible with the regional platform. Honduras still has a paper-based registry but will likely have it digitized by end of 2017. Guatemala, despite being provided with a digitized national sanitary registry, continues its data entry into the regional platform from an Excel spreadsheet. Whether it was PAPA, RUTA, RTMA or IFC, all the four activities in different degrees found difficulties, in the three countries, to work with the governments on their institutional strengthening. They were difficulties derived of the way those national governments work: high bureaucracy turnover, lack of follow up, clientelism, cronyism, ever changing policy priorities, lack of budget for public investment, weak/inexistent communication among government offices, even corruption charges that brought high level official to prison. "You always think that you are doing things right, but it is fundamental to know other points of view and experiences to rethink things…In addition, this year we were all starting a new administration (government) and the diploma arrived at the most appropriate time. The RUTA diploma served to foster greater political will and an integral approach." – Representative of SESAN, the Food Security and Nutrition Secretary, Government of Guatemala. Gisela Vergara, General Secretary of SIECA states: “RTMA was a fantastic, excellent support. For the Secretary, there was a before and after. With RTMA we made changes in the structure of the Secretary by which we got the ISO [International Standard Organization] certification. Project staff seemed as if they were SIECA personnel because they worked very close to us. USAID is the only donor who has worked on our institutional strengthening its assistance has had much impact on us”. 10 Within such context, some shortcomings were noted. First, the first component of RUTA in its original form was to support developing the capacity of the Central American countries’ national governments to support analysis and policy formulation for food and nutritional security public policy agendas and instruments that have greater impact on food availability and food prices. However, due to a lack of receptiveness by national governments to policy changes proposed, such as a Policy Analysis Matrix Analysis Platform, and the activity´s lack of successful engagement with CAC, this component was greatly modified to concentrate on the food policy course offered to a broader group of participants. And for PAPA, scarce results can be found in regard to production statistics, which was of equal importance with market information systems in its design, with merely the provision of USDA National Agricultural Statistical Service experts in 2012 who participated in training on production statistics in the region, but with no mention of resulting changes in Ministries of Agriculture´s activities in this area. 4.1.2. To what extent did technical assistance provided to SIECA resulted in improved organizational capacity to support the Central American economic integration process? SIECA is the technical and administrative entity of the Central American economic integration process for Costa Rica, El Salvador, Guatemala, Honduras, Nicaragua, and Panama, members of the Central American Integration System (SICA). SIECA is headed by a General Secretary named by COMIECO for a period of four years. Through MOUs, Nathan Associates and SIECA agreed on several initiatives carried out between 2013 and 2016 to work on institutional capacity building of SIECA. The agreements comprised a wide range of institutional strengthening, capacity building, and modernization initiatives such as updating the internal regulations and operations manuals; improving its strategic planning; increase M&E capacities, including the adoption of the Balanced Score Card; implementing an information platform on technical trade regulations, regional trade institutions, and technical barriers to trade; and improving training capacities through e-learning. Other activities were more oriented towards widening SIECA’s relevance for the economic integration process, such as workshops on the World Trade Organization agreements for trade facilitation; studying the feasibility of an external services unit; the establishment of a training program on leadership in trade facilitation; development of a regional economic intelligence observatory, as well as several studies on matters such as the incorporation of Belize into the Central America integration system; and an analysis of the status of the Central American integration process and obstacles towards a customs union. Williams García, SIECA’s Director of Integration, states that “in trade facilitation, the RTMA project did identify priority areas based on its demonstrated knowledge of the needs of customs and trade ministries. It helped with studies, technical recommendations, as well as producing information previously unavailable, and proposals for implementation in border crossing stations.” In conversations with stakeholders in El Salvador, Guatemala, and Honduras, the ET found evidence that the technical assistance provided to SIECA for institutional strengthening did improve its capacity to support the Central American integration process. The general opinion is that SIECA has moved from invisibility to relevance, leadership, and a heightened technical profile based, to great extent, on the production, use, and dissemination of trade facilitation-related information. Moreover, in the context of the Strategy, other trade facilitation-related information systems will be transferred to SIECA. They include: 1) the Trade Facilitation Monitoring System (RFID) also created by RTMA; and 2) the Regional Integrated System for Sanitary Registries being created under a USAID-IFC collaboration agreement. These transfers demonstrate that other institutions within the Central American 11 integration institutional framework trust SIECA’s capacity to continue working on regional integration for years to come. 4.1.3. To what extent did the grants program and technical assistance build the capacity of agriculture local partners and small producer organizations? The ET was not provided with thorough, complete, and relevant documented information (neither organizational, technical, or financial) for each of the grants. Consequently, the ET resorted to interviews, focus groups, and site visits of a few producers’ organizations in El Salvador, Guatemala, and Honduras. The structure of RTMA beneficiary organizations consisted of: 1) local technical partners; 2) producers’ organizations; and 3) umbrella organizations. The local technical partners were: in El Salvador, Salvadorian Foundation for Social and Economic Development (FUSADES); in Guatemala, Guatemalan Exporters Association (AGEXPORT); and in Honduras, Foundation for Rural Development (FUNDER). Additional project partners were a group of producer organizations working directly with the program that were not large or developed enough to need assistance in the project dollar values range established by RTMA, $250,000 - $400,000, but had considerable institutional capacities. Those organizations were Cooperative Association La Rafaleña, Integral Farming Cooperative ATESCATEL, Organizations Association of Los Cuchumatanes, and Cooperation Studies Association in Guatemala (ECO); ACOPASCA in El Salvador; and ECARAI, ARSAGRO, Regional Association of Livestock and Agriculture Services of the East, and Global Village Project in Honduras. Finally, the umbrella organizations, working with smaller producer organizations, included: In Guatemala: Integral Development Association for the Development of Buenos Aires-ADIBA and Atitlán Cooperative, which worked with AGEXPORT; Farmer’s Association for the Integral Development of Tejutla (ASADIT), Tiuchanenses, and Farmer’s Association for the Integral Development of Chiquirichapa (ADECHI) working with ECO; Trading Company Meseta de los Cuchumatanes COMECSA working with Organizations Association of Los Cuchumatanes (ASOCUCH); and Integral Production Cooperative of Apiculturists of the South of Guatemala (COPIASURO), Santa Maria Cooperative, and Ixcan Cooperatives Commercialization Integral Federation (FICCI) working with the Tropical Agricultural Research and Higher Education Center CATIE. In El Salvador: Apis1 Morazán, Apis Las Cabañas and Los Izalcos working with CATIE. In Honduras: HORTISA and Vegetable Producers of Izopo and Azacualpa (PROVIASA) worked with the Consortium (a FUNDER sister organization), Producers Association of Celaque -APROCEL worked with Global Village Project (PAG), and Cocoa Farmers´ Association of Honduras (APROCACAHO) with CATIE. It was intended that CATIE be a regional umbrella organization but, according to RTMA staff members, that did not work out and so RTMA had to take over its responsibilities. The local technical partners/umbrella organizations are well-known, reputable organizations who benefitted from RTMA mostly by obtaining additional resources to do the work they usually do, renewing their ties with producer organizations they already had a relationship with, and practicing their technical skills. On the other hand, these partners complemented RTMA’s work by providing producers with technical assistance and training in agricultural practices related to but not included in the grant’s funded activities. 1 Apis: short for Apiculture, referring to Apiculture Cooperatives. 12 Producer organizations, besides the grant-supported investment in manufacturing infrastructure, were also given technical assistance and training on administrative, organizational, accounting and financial decision￾making subjects. An outstanding part of this assistance was providing these organizations with SIFER, an accounting tool designed by RTMA, which is not only still being used by most of them, but is also praised by the staff and the producers themselves as easy to understand, useful, and meaningful for them, particularly to make decisions. Results of the grants program are a mixed bag. As per this evaluation’s site visits and focus groups, it can be said that the investment projects were implemented as planned. However, the machinery and equipment acquired are in a fledgling stage in most cases. The implementation level can be characterized as follows: 1) projects with 100 percent of investment made as planned, with producers optimistic and satisfied, with machinery/equipment working, and with the organization having new market experiences, such as in the case of ECARAI, a potato producer in Honduras; 2) projects with 100 percent of investment made as planned, with producers optimistic, but with machinery/equipment still not working, and with the organization still having barriers to local sales and exports as it is the case of ACOPASCA, a plantain and dried fruit producer in El Salvador; and 3) projects with 100 percent of investment made, supposedly as planned, with hopeful but mostly frustrated producers, with machinery/equipment installed but not able to work, or where equipment, almost a year later, still does not have electric power service, and a ThermoKing purchased by the project ended up being useless and has been discarded, as is the case of HORTISA, a vegetable producer, in Honduras. Finally, despite the stage of each project, the grant-supported investments in industrial infrastructure have brought manufacturing experience and capacities to farmer organizations previously devoted only to agriculture, which is expected to increase the value added and/or quality of their products. 4.1.4. To what extent have activities contributed to market alliances and mechanisms that will likely last beyond USAID activity implementation? RTMA’s work on trade alliances basically consisted of facilitating producers’ participation in trade events such as fairs, business roundtables, and producer-buyer networking opportunities. Some of those producer￾buyer relationships were pre-existing. However, the ET found no evidence of export-oriented market alliances crafted by RTMA or with its assistance that did lead to actual exports whether to neighboring countries or the U.S. market. The alliances that could be verified by the ET consisted mostly of a document signed by producers and local buyers such as supermarkets, more in the fashion of an MOU that an actual commercial contract. These letters, basically, corroborated relationships with current or former customers, with expectations of obtaining products of better quality or in larger volumes due to the investments made by the grants. During field work, the evaluation found that some producer organizations, such as ECARAI in Honduras, did recognize RTMA’s support in looking for export opportunities, although at the time of the evaluation’s visit, ECARAI had not been able to export. Nathan Associates recognized that only one third of the grant funds went to organizations that did export, although the ET had no means to establish whether these exports can be directly attributed to RTMA. Moreover, CAFTA-based opportunities for Central American products to access the U.S. market were missed by RTMA activities. CAFTA-DR provides for multilateralism among members and accumulation of origin among Central American countries for export to the U.S. under quotas. In other words, food exporters, under a quota system, can accumulate origin by using inputs originating from another Central 13 American country. This interpretation of CAFTA-DR regulations offers a clear possibility for establishing value chains for food products to be exported under tariff rate quotas. However, some of the agriculture import quotas issued by the U.S. are being underutilized by Central American countries. This implies that Central American countries export less to the U.S. than what the quota allows. Besides RTMA and its market alliances, there were other activities, such as RUTA, which by giving agricultural assistance, training in best practices, and supporting communities of practice of cacao producers, opened doors for piquing the interest of buyers, who also supported these farmers so that a better￾quality product could be offered to international markets. That is the case of Cacao Verapaz in Guatemala, an exporter that has invested its own resources in technical assistance to producers after RUTA activities ended. This type of market-driven support is most likely to remain beyond USAID activities. PAPA activities, rather than market alliances, have created market opportunities (or, better said, reduced lost market opportunities) by dramatically reducing the volume of agricultural exports from Guatemala and Honduras detained at U.S. ports due to pesticide-related violations. This structural change on pest management practices due to PAPA, has led to market opportunities likely to last beyond USAID activity implementation. 4.1.5. How has collaboration between the four activities affected Strategy and activity results? Whereas the Strategy, in several instances, mandated or encouraged collaboration between its implementers and their stakeholders, and with the USAID bilateral programs, it did not foresee, mandate, or structure ways of practical collaboration among the four implementers (except for PAPA). Coordination between the four Strategy activities typically occurred by chance, when technicians in the field from each activity found they were working with the same groups, but generally not by design. The Economic Growth Office in CAM began an annual partners meeting, where each made a brief presentation of their activities, intended to foster collaboration. However, there is no sharing of work plans or discussion during their preparation among COPs or CORs/AORs. One USAID representative suggested that “Institutional jealousy…typically trumps efforts for collaboration.” Further, the COP for a Feed the Future (FTF) project admitted that each project has their own contract that they must fulfill, and these typically do not mandate collaboration with other projects. The only example found of collaboration between PAPA and the other Strategy activities is with RTMA, for potato producers in Guatemala who had received RTMA support for potato cleaning equipment, where a study on inspection requirements for export was contracted. Collaboration with RUTA or IFC was not found. Since there were no significant instances of collaboration among the four Strategy activities, nor they were mandated or monitored, it would be mere speculation to say how collaboration (or lack thereof) affected them or the Strategy as a whole. In contrast, the Strategy did mandate USAID/CAM and implementers to collaborate with Central America´s FTF bilateral programs (Guatemala and Honduras) “to promote cost effectiveness, experience and knowledge sharing.”2 Therefore, the ET asked those USAID Missions about their collaboration with CAM. They see themselves stronger than CAM regarding food security and nutrition (El Salvador is not a FTF focus country) than on trade facilitation. They see CAM as a welcomed vehicle to deal with regional 2 USAID / CAM. RITFS page 17 14 organizations, such as SIECA, rather than each country working individually. The Honduras Mission even has named a person to work, among other things, as a liaison with CAM, neighboring USAID Missions and regional endeavors. Both Missions see CAM as a great complement to their work, especially in the area of trade, as well as in the regional public policy arena. Otherwise, collaboration between the bilateral programs and Strategy implementers is by far stronger, with PAPA due to the high relationship between food safety, which PAPA works very much on, and food security, which is the core of FTF. Relation between RUTA and RTMA was mostly coincidental as a few times agriculture technicians of each activity happened to have worked with one common project. Collaboration was inexistent with IFC. 4.1.6. To what extent have the activities contributed to the dissemination and use of science, technology and innovation for sustainable agricultural practices among small farmers? The Strategy detailed that RUTA activity would “consolidate a regional platform for research and innovation of sustainable agriculture practices to support dissemination for farm level application…This activity will coordinate with regional centers of excellence for agriculture research such as CIAT, CATIE, Zamorano, FHIA, government agriculture research and development (R&D) units, national universities, producers´ associations, other non-governmental organizations (NGO) programs. Solid breakthrough proposals will be validated through implementation of pilot projects with selected small producers´ groups” 3 Through training and technical assistance, two activities engaged in actions that contributed to the dissemination of technology and innovation for sustainable agriculture to small farmers. First, RUTA provided grants that included funding for training for cocoa and coffee farmers in sustainable agriculture in participant organizations in the Northern Triangle. Activity reports indicate that a total of 3,002 persons (68 percent over the initial target) received training on agricultural productivity, and 549 on environmentally friendly practices (2 percent under the target). In both cases, the participation of women was under the target number, by 24 percent and 13 percent, respectively. The training model utilized “community facilitators,” who are farmers that received special training to then act as local change agents and engage with other farmers. The RUTA activity also created and/or supported various communities of practice, such as for coffee in the Trifinio region of the Northern Triangle, and for cocoa in each of the countries evaluated, providing funding for training materials and activities in sustainable agriculture. These communities of practices are described in more detail in Section 4.5.2 of this report. As to the “regional platform for research and innovation of sustainable agriculture practices,” there is no evidence that such a platform was created. Engagement with Centers of Excellence as suggested by the Strategy included the provision of genetic material for cocoa from the Honduran Foundation for Agricultural Research (FHIA), initial support from the Inter American Institute for Agricultural Cooperation (IICA) and CATIE in the design of coffee training events, and providing funds for a consultant to the Regional Coffee Improvement Project (PROMECAFE) for the design of a regional coffee rust strategy. The PAPA activity provided training for integrated pest management for snow peas, garlic, potatoes, tomato, and carrots for FTF farmers and their technicians in Honduras and Guatemala. The training was 3 USAID / CAM RITFS page 24 15 designed with support from experts from institutions such as Penn State University and Ohio State University. By reducing pesticide by using integrated pest management (IPM), net income for snow pea farmers in Guatemala increased by 36 percent (USAID/USDA PAPA, 2012). Similarly, according to the COP, the number of containers of snow peas from Guatemala detained for inspection in the U.S. for pesticide residues has declined from over 1,800 in 2012 to approximately 100 in 2016, ultimately reducing export costs. 4.1.7. To what extent have the activities contributed to the dissemination and use of science, technology and innovation for trade facilitation? The Strategy proposed that “the program will promote the use of adapted information technology tools to streamline trade related administrative processes” (USAID, 2011). A series of examples of the use of information systems technology and administrative tools to facilitate trade can be found in the work of the activities. First, with regards to facilitating government procedures related to trade:  RTMA introduced the regional trade facilitation monitoring system, which will be administered in the future by SIECA, to provide real time information on times for administrative and inspection procedures at major border crossings.  The IFC activity is concentrated on the development of the regional platform for the mutual recognition of sanitary registrations, a system which will also be administered by SIECA in the future.  The PAPA activity provided technical assistance for the formalization of an official quarantine pest list in Guatemala and, soon, in Honduras. The pest list allows for science based determinations for admissibility for imports, and provides a more solid foundation when these countries seek admissibility for exports. This advance has, for example, permitted a reduction in the time for analyses for admissibility in Guatemala for fresh product imports from the U.S. and other countries. The other side of trade facilitation has been support to farmers and exporters, for greater market access. For example:  In the RUTA activity, the grant funding FundaSistemas´ work with cocoa producers in the Coban region, Guatemala, provided new techniques for cultivation, harvest, postharvest management, and processing. Cocoa quality has improved, which has opened new market opportunities through the Cacao Verapaz social enterprise, a growing cocoa exporter.  As detailed in Sections 4.1.3 and 4.1.4 of this report, the RTMA activity provided grants to farmers´ associations and cooperatives for the purchase of equipment for processing and/or packaging, leading to greater quality and value added. Of these groups, 1/3 were engaged in exporting and, in some cases, the equipment provided allowed them to fulfill market expectations or sanitary requirements to be able to begin exporting.  The Trilateral Cooperation Agreements with Chile for Honduras and Guatemala, through the PAPA activity, led to improvements in the pre-certification programs for some fruit and vegetable exports to the U.S. The cooperation facilitated technical assistance from Chilean experts and site visits to observe these processes in Chile. Pre-certification programs can reduce border processing times and costs when exports arrive in the U.S.  PAPA also organized a series of training events for government officials and exporters in Central America on new food safety requirements in the U.S., as legislated by the Food Safety Modernization Act (FSMA). The activity also contributed to the development of pre-audit tools, which better help exporters identify deficiencies in their facilities or procedures before an official inspection. Both actions were designed to help these exporters maintain access to the U.S. market. 16 4.1.8. To what extent have the activities contributed to the dissemination and use of science, technology and innovation for a reduction in times and costs to cross borders? With the improvements to the physical and administrative processes for four border crossings in the region, indicators from the RTMA activity point to reduced times. For example, according to reports from the activity, the wait time to arrive at the border crossing between Pedro Alvarado, Guatemala, and La Hachadura, El Salvador, for shipments coming from Guatemala, has fallen from an average of 8 hours and 14 minutes to 5 hours and 15 minutes, a reduction of 36 percent. Similarly, for shipments from El Salvador, the wait time to arrive at the border has fallen from 2 hours 29 minutes to 1 hour 39 minutes, a reduction of 34 percent. At the Salvadoran customs office, once the trucks get to the front of the line at the border facility, the administrative processing time has fallen 45 percent for imports and 29 percent for exports. The ET did not find any calculations or estimates of the reduction in transportation costs due to these reductions in shipping times. This would depend in part on market factors and the structure of the transportation sector, as to how much of this potential cost savings is passed along to export companies. With the Regional Digital Platform for Mutual Recognition of Sanitary Registries, the process time for recognition of the registration of a food product in a country that has been previously registered in another Central American country, has fallen. Reports from the activity show for the second half of Calendar Year 2016, an estimated average time for the mutual recognition process falling from six days to an average of four days. The current times in Nicaragua and Honduras are one day; in Guatemala, three days; in Costa Rica, five days; and in El Salvador, eight days. The cost for this process has not fallen in any of the countries since the implementation of the platform. 4.1.9. How did each activity promote gender integration and social inclusion? Gender Upon reviewing the Strategy, the ET found the word “gender” twice4 regarding its importance for development but nowhere in the design, programming, or budgeting of its activities. The Strategy’s IPs, however, indicated that USAID/CAM, during implementation, had insisted in counting women participation in activities, especially training (which is not necessarily gender-centered training). One interesting exception is RUTA which, even though was not given a proper gender strategy (according to ADS 205, USAID Gender Equality and Female Empowerment Policy) by the RITFS, from the very beginning was oriented to gender equity by, for instance, anticipating that: “Program activities will be carried out with a focus on inclusion, equity and gender equality, valuing especially the role of women, their participation in rural economic, social and environmental activities, and in policies and their instruments.” Throughout its implementation, RUTA achieved an average 41.6 percent female participation in the course on Public Policies for Food Security and Nutrition its versions in El Salvador, Honduras, and Guatemala. Likewise, RUTA provided technical assistance and training to a female producer organization, REDMUCH. Even though the RUTA activity did not meet its 40 percent goal of women participants (224 planned participants vs. 194 actual participants), it managed to incorporate gender equity program elements found in no other Strategy activity. Focus groups this evaluation hosted for purposes other than gender analysis, showed examples of real female leadership in Cooperatives where women held key positions in management, or in the Board of Directors, or had a leading vision for the future of the organization. Moreover, such meetings gave evidence 4 USAID RITFS page 18 17 of women’s indispensable role in the adoption of better agricultural practices. These considerations underscore the great potential of a gender perspective as part of any value chain and food security strategies. Social Inclusion The ethnic makeup of targeted farming populations/organizations automatically favored social inclusion. This is particularly true regarding indigenous people in Honduras and Guatemala, where many Mayan ethnicities exist. For example, one of the FGDs of this evaluation was conducted mostly in the Q'eqchi' tongue with cacao producers from various communities in the Guatemalan department of Verapaz; another focus group in San Pedro La Laguna, in Lago Atitlán, department of Sololá, Guatemala, gathered people of the Tz'utujil Maya community, in a village with over 90 percent indigenous people who speak Kaqchiquel. Likewise, most of the producers belonging to ECARAI, a producer organization in La Esperanza, department of Intibucá, Honduras, belong to the Lenca ethnicity. These are a few examples of how Strategy activities, however unintendedly, did promote social inclusion. 4.1.10. (IFC) How has the Regional System of Mutual Recognition of Sanitary Registries facilitated, or not, trade of food and beverages in Central America? The system has begun facilitating trade of food and beverages in February 2017. The Sanitary Registries Regional Integration System https://www.sirrs.sieca.int/ is a digital platform developed by EVERIS, an ICT company, under a contract between IFC and SIECA which, in turn, is charged with administering it with its own staff. The digital platform was launched in June 2016. At the time that this evaluation interviewed SIECA staff, February 2017, it was working at full capacity and showing the following data:  1,347 requests for mutual recognition made.  Use of the platform is led by El Salvador, since the country already had a digital system for sanitary registry before the IFC activity; however, within the activity, El Salvador redesigned its system and made it compatible with the regional platform.  Not all countries connected themselves to the regional platform at once. Nicaragua and Costa Rica started using it in January-February 2017.  Online payments are available only in Costa Rica. On average, time for mutual recognition has decreased two days thanks to the system. The time in Honduras and Nicaragua is one day; Costa Rica, five days; and El Salvador, eight days. The level of system adoption by each country varies greatly. El Salvador had its sanitary registry thoroughly digitized before the USAID-IFC activity started; however, the country took advantage of it to make it compatible with the regional platform so that the recognition of sanitary registry for foods and beverages is done through this platform. The country went further and got rid of its Certificate of Free Sale5 (CFS) considering that the system already has the information required by the CFS. Such thinking is not shared by either Honduras or Guatemala, as these countries deem it indispensable,6 arguing that a 2004 COMIECO resolution makes it mandatory. An obstacle El Salvador still faces is a legal restriction that makes impossible to have online payments, although the electronic payments law has been passed but not yet entered into effect. 5. A Certificate of Free Sale, sometimes called a "Certificate for Export" or "Certificate to Foreign Governments", is an assurance from a foreign country that the products listed on the certificate are freely sold and manufactured in such country. 6. The Certificate of Free Sale is an important source of revenue for the ministries responsible for it. 18 In Honduras, the sanitary registry is still paper-based. The ET learned from SIECA that the country will be supported by IFC to have its system digitized by the end of 2017, to be developed by the same IT company that built the regional platform. An extra obstacle in Honduras is that, by law, online payments must be made through National Bank of Agriculture and Livestock Development (BANADESA), a government bank which lacks the technical infrastructure to process them. Despite having been provided with a digitized national sanitary registry, Guatemala continues feeding the regional platform from an Excel spreadsheet. Moreover, its sanitary authorities not only are persistent in demanding the CFS, but also consider that the regional platform must require laboratory analysis with a signature and stamp from the country of origin. From the business user standpoint, companies such as UNILEVER, DINART, WALMART, and MONDELEZ, were invited to an online pilot test of the system that left them satisfied; however, later real￾life use, some of they say, has proved the system to be slow and not very user-friendly. 4.1.11. How did RUTA’s activity change the capacity of communities of practice to perform as networks disseminating best practices? The platforms and networks of the cocoa and coffee value chains supported by RUTA, known as "communities of practice," have improved their operations and management systems. The communities of practice sought to bring together farmers, processors, companies, universities and other research institutions, government, and other programs and projects related to the value chain of cocoa in each country and to coffee in the Trifinio region for the dissemination of knowledge and learning experiences, in this case, good agricultural practices or other technologies for sustainable agriculture. RUTA:  Provided technical assistance to establish the Coffee Working Group of the Trifinio Region and gave technical and financial support to various Trinational Fora, Specialty Coffee Competitions, and training for farmers and small-scale processors. Figure 2 Screenshot of the Sanitary Registries Regional Integration System 19  Trained the Cocoa National Committee of Honduras on the “community of practice” methodology and provided technical assistance to strengthen its organizational skills.  Provided technical assistance to the Cocoa National Working Group of El Salvador to facilitate the organization of the sub-working groups and their coordinating body; and provided technical and financial support to produce educational materials on cacao genetics.  Provided technical assistance to the Cocoa Value Chain Group of Guatemala, which is organized by the National Council of Agricultural Development (CONADEA), to develop a national strategic plan for the value chain. 4.1.12. (RTMA) To what extent have the activities under the RTMA supported producers of selected agricultural value chains to improve access to regional and export markets? This question has been answered under the question found in Section 4.1.4., which asks: To what extent have activities contributed to market alliances and mechanisms that will likely last beyond USAID activity implementation? 4.1.13. RTMA Focus groups: How did the RTMA project contribute to changes in income, consumption patterns and living standards of small farmers? In FGDs and key interviews, no evidence of change in farmers´ incomes, consumption patterns, or living standards were found yet to be attributable to the activity. The development of each one of the RTMA projects is different. They range from those which have not even started operating yet, despite being in place for a while, to those whose production has started as recently as December 2016, and therefore are not applicable for effects of this evaluation, which ends with September 2016. Concerns on project implementation arise from situations where costly infrastructure has been inoperative for months due to the absence of electric power, flawed equipment purchasing decisions, lack of government permits and obstacles to market access, including government red tape, trade and food security policies, and price controls. Corruption in various sectors and levels is mentioned as an impeding factor as well. RTMA was intended to work with farmers’ groups (associations, companies), not at the farm-level or directly with producers; therefore, its income benefits to farmers will be necessarily indirect; and, even if present, have not reached farmers’ pockets yet. For example, at the time of this evaluation, ARSAGRO, one of RTMA-supported projects, was expecting profits from the investment project at the end of the 2016- 2017 accounting cycle. The previous observation highlights a significant contrast in the Strategy design versus some of its expectations. For the most part, the Strategy was conceived to act, particularly on trade, at the institutional or organizational level, not the farm-level. However, on the food security and nutritional side, the Strategy seeks to match FTF expectations regarding improvement of income at farm level. This is a mismatch between high level regional purposes and the project-level means assigned to attain them. Communities of practice are inter￾institutional and intersectoral coordination spaces that contribute to the strengthening of the joint learning of the actors, since they generate solutions, propose innovations, and join efforts for the continuous development of a common area of specialized knowledge. The groups collaborate and learn from each other, and are united by a common goal and the desire to share experiences, knowledge, and best practices within a theme or discipline. 20 Out of the three producers’ groups, one has been able to successfully increase their profits so that they might eventually improve producers’ living conditions. The other two groups have not yet obtained direct additional income from the RTMA projects, and so their families spending and consumption patterns remain unchanged. 4.1.14. Have the activities had any impact on migration? As a verbal request, outside this evaluation´s SOW, USAID/CAM asked the ET to research whether the activities had any influence in decreasing migration from the areas where they were implemented. The specific question was: Since the activity began, has the increase in income or food security been sufficient so that fewer young people and families emigrate from the area? During focus groups and interviews, the subject was discussed with stakeholders. The ET learned that in the farmer communities visited, such as Cobán or Petén in Guatemala, La Esperanza Intibucá in Honduras, or El Paisnal in El Salvador, farmers believe that migration, even if it does occur, is not a significant issue, and therefore, a relationship between the Strategy activities and migration could not be found. Opinions on the issue vary. For example, while a coffee industry leader in El Salvador attributed migration mostly to violence, one representative from USAID/Guatemala, referring to young people, stated that “we have anecdotal evidence that international migration has declined because they can stay with their families in coffee and horticulture areas.” However, no concrete statistics are available. Nonetheless, it must be noted that PAPA introduced a training initiative, per its COP, as a response to the influx of young immigrants from Central America to the U.S. It is an integrated pest management curriculum for three agricultural high schools in the Western Highlands of Guatemala, with support from experts from Penn State University. The program trains approximately 120 students per year to be technicians for agricultural export companies. The pilot program has been qualified as “successful” by USAID (USAID, 2016). However, placement with export companies has lagged. The COP theorized that export companies prefer they become farmers; they do not yet see the benefit of hiring them as technicians. The findings of this evaluation fit in a context where neither the Strategy, nor the SOW of its four activities include migration as a programming consideration. 4.2. FINDINGS ON THE EVALUATION CRITERIA After responding to the questions asked by the SOW, in the following pages, this evaluation analyses the Strategy under a complementary approach using the following evaluation criteria: 1) relevance, 2) effectiveness, 3) efficiency, and 4) sustainability in addition to cross-cutting issues such as gender, social inclusion and the environment. These criteria are not only a set of standardized, globally understood, and accepted evaluation measures, and are used by major development organizations, including USAID (in its Evaluation Policy7 ), but also, in practical terms, most of SOW evaluation questions, essentially refer to one or more of these evaluation criteria. Based on these criteria, this evaluation provides a more complete and consistent assessment of the Strategy. It must be noted that the evaluation criteria work in a relational manner; they both complement and condition each other. For instance, a “highly effective” project must be scored lower if its results came to be too costly or too late, which means it is less efficient, and so on. 7. USAID. Evaluation Policy 2011, updated 2016, p4. 21 Finally, since the Strategy is considered a project as per USAID policies8 the principles, practices, and phases of the project lifecycle (project analysis/initiation, project design, execution, monitoring, and closing) are considered along with the evaluation criteria. 4.2.1. Relevance For the purposes of this evaluation, relevance is defined as: “The extent to which the objectives of a development intervention are consistent with beneficiaries’ requirements, country needs, global priorities and partners’ and donors’ policies”9 . The definition notes that “Retrospectively, the question of relevance often becomes a question as to whether the objectives of an intervention or its design are still appropriate given changed circumstances”. While this definition focuses only on the objectives of an intervention, as the Strategy is a project, relevance must include not only its objectives (ends), but also its four activities (means). Both must be relevant, which is to say be an adequate response to needs, expectations or requirements that people, institutions, or policies have. In terms of project analysis (background), considerations taken by the Strategy to validate itself and, therefore, attest its initial relevance, were as sound in 2011 as they are now and thus, clearly have justified such initiative over time. They are: the food security challenge, the potential of a Central America common market, regulatory and other market distortions, logistical complexities, the potential of agriculture value chains, etc. For USAID, the Strategy is relevant as it is attuned with its own, as well as other U.S. Government (USG) policies as it intends to support FTF as part of USAID/CAM Increasing Regional Economic Integration Project and its RDCS. Regarding project design, the ET finds certain mismatching between the Strategy objectives and goals, and the means (the four activities and its interventions) chosen to accomplish them, which inevitably undermines its relevance for certain key stakeholders. In simple terms, whereas the Strategy’s main development goal is to “eradicate hunger and poverty” (in rural households and small farms) in Central America, the means it chooses to accomplish that goal are significantly less oriented to individual small farmers and more towards institutional strengthening, capacity building, and improving policy-making, as seen in Figure 3. That is why when the ET interviewed government institutions, large businesses, other private sector organizations, regional institutions, and USAID bilateral programs, they opined that the Strategy’s four activities have been highly relevant, as they supported their mission and programs. The Strategy’s rationale is sharply focused on food security and nutrition as it intends to “support the U.S. Government Global Hunger and Food Security Initiative known as Feed the Future (FTF)”, and so, it recognizes “The Food Security Challenge”10, or advocates for “Better vs. More Food Production”11, or appeals to the FTF 2010 Strategic Review to state that “the CAM strategic focus will improve regional food 8. Per the ADS 201.6 Program Cycle Operational Policy, “A project is a set of complementary activities, over an established timeline and budget, intended to achieve a discrete development result” 9. Organization for Economic Cooperation and Development, OECD, ib 10. USAID, RITFS, p4 11. USAID, RIFTS, p6 22 security and nutrition…”12. However, of the two pillars of the Strategy: food security-nutrition and trade facilitation, the latter is the subject of more effort, initiatives, and funding. Regarding relevance at the activity level, PAPA training shows a contrast in relevance because although it reached to more government officials than farmers, the kind of measures and agricultural practices it promotes, are ultimately of benefit for farmers. PAPA interventions in food safety have been highly relevant to all stakeholders interviewed. The relevance of RUTA’s interventions on regional public policy effectiveness, initially focused on analysis tools, was low, but after changing towards the course on food security and nutrition policy, its relevance increased significantly. There is a sign, however, that the relevance of the RUTA activity for USAID decreased as it was cancelled six months early. Relevance of the IFC activity is high in theory, as registry mutual recognition of manufactured foods and beverages is a highly desirable objective; however, the persistence of some governments in keeping their isolated and antiquated information systems, indicates otherwise. In analyzing the programming tradeoffs between hunger/poverty vs. institutional strengthening as well as food security-nutrition vs. trade facilitation, one element of the Strategy that became a key factor and potential differentiator in terms of relevance, was the selection of value chains. And that is because strengthening value chains was expected to be the source of income for small farmers which, in turn, would lead to food security. Whereas RUTA´s choice of value chains (coffee and cocoa) was a straightforward process, RTMA’s was, in opinion of several interviewees, a convoluted and unnecessarily long effort. The following table shows how RTMA criteria to select the value chains to be supported kept changing over time: from the Strategy document, to the first year of implementation. 12. USAID, RITFS, p11 Figure 3 RITFS Conceptual Design 23 Table 1: Comparison of Criteria for Value Chain Selection (RTMA) The Strategy RTMA RFP RTMA CONTRACT Nathan 1st Year Work Plan Integration of small producer groups in value chains, Rural households and small producer’s income generation potential, Food and nutritional security for rural households, Achieving maximum multiplier impact on the development of infrastructure and investment. Tapping the capabilities of research centers in the region Benefit small producers; Job opportunities for women in the rural areas; Improvement of farmers’ and poor rural population incomes; Improving the region’s food security and nutritional needs; Achieving a multiplier impact on the development of productive investment. The Program seeks to expand regional markets for commodities with impact on small farmers’ incomes: horticulture, fruits, beans and other vegetable proteins, fish, meats, eggs, dairy and dairy products. Choose products for export (or value chains) using the following criteria: Benefit to small producers. Job opportunities for women in rural areas. Improvement of farmers’ incomes and those of poor rural inhabitants. Improvement of the region’s food security and nutrition. Achievement of a multiplier effect on investment. The selection of the value chains is based on criteria such as: Market growth potential, Enabling environment (trade, policy, institutional), Availability of information, Social impact (job creation potential), Comparative advantage regionally or nationally, geographic focus and accessibility, Political and development support (ability to leverage USAID funding or another donor funding) and Infrastructure availability. Table 1 shows how some criteria originally heralded by the Strategy, at the time of RTMA implementation ended up being diminished or dismissed altogether: i) food security and nutrition, ii) improvement of rural households and small farmers’ incomes, and iii) job opportunities for rural women. Likewise, criteria such as “enabling environment” that are more abstract became an adopted criterion. Finally, USAID/CAM, by deciding that “direct targeted interventions to affect individual farmers’ incomes or household incomes are out of the scope of this project”13, stripped the Strategy of several of its foundational elements, and therefore its relevance for rural household and small farmers was nullified. In terms of relevance, there are other activity-level design flaws noted by the Office of Inspector General, such as the RTMA-adopted notion of value chain, proposing increasing exports for producers that have never exported before, setting targets for participating small producers without solid knowledge/research supporting it, and adding or removing participating countries at will. 14 The degree to which the Strategy was relevant for each category of stakeholders is illustrated in Figure 4. 13. USAID office of Inspector General. Audit Report No 1-159-17-001 P. October 2107, p 11 14. USAID office of Inspector General. ib, p 6 Figure 4 Relevance of the Strategy for its Stakeholders 24 4.2.2. Effectiveness This evaluation adopts the definition of effectiveness crafted by the Organization for Economic Co￾operation and Development (OECD): “The extent to which the development intervention’s objectives were achieved, or are expected to be achieved, considering their relative importance.”15 The evaluation complements the definition emphasizing that the word “achievement” must refer to actual results, not merely to intended ones. The effectiveness being evaluated here is the Strategy’s. Although the results of its four activities are necessarily considered, they are not the focus of this effectiveness analysis, because the SOW of this evaluation requires it to “assess the implementation of the Strategy as a whole with all four activities contributing directly and designed to act in coordination and in a complementary manner, and determine if the design and implementation of the four activities under the Strategy were effective in achieving the Strategy’s expected results.” Consequently, the analysis of effectiveness is based on: 1) the Strategy as a high-level document, its objectives, and expected results; 2) the results the Strategy assigned to each one of the four activities; 3) the actual results obtained by the four activities; and 4) the assessment of the extent in which the latter contributed to the achievement of the Strategy’s results. The Strategy development objectives16 are stated as follows: Its main goal is: “To help Central American countries to eradicate hunger and poverty”. Its strategic focus is: “To improve regional food security and nutrition”, and the results to be achieved by the Strategy are: “R1: Consolidated regional value chains and export market access for food and nutrition critical products; through increased trade facilitation, harmonization and institutional capacity. R2: Increased effectiveness of regional food security policy and programs through the establishment of a monitoring and evaluation system; facilitation of dialogue and peer review process; and promotion of research and dissemination of innovative sustainable agriculture practices.” The ET did not find evidence that the top-level development objectives of the strategy have been achieved. The overarching goal of helping Central American governments to eradicate hunger and poverty was not supported by tangible actions that occurred within the Strategy’s activities or the dedication of resources for that specific purpose. Likewise, the Strategy’s stated focus on food security and nutrition got lost on decisions that made food security a subsequent objective, the indirect result of expected income improvements due to access to new or increased market opportunities for producer organizations; and decisions supposing that better nutrition would be achieved through the selection of key agriculture value chains. In other words, the major development goals (poverty, hunger, food security, and nutrition) vanished in budget, programming, and activity decisions and implementation. At the activity level, the Strategy defined a set of very general indicators that later would be specified (and at some extent changed) by the activities’ SOW. The performance of those indicators, as a measurement of the effectiveness of each activity, is analyzed in the following pages. 15. OECD Glossary of Key Terms in Evaluation and Results Based Management”. 2010 16. It also addresses the following FTF objectives: i) expanding markets and trade, ii) increase investment in agriculture and related activities, and iii) increase employment opportunities in targeted value chains 25 PAPA The overarching goal of PAPA is to increase income of the rural poor and small farmers through agriculture￾led market opportunities and trade facilitation by strengthening the capacity of regional governments’ agricultural institutions and private sectors with targeted technical assistance, exchanges, and training activities. The objective of PAPA is to enhance food security and promote regional agricultural trade. The scope of PAPA emphasizes technical assistance and training to: 1) build public and private sector sanitary and phytosanitary (SPS) capacity; 2) improve access to regional and international markets, including market information; 3) improve statistical systems to better inform policy makers on matters of food security; and 4) address other agricultural products under the Dominican Republic-Central America￾Unites States Free Trade Agreement (CAFTA-DR). PAPA has proven to be effective in providing training on SPS and market access subjects to various groups of stakeholders, for a total of 4,268 people between 2012 and 2106. Although no specific activities were programmed for women, 29.1 percent of trainees were female. Training results exceeded expectations by 34.4 percent (108 percent among government officers); however, it fell somewhat short in outreach to producers by meeting 95 percent of the expected results. Likewise, PAPA did not meet its goals for the period 2012-2016 regarding the number of policies/regulations/administrative procedures it advanced (16 advanced /18 planned). In this regard, it must be noted that in project reports, these measures are frequently referred to as “laws,” which implies legislative approval, when they are principally regulations approved at the ministerial or executive level. PAPA was also expected to help 50 organizations, either private firms or government entities, make procedural and/or operational improvements in areas such as: integrated pest management, FSMA compliance, good agricultural practices, hazard analysis and critical control points, good manufacturing practices, laboratory diagnostics, pest's surveillance programs and market information systems. Reports show that 93 organizations have participated in projects activities; however, PAPA has no information on improvements (or lack thereof) those organizations might have experienced because of the training. Likewise, PAPA did not monitor, as expected, other performance indicators such as “percent change in value of intra-regional exports of targeted agricultural commodities as a result of USG assistance,” and “increase of institutional capacity of relevant national statistical offices to collect high-quality agricultural data.” Listed in decreasing order, the degree of effectiveness of PAPA interventions would be: It is greater in training (which, by the way, is more focused on food safety than food security), and then lower regarding policy-making, market information, organizational improvement, strengthening of statistical systems, and the least effective intervention is in increasing the value of intraregional exports. RUTA The purpose of the RUTA activity is to promote inclusive economic growth, improve food security, and facilitate trade in the region. It focuses on two specific objectives: 1) regional food security policy effectiveness; and 2) promotion of sustainable agricultural technologies in Central America and the Dominican Republic. RUTA’s expected results are named and discussed below. One of RUTA expected result was “A regional framework for monitoring and evaluation at the national level for Central American governments and for the implementation of the food security agenda developed and consulted with governments, donors and private sector; and presented to the Central American Council of Agriculture (CAC)”. This expected result, as described, was never achieved; however, several steps in that direction were taken by: 1) building a database on food security policies for El Salvador, Guatemala and Honduras (http://www.ruta.org/bdpsaweb/index.php); 2) a series of research documents on food security policies and agriculture sustainable technologies, all valuable for policy-making purposes; iii) 26 adaptation and training on the Policy Analysis Matrix (http://www.ruta.org/programa-agroalimentario￾sostenible/?q=map) as a methodology for analysis of profitability and competitiveness of agricultural products key for food security and the impact of policies on them; 3) design and implementation of policy M&E system called SISMEPA-Agrifood Monitoring and Evaluation Regional System. These efforts, however, were dismissed by USAID in the middle of the component’s implementation because, as stated by a stakeholder: “it made no sense to build tools (for policy analysis) if people are not trained to use them.” As an alternative to the unsuccessful M&E platform-building attempt, in 2016, RUTA, with support of FAO and IPFRI, designed and implemented a training program in food security public policy called “Food Security Public Policies Diploma” offered in El Salvador, Guatemala and Honduras. It was a university￾level program on policy analysis, policy-making, and policy M&E received by 88 people (31 in El Salvador, 30 in Guatemala, 27 in Honduras) with an average of 41 percent of women participants. The training program has been considered highly successful by most food security stakeholders in the three countries, particularly government institutions. And that is how a flawed activity design (the regional platform) turned into a successful one (the Diplomado), achieving an increase in relevance that, in turn, led to greater effectiveness in another RUTA intervention, described as follows: “Development of at least four policies, regulatory or programmatic actions to improve the regional agri-business environment, and adopted by the CAC”. Because of the Diplomados, 11 food security policies were crafted by the students: 6 in El Salvador, 1 in Honduras, and 4 in Guatemala. The intervention consisting in developing the policies, presenting them to the CAC, and attempting to have CAC adopt them, was effective in the proposal-developing part, as those were new proposals, elaborated under a specialized training program, and worked on by students who were mostly government officials. However, although some of those policies are being considered by authorities as they develop their own official policies, they have not been effectively adopted by those governments, much less by CAC. Regarding its expected result “At least six memorandums of understanding to be signed by CAC/RUTA/UNOPS with regional agriculture research centers for identification, validation, and dissemination of sustainable agriculture practices relevant to food security and applicable to small farmers”, RUTA signed 3 MOUs with regional agricultural research centers and 15 with communities of practice and producer organizations. Regarding the other expected result: “At least six new sustainable agriculture technologies to be adapted for small farmers, and disseminated through government extension services and relevant small farmers’ technical assistance programs,” RUTA had 12 new technologies or management practices disseminated and adopted by small farmers. Likewise, RUTA was expected to have a training plan designed and implemented for capacity building. The result in this aspect is a total of 3,002 persons (2,169 men and 833 women) have received USG supported short-term agricultural sector productivity or food security policy training. On the technical assistance side, RUTA assisted in institutional strengthening 15 organizations and institutions. In its interventions, RUTA was more effective in the strengthening and establishment of networks and communities of practice, as well as promotion of sustainable agriculture best practices and technologies, and least effective in creating a regional framework for M&E for the implementation by CAC of a regional food security agenda. RTMA The purpose of RTMA is to promote inclusive economic growth, improve food security, and facilitate trade in the region. The activity has two major components: 1) market access for critical food security value chains; and 2) increased trade facilitation and improved trade and institutional capacity. 27 Determining the effectiveness of RTMA in general, but especially its first component, was a great challenge for this evaluation due to several factors: 1) the head of component one no longer works with Nathan Associates; 2) availability of information regarding the grants and their performance has been very scarce; 3) access to documents with regard to key implementing decisions has been close to none; 4) information on execution of resources was denied as the IP argues it is “classified information”; 5) some clarifying documents were provided only a couple of days before field work ended; and 6) RTMA changed baselines and indicators several times throughout activity´s lifecycle. The first component supports critical food security agriculture value chains, linking small and medium producers to regional and export markets (to and from Central America, and to other export markets). It also consolidates intra-regional value chains to leverage underutilized market opportunities. The main purposes of this component are to link producers to buyers, servicing markets in the region, as well as in the U.S. and other export markets; promote compliance of quality and environmental sustainability standards; and assess investment and trade provisions with an emphasis on creating private sector alliances. The first intervention of Component I was the selection of the food security critical value chains, a decision process whose effectiveness was impacted by factors such as: 1) the criteria to make this selection kept changing and being re-prioritized from the Strategy’s design, through its SOW, the contract, and the first￾year work plan; 2) considerations of food security and nutrition were left out, according to RTMA officers, with the argument that FTF funds had been reduced in the activity; 3) there is no evidence that RTMA was implemented with an actual value chain approach (including all the chain´s productive links); 4) emphasis was placed more on some products that on overall value chains; 5) government officials stated that their institutions were not made part of the value chain selection process, they were only informed; likewise, in their opinion, although such chains are important for their countries, they were not prioritized based on participation of small, vulnerable farmers; and 6) effectiveness was compromised by the duration of the value chain selection process which was too long, about 2/3 of the component´s lifetime. With respect to the RTMA grants program must be noted that it cannot be considered a gifts program; by design, it is a program of investment in manufacturing infrastructure. As such, evaluation of each one of those investments must be based on their own financial, technical, and market data, at least. Regrettably, such information was not made available for this evaluation. Another expected result was an increase of at least 15 percent in exports sales value of participating producer groups and buyers. In that regard, this evaluation shares the comment of the Office of the Inspector General, questioning such an indicator for farmer organizations that had never exported before. Nonetheless, a few of these organizations, jointly, did exceed the indicator in 0.7 percent. While they represented 38 percent of the grants total dollar value, the other farmer organizations, recipients of 62 percent of the grants value, did not export at all. This finding applies as well, to the expected result asking for, at least, 1,500 small producers units participating in regional and export value chains. Moreover, at least three MOUs were expected to be signed with regional or national institutions to support implementation of regional value chain development; at least four special regional trade mechanisms implemented; and, at least, 20 regional buyer-seller alliances established to link small producers and small enterprises through a value-chain development approach with at least five resulting in sustainable market driven contract relationships. These three expected results relate to a key component that gives its name to this initiative: “Market Alliances,” an area of implementation where RTMA fell short. The evidences the ET could find attest that RTMA´s effort in this regard, consisted of supporting producers’ participation in trade fairs and similar events. No evidence was found of any RTMA achievement in structuring any export￾oriented alliances, either towards intra-regional markets or the U.S. The “alliances” that could be verified 28 were more letters of intention, without any contractual-commercial value and, mostly, between parties already known. In promoting this kind of “alliances,” RTMA did meet the indicator. Regarding performance of RTMA’s Component 1 (Market Access for Critical Food Security Value Chains), the ET, based on the information it was provided with, found fully applicable the comments made by the Office of Inspector General in its Audit Report dated on October 28, 2016 titled: “USAID El Salvador Needs to Improve its Management of the Regional Trade and Market Alliance Project to Achieve Expected Results,” chiefly the ones related to the following issues: 1) 15 percent in increased exports; 2) export￾oriented market alliances; 3) project impact on farmers´ income and nutrition; 4) performance monitoring and data standards; and 5) sound rationale for defining intervention targets. The second component of RTMA aimed toward increasing trade facilitation and improving trade and institutional capacity. To that end, the expected results were: 1) reduce to two days the average time to trade goods across borders in the region; 2) trade facilitation monitoring system implemented and adopted by a regional private sector organization or national organizations; 3) a trade and customs specialist certification program implemented and adopted through partnership(s) with academic center(s); 4) one regional or five national MOUs with private sector organizations to promote the Central American Trade Facilitation Agenda; and 5) at least four policies, regulations, and administrative procedures consulted among stakeholders and presented to COMIECO. RTMA’s result regarding the above-mentioned interventions was the two days’ reduction in trading goods across borders proved to be a too simplistic and an undeterminable indicator and so, it was changed to something more meaningful as part of using the FAST PATH methodology, developed by Nathan & Associates which, based on research and analysis, led to a series of unprecedented suggestions to be implemented at customs border crossing stations. Such changes have resulted in a 45 percent time reduction between 2014 and 2016, or from 2 hours and 5 minutes to 1 hour and 7 min; likewise, the time to process exports has seen a decrease of 29 percent. Beyond the figures, it is worth noting that this intervention is based on a consistent methodology applied over time, bringing about a series of measurements and data bases that were not available before. Another intervention of this component and an expected result is a “trade facilitation monitoring system implemented and adopted by a regional private sector organization or national organizations.” From its inception, RTMA intended to work with El Salvador, Honduras, and Guatemala, but had to face several customs-related institutional and political obstacles, and ended up working on implementation mostly with El Salvador. The system is designed to facilitate decision-making regarding imports, exports, operations, logistics, traffic, etc. RTMA has supplied cameras, antennas, transmitters, to implement some Radio￾Frequency Identification (RFID) systems based on data from measurements at border crossing posts made on a daily basis. To make this possible, RTMA hosted technical national and binational work roundtables. Currently, as stated by the Honduras Director of Integration, the system can be accessed only by high level officials until it is transferred to SIECA. A “Trade and customs specialist certification program implemented and adopted through partnership(s) with academic center(s)” is another result expected from RTMA. Three MOUs were signed by universities, but training was implemented only by Universidad Dr. José Matías Delgado in San Salvador. Some courses were given by Central American Federation of Chambers of Commerce, whereas SIECA hosted a second round of this certification and it is intending to keep doing it in the future. An additional result expected was: “One regional or five national Memorandums of Understanding with private sector organizations to promote the Central American Trade Facilitation Agenda.” In this area, RTMA contributed to an agreement for regional trade facilitation between the Consulting Committee for Economic Integration (CCIE), the Nicaraguan Foundation for Social and Economic Development, the 29 Central American Federation of Exports Chamber and Associations, and FUSADES, an agreement that the signers deem as a way to “strengthen the public-private dialogue for trade facilitation in the region.” Likewise, RTMA signed an agreement with CAC to cooperate in training activities, both in person and online. At national level, RTMA signed several MOUs with organizations such as the Economic Development Secretary in Honduras, the Costa Rican Chamber of Transporters, the Central America Chamber of Transporters, among others. “At least four policies, regulations and administrative procedures, consulted among stakeholders, and presented to COMIECO” is another expected result which led to documents/proposals such as “Analysis and Evaluation of Central America Economic Integration, recommendations and a guide for an agenda for the Implementation of the Customs Union” (2014); and “Recommendations and Guide to COMIECO for the Implementation of the Customs Union.” Along with other regional stakeholders such as the Inter￾American Development Bank (IADB), the Federation of Industrial Chambers and Associations of Central America and the Dominican Republic (FECAICA) and others, RTMA worked on the “Central America Strategy for Trade Facilitation and Competitiveness with Emphasis in Coordinated Border Management” proposal that was approved by COMIECO in 2015. Finally, technical assistance provided to SIECA has resulted in greatly improving its organizational capacity to support the Central American economic integration process, as discussed in Section 4.1.2 of this report. Based on the scarce information received from RTMA regarding its first component (Market Access for Critical Food Security Value Chains), the ET can only assert that its effectiveness is inconclusive. In contrast, Component 2 (Increased Trade Facilitation) per the evidence found, showed significant effectiveness in interventions such as support to institutional strengthening of SIECA, the studies and proposals to customs for trade facilitation at borders, and promoting regional trade facilitation. IFC The objective of the USAID-IFC Central America Agribusiness Trade Logistics activity is to streamline, harmonize, and automate procedures for mutual recognition of sanitary registration for processed food and beverages in Costa Rica, El Salvador, Guatemala and Honduras. These procedures include first time registry and mutual recognition of registries among ministries of health. Expected results at the end of this activity were: 1) average number of days to register a processed food product for the first time reduced from 80 days to 55 days; 2) average number of days to register a processed food product for mutual recognition reduced from 8 days to 1 day; 3) average cost to register a processed food product for the first time reduced from $400 to $300; 4) average cost to register a processed food product for mutual recognition reduced from $145 to $120; 5) number of documents required to trade for mutual recognition registration reduced from 4 to 2; and 6) number of product registries mutually recognized increased from 4,633 to 5,096. Evaluating the effectiveness of this activity was impossible due to a lack of M&E data. IFC officials told the ET that at the time of the interviews a baseline report was being prepared for USAID. Moreover, although the activity has a document called “Monitoring and Evaluation Plan,” there is no actual follow￾up of it since officials believe that “given the nature of such indicators, data on them is possible to gather only until the intervention has ended,” which is the same rationale for the activity not having a mid-term evaluation. Overall, the most outstanding side of this activity is promoting and facilitating regional dialogue on mutual recognition of sanitary registries for foods and beverages, an effort that entails a great deal of lobbying, persuasion skills, and persistence. 30 IFC has hosted several training activities for various audiences: public, small business, private sector as well as the officials in charge of the registry in each health ministry. Those training sessions were meant to promote the Regional Integrated Sanitary Registries System, as well as effort being done on simplification, automation, and harmonization of the mutual recognition process for food and beverages sanitary registry across the region. For the officials, training activities also included hands-on management of the platform. On performance of the Regional Integrated Sanitary Registries System, an end-of-year IFC report informed that: 1) it was activated in June 2016; 2) 1,347 requests for mutual recognition have used the platform; and 3) the days required for mutual recognition is estimated to decrease an average of two days to one in Honduras and Nicaragua, three in Guatemala, five in Costa Rica, and eight in El Salvador. This progress, however, keeps been hindered by operational obstacles such as legal conditions for online payments in El Salvador and Honduras, or government reticence to eliminate the CFS (which is a source of revenue). Since the core intervention of the USAID-IFC Central America Agribusiness Trade Logistics is its digital platform called the Regional Integrated Sanitary Registries System, and no performance indicators of the system have been registered and followed, the ET cannot present an evidence-based effectiveness assessment about this activity. 4.2.3. Efficiency The evaluation criteria of efficiency can be defined as “A measure of how economically resources/inputs (funds, expertise, time, etc.) are converted to results” (OECD, 2010). Collaboration with other institutions, as is described in Section 4.2 of this report, can enhance an activity´s efficiency through cost-sharing. PAPA and RUTA activities made extensive use of the benefits of collaboration with other institutions. This synergy was pointed by various interviewees concerning the RUTA activity which, in their opinion, generated a high level of efficiency in the use of its financial and human resources. Similarly, the PAPA activity has only one direct employee, its COP, but through coordination with U.S. and Central American public, private and academic institutions, it was able to organize actions valued at over $1.25 million per year. Furthermore, because it is based on a USG inter-agency agreement, it has been able to leverage human resources from within USDA and land grant universities, instead of using more expensive outside consultants. Experts used were generally from USDA for work in market information systems, and from universities for SPS issues, including Penn State, Ohio State, and Texas Tech. Also, both PAPA and RUTA achieved a high level of financial execution. As mentioned in this report, not enough budgetary information was provided by the RTMA activity so that the team could evaluate its efficiency. Looking towards the future, the option to move courses online was examined with interviewees from ministries and export companies, which would allow the courses to reach more participants. Experience with previous virtual courses on SPS matters, such as those organized by IICA and by the Regional International Phyto and Zoo-Sanitary Organism (OIRSA), has been generally positive, allowing for participants in different regions to attend without having to gather in one central location. As one ministry representative stated, “in the medium term, we have to migrate to virtual training.” 31 4.2.4. Sustainability Sustainability, as it relates to project evaluation, can be defined as “The continuation of benefits from a development intervention after major development assistance has been completed. The probability of continued long-term benefits. The resilience to risk of the net benefit flows over time” (OECD, 2010). When analyzing sustainability in terms of the Strategy and its activities, one must first determine if that sustainability is independent of current or planned USAID activities, or not. For example, the PAPA activity will continue until 2019 and, according to its COP, may possibly be extended to coincide with the ending of other USAID activities in the region. The RTMA activity was also extended, and USAID/CAM representatives have suggested that further efforts for trade facilitation in the region are expected. However, sustainability should be gauged independently of USAID´s future actions, and was construed in this fashion in the activities. For example, the signed agreement of the PAPA activity (USAID, 2011) indicates that “appropriate partners that can provide in-country assistance in capacity building aspects should be identified and alliances built in. Sustainability should be pursued to this end.” Several aspects that are independent of current or planned USAID activities can shed light as to the sustainability of the Strategy´s activities. First, is the extent of collaboration with other institutions, as described in this report. For example, not only did the RUTA activity begin its closure in early 2017, the whole RUTA project is shutting down due to a lack of future funded activities. However, despite the closure of RUTA, stakeholders expect that the work done through grants to farmers´ organizations will generally continue through other collaborating partners or cooperation agencies such as Vredeseilanden from Belgium, with the cocoa￾related regional initiative called VECO Mesoamerica in which Catholic Relief Services, and Lutheran World relief are involved as well. El Salvador also has a rather large cocoa initiative ongoing, the National Cocoa Roundtable, and FHIA is expected to continue assistance in Honduras. Another factor for sustainability in the RUTA activity may be through self-financing mechanisms. For example, the Trifinio Coffee Working Group expects to begin charging for participation in events and retain a small percentage of coffee sales made through the group. Continuity of the food security policy course designed and funded through the RUTA activity is less certain, although FAO has indicated its interest if funding can be secured. For the PAPA activity, some interviewees from the public and private sectors stated that the necessity to keep markets open in the U.S. for Central American products would hasten the strengthening of public￾private sector alliances to maintain further training efforts for FSMA and other future requirements in the absence of USDA support for these activities. As the private sector, in particular, continues to take ownership of these future challenges, they will be more capable of organizing training. Sustainability of the RTMA activity is tied, in part, to its coordination with Inter-American Development Bank (IADB), which has taken leadership among the development community for the implementation of trade facilitation measures in the region. As IADB efforts continue, the advances achieved by RTMA are more likely to be maintained. Furthermore, perhaps the strongest indication of sustainability for this component of RTMA and for the IFC activity are that SIECA will take over the administration of regional systems for trade facilitation and for mutual recognition of sanitary registries. A much more nebulous question is related to the sustainability of the RTMA grants to farmers´ organizations. As discussed previously, several organizations have not effectively used the equipment given, and their financial sustainability is questionable. 32 4.2.5. Cross Cutting Issues The findings on cross-cutting issues of gender considerations and social inclusion have been described in evaluation question number 4. The remaining relevant cross-cutting issue is in relation to environmental considerations, or as the Strategy (USAID, 2011) states, “Trade-led economic growth and poverty reduction strategies must be pursued in the broader context of sustainable development.” These environmental considerations are found in only two of the activities of the Strategy. In the RUTA activity, environmental considerations are a common thread in the activity´s work with farmers and their organizations, by promoting practices such as IPM, protection of water sources, agroforestry, and organic farming. For example,  Fundación Propetén in Guatemala implemented the grant project for the strengthening of livelihoods and increasing environmental services provided by cocoa farmers in Petén, Guatemala.  CAFEL in Honduras has been strengthened by the project´s assistance for promoting coffee production with a focus on environmental and economic sustainability. A total of 86 farmers have been certified under Rainforest Alliance and Starbucks Coffee and Farmer Equity Practices.  In the Trifinio Region, the Coffee Working Group´s actions included training on the use of organic fertilizer, water use management, and reduction of sources of water contamination.  In El Salvador, the National Cocoa Working Group created a web page with manuals for best practices in cocoa cultivation, which include IPM and water use management. For PAPA, the principal environmental consideration is reduced pesticide use through IPM training, which may also benefit the health of farmers and their families in FTF areas in Guatemala and Honduras, and consumers (in cases of crops that are also destined for domestic consumption). For example, according to the COP, in Uspantán, Guatemala, before receiving training, farmers were fumigating their snow peas crop 11 times; after training that number has fallen to eight applications, and the goal is five. 5. CONCLUSIONS 5.1. INSTITUTIONAL STRENGTHENING The four activities accomplished several relevant initiatives that brought new capacities to national institutions, such as national governments and universities. That is the case of PAPA on food safety, RUTA training government officials on food security, RTMA bringing capacities to Customs, and IFC improving conditions on ministries of health towards the establishment of a sanitary registries regional platform. At regional level, strengthening of SIECA has been a well-recognized result of RTMA, and ultimately of IFC. With respect to the creation of regional capacities for policy formulation on food security and trade, as envisioned by the Strategy in its “Complementarity and Coordination” section17, except for the significant support received by SIECA, not much progress was obtained. SIECA received substantial improvements in terms of institutional strengthening, capacity building, and modernization initiatives that increased its relevance for the regional integration process. 17. RITFS, p 16 33 The evaluation was not provided with thorough, complete, and relevant documented information (neither organizational, technical, or financial) for each of the grants to answer the question on the grants program and its contribution to build the capacity of agriculture local partners and small producer organizations. Strategy activities contributed very little to create or implement actual market alliances that increased producer organizations actual sales or exports. 5.2. ON COLLABORATION AMONG THE FOUR ACTIVITIES AND ITS EFFECT ON THE STRATEGY Since there were no significant instances of collaboration among the four Strategy activities, nor they were mandated by it, or otherwise monitored or evaluated, it would be mere speculation to say how collaboration (or lack thereof) affected them or the Strategy as a whole. Collaboration among the four activities was a lost opportunity. 5.3. DISSEMINATION AND USE OF SCIENCE, TECHNOLOGY AND INNOVATION This is one of the more outstanding area of the activities’ accomplishment as they did disseminate technologies whether for small farmers, trade facilitation, or reduction in time and cost for cross borders. In fact, PAPA, RUTA, and RTMA directly or indirectly brought training and technical assistance to farmers in many kinds of good agricultural practices, including export-oriented ones, and food safety practices. Likewise, information and communication technologies were used by both RTMA and IFC for trade facilitation. However, it would be more accurate to say that the activities contributed to the use and dissemination of technologies, rather than science and innovation. Likewise, the Strategy’s purpose of creating a regional platform for research and innovation of sustainable agriculture practices to support dissemination for farm level application, did not have any activity or initiative that supported its realization. It was left unfulfilled. 5.4. GENDER AND SOCIAL INCLUSION Although IPsrecognized USAID/CAM’sinsistence in counting women’s participation in training activities, basically it can be concluded that for the Strategy and its activities’ design and implementation, gender equality policies or initiatives were an afterthought. None of the activities complied with ADS 205 regarding the integration of the USAID Gender Equality and Female Empowerment Policy throughout the project cycle. Evidence of non-compliance were that: the evaluators were informed that the project design did not request it; a sample of activities’ annual reports did not show any reference to gender; activity documents did not show evidence of the identification of key gender gaps that the project could address, nor reported on indicators to follow up the project´s effect on such gaps; and activities, mainly RUTA, that showed measures to promote women´s participation seemed to do it of their own accord and not as per any formulated strategy. In the trade sector, USAID officials, and other stakeholders found gender integration to be a matter of good intentions rather than a practical activity programming and budgeting option. Something similar can be said about social inclusion. Even though, nor the Strategy or its activities considered social inclusion in their design, or implementation, this evaluation found evidences that the ethnic makeup of targeted farming populations/organizations automatically favored social inclusion. 34 5.5. IFC, SANITARY REGISTRIES AND TRADE FACILITATION The regional system for mutual recognition of sanitary registries has just begun working at the time of this evaluation. An average of the five countries shows that the time for mutual recognition has decreased in two days thanks to the system. Due to bureaucratic and legal obstacles, its adoption varies by country being led by El Salvador. Stakeholders believe that collection of government fees for sanitary registration plays a role in impeding countries’ adoption of the regional system. The private sector, although interested in having a regional system for mutual recognition of sanitary registries that facilitates trade of food and beverages, has found the system not to be particularly user-friendly. 5.6. RUTA, COMMUNITIES OF PRACTICE AND DISSEMINATION OF BEST PRACTICES The communities of practice seek to bring together farmers, processors, companies, universities and other research institutions, government, and other programs and projects related to a value chain. RUTA was successful in strengthening operations, management systems, and export-orientation of cocoa and coffee producers in the three countries and in the Trifinio region. Some of those communities of practice will remain after the end of the RUTA activity. 5.7. RTMA, SUPPORT TO VALUE CHAINS TO IMPROVE ACCESS TO REGIONAL AND EXPORT MARKETS Conclusions on the RTMA activity were impossible to reach due to lack of access of the evaluation to information on the grants awarded to farmer’s organizations. Based on several documents reviewed, can be concluded that the concept of national or regional value chains applied to producer’s clusters from seed production to consumer availability was absent of RTMA’s implementation. Likewise, the criteria to select awardees of the grants did not take food security and nutrition as its priority, nor the activity focused on assisting the rural agriculture producers to utilize market and trade opportunities, not available for small producers under CAFTA-DR as conceived by the Strategy. 5.8. RELEVANCE The Strategy and its activities did respond to actual needs of Central America countries and regional institutions. However, although the Strategy intended to eradicate hunger and poverty in rural households as its main development objective, its activities were conceived much more towards institutional strengthening of national governments and other institutions. Likewise, being that food security and trade are the two pillars of the Strategy, the emphasis on the former was negligible. In addition, the criteria set by the Strategy for value chain selection ended up being lowered or dismissed altogether during activities’ implementation. In summary, the relevance of the Strategy was significantly greater for USAID/CAM and national governments than for small farmers and poor rural households. 5.9. EFFECTIVENESS The ET did not find evidence that the top-level development objectives of the strategy have been achieved. The overarching goal of helping Central American governments to eradicate hunger and poverty was not supported by tangible actions that occurred within the Strategy activities, or the dedication of resources for that specific purpose. Likewise, the Strategy’s stated focus on food security and nutrition got lost on decisions that made it just an indirect potential result of expected income improvements due to access to new or increased market opportunities for producer organizations; and decisions assuming that better nutrition would be achieved through the selection of key agriculture value chains. In other words, the major development goals (poverty, 35 hunger, food security, and nutrition) vanished in budget, programming, and activity decisions and implementation. Considerations of Strategy/activity design as well as their implementation played a role in affecting their effectiveness. Strategy / Activities Design A Strategy? Although the ET found that top-level USAID officials, both in DC and in the region, consider the Strategy a worthwhile mechanism that potentially gives direction, promotes collaboration among activities and stakeholders, maximizes resources, and should be greater than the sum of its parts, it did not find evidence of the Strategy playing that role, that the IPs had the Strategy or the other activities in mind, or that there was a continuous, decision-making oriented M&E of the activities to ensure they actively contributed to achieve the objectives of the Strategy. One prime example is the non-existent collective contribution of the activities to strengthening local institutions. A Food Security Strategy? Despite its initial extensive conceptual focus on food security-nutrition, and identifying itself essentially as FTF for Central America, over time and for many reasons, the Strategy de￾emphasized this subject and focused more on trade facilitation. That is something USAID/CAM is entitled to decide; however, distancing itself from the food security-nutrition priority in an inertial and silent manner was a choice contradicting the Strategy’s own implementing principle of “consultation and collaboration,” not consistent with the magnitude of something called a “Regional Strategy,” nor a good government policy, or a sound project management practice. Food Safety or Food Security? Even though the two terms seem intuitively synonyms, they are not, and therefore, development policies, activities or interventions for each are different in nature. Indeed, whereas food safety concerns food not causing danger, risk or injury to consumers, oftentimes the term food safety is misused to refer to food availability, food access, and food utilization (the elements of food security). Several interventions of the Strategy, mainly through PAPA, were food safety interventions more closely related to increasing market access for those foods than to securing food security for people. Does income generation lead to improved nutrition? There is abundant research on this subject, not necessarily conclusive. Some state that it does over time and, most importantly, that is not possible to improve a household´s nutrition without improving its income. Others (Fintrac-Honduras) find that income generation alone does not lead to improved nutrition; hands-on nutritional training and technical assistance are required. In any case, this evaluation concludes that agricultural production (income generation) targets and nutrition activities need to be integrated from the beginning of project design and must be resourced accordingly. Moreover, this evaluation conducted several FGDs in El Salvador and Honduras among farmers belonging to RTMA producer organizations, finding no evidence of significantly greater household income due to Strategy interventions. Strategy Implementation Determining the Effectiveness of the Strategy. There is no question that some of the interventions were successful, well recognized, and made a significant before-after difference, as it has been properly noted in the Findings section. The question is whether those accomplishments suffice to assess the Strategy per se as a successful endeavor. In this regard, it can be concluded that those intervention-level successes were not always enough to achieve the expected results from the activities nor the Strategy. Moreover, measuring quantitatively the effectiveness of the activities and the Strategy was quite difficult due to the evaluation limitations mentioned in Section 3 which, despite the many ET efforts, eventually could not be overcome. 36 Achieving the Strategy’s Objectives. The ET did not find evidence that the top-level development objectives of the strategy have been achieved. The overarching goal of helping Central American governments eradicate hunger and poverty was not supported by tangible USAID CAM-national governments actions within the Strategy activities, or resources designated for that specific purpose. The Strategy’s stated focus on food security and nutrition got lost on decisions that made food security a subsequent objective, the indirect result of expected income improvements due to access to new or increased market opportunities for producer organizations; such decisions assumed that better nutrition would be achieved through the selection of key agriculture value chains. In other words, the major development goals (poverty, hunger, food security, and nutrition) vanished in budget, programming, and activity decisions, and implementation. Implementation Partnering. The Strategy heavily emphasizes institutional strengthening and capacity building through its interventions, especially regarding Central America regional institutions related to food security and trade. From that point of view, it is difficult to understand the impact on regional institution capacity building of USAID having shortened its agreement with UNOPS/ RUTA, while prolonging its contract with Nathan & Associates. In both cases, no mid-term evaluation of the Strategy or any major assessment of the four activities´ performance (apart from the Inspector General audit to RTMA) was undertaken. Timeline and Programming. Launched in 2011, the Strategy took more than a year to begin most of its activities, which along with other factors impacted the interventions’ timeliness. A prime example is RTMA, which closed Component 1 even before some grant-supported investments started to run, but had its contract extended and received increased funds for Component two. A more realistic timeline at the activity level would had been desirable, particularly having in mind the overcomplicated process that became RTMA’s value chain selection, or the nature of the grants program which essentially was an investment program in manufacturing facilities. Market Access for Small Farmers. The contrast between RUTA which, by design, did not include an export focus but achieved some successes in export-oriented market alliances for cocoa producers, and RTMA, which although did have such focus, eventually achieved little success, highlights a major element of both design and implementation of the Strategy: that interventions focused on market opportunities, value chains, exports, etc., must be demand-driven (not supply driven). In other words, it is of little use for farmers to have their grant-supported manufacturing capacities if they do not have buyers, or better yet, importers, to purchase their products. Market alliances are a complex, continuous relationship whose main and most important purpose is to do business. A MOU is not an objective per se (nor should be a performance indicator); it is only the beginning of a buyer-seller continuous relationship for mutual benefit. Moreover, a MOU that creates no business commitment, is merely a statement of good intentions. An intervention focused on market alliances for agriculture products must go beyond studies and documents. It must demonstrate through its lifecycle concrete work in promoting business and exports, practical experience in handling the local regulatory environment, shaping commercial connections, and deploying marketing capacities to obtain results in terms of business, and sales, not merely seller-buyer letters. It must also be able to make the Strategy’s value proposition happen: “to transform producers into exporters.” These capacities must be more focused on the “doing” than the “knowing” of business. Developing export markets, among many other things, requires a specific knowledge of non-tariff barriers to trade which came to make a vicious circle in the implementation RTMA, whose IP argued that technical barriers to trade became an obstacle for the grant-supported investments to export, while such barriers were 37 not only known in advance but, most importantly, it was expected to work on overcoming them. This is particularly relevant when grant-supported manufacturing investments are being made. A more active involvement of two key stakeholders: business people and government officers was a missed opportunity for selecting value chains to be supported. This evaluation learned that this process was more academic in nature and disregarded an active engagement with business and government in key initial decisions. Management of the Strategy. Although the four activities under the Strategy had each their own COR/AOR who, in varying degrees, have been involved in their implementation, the ET found no evidence that the USAID CAM designated through the whole Strategy lifecycle, a leading individual or team, dedicated to make sure that the Strategy was driving the activities, that activities were not only collaborating among each other, but all contributing to the Strategy’s success, and at the end the Strategy was something greater than the sum of its four projects. The ET, therefore, based on the indirect evidence it found, shares the recommendations of Office of Inspector General in its Audit Report dated on October 28, 2016 and titled: “USAID El Salvador Needs to Improve its Management of the Regional Trade and Market Alliance Project to Achieve Expected Results”, chiefly the ones related to the following issues: i) 15 percent in increased exports, ii) export-oriented market alliances, iii) project impact on farmers income and nutrition, iv) performance monitoring and data standards, v) rationale for defining intervention targets. Although those comments were directed to RTMA, this evaluation states than at different extents, they apply to other Strategy activities as well. Monitoring and Evaluation. Sound project management supports its decisions based on continuous performance monitoring and frequent evaluation. M&E systems are meant to support results-based project management. In that regard, the ET found in the Strategy, the activities and their interventions such M&E flaws such as: i) the Strategy by itself lacked a M&E plan; ii) no mid-term evaluation of the Strategy was made; iii) an audit of the largest activity became necessary; iv) several activity indicators were not results￾oriented (e.g. number of MOUs, number of policy proposals consulted); v) activities lacked a sound baseline against which to compare their performance; vi) key indicators, such as farm/family-level income, were never monitored; and vii) there were cases in which indicators varied in nature year by year, thereby making a time-series impossible to build and follow. 5.10. ON CAPACITY BUILDING Institutional Strengthening vs. Capacity Building. Although these two concepts are related and complementary, they are different in nature and use different implementation mechanisms. Whereas the former targets institutions, organizations, entities, and processes, the latter targets people and individuals. This difference must be considered in strategy/activity/intervention design. RITFS’ design placed a great emphasis on “platforms,” “frameworks,” to find that people expected to create, manage, or use them, lacked the knowledge, experience, or even interest in those tools. As all four Strategy activities came to demonstrate in different ways, quite often training, continuous advice, and technical assistance are as important or even more important than tools. Greater emphasis on human capacity building paid off better in interventions such as the support of RUTA for communities of practice which worked to: i) gather and promote engagement among different kinds of farming and other stakeholders; ii) promote generation, dissemination, and exchange of knowledge and experiences; and iii) facilitate design and implementation of development initiatives and projects. However, as noted several times in the analysis of effectiveness, when it comes to the public sector, institutional strengthening and capacity building face a challenge that was not absent to the Strategy’s 38 implementation, and that is the negative influence of politics, both national and regional, civil services flaws, bureaucratic dynamics, high staff high turnover, and budget limitations. Those factors did negatively affect activities´ implementation. The trilateral cooperation agreements with Chile are an important innovation in development cooperation, where Chilean experts with more relatable experiences and institutional environments can provide technical assistance, and an activity provides funding for these activities. The preparation and approval of policies and regulations, although rightly touted as an advance of the project, was overstated in terms of being “laws.” However, often those policies and regulations (not laws), proved to be more effective because they can avoid drawn out legislative processes for approval. Often the Strategy and activities assigned themselves laudable but too grandiose regional objectives. That is the case of formulating a regional food security policy, which was not achieved, or the coordination of “regional centers of excellence” around “solid breakthrough proposals will be validated through implementation of pilot projects with selected small producer’s groups. Activities under this component will report Global Climate Change attribution for the three pillars Sustainable Landscapes, Adaptation and Clean Energy”. 5.11. ON SUSTAINABILITY AND THE FUTURE OF THE STRATEGY The Strategy: Policy Sustainability. Determining the future of a strategy such as RITFS is beyond the scope of this evaluation; however, it’s worth noting some hypothetical elements favorable and unfavorable to its continuation (or some form of it). On the unfavorable side: 1) El Salvador is not a FTF focus country; 2) the Strategy proved to be much less effective on the food security side and more on the trade facilitation side; 3) with the “Global Food Security Act of 2016”, and the ensuing “US Government Global Food Security Strategy - 2017-2021,” there is a new policy framework taking shape at the time of this evaluation, which is an open window for uncertainty, as new FTF focus countries might be chosen. On the favorable side to a continuation of the Strategy (or some form of it), the following factors can be considered: 1) USAID/CAM´s past performance in key regional issues (such as the coffee rust crisis) proves that regional strategies are worthwhile; 2) the new Global Food Security Strategy favors activities USAID/CAM has experience with, such as: “Strengthened and expanded access to markets and trade”, “Implementing regional and multilateral agreements on technical barriers to trade, sanitary and phytosanitary (SPS) standards”, or “Opportunities for regional economic integration”, among others. How the new Global Food Security Strategy will influence a continuum of the Strategy is something to be seen. 6. RECOMMENDATIONS Project Management for Development Results. Although this is not an evaluation of the management performance of USAID/CAM, based on all the findings related to the effectiveness of the Strategy and each one of its activities, mentioned in Section 4.2 it is recommendable to emphasize that implementation of activities be more than managing a contract or an award with an IP. It should include an active process of defining realistic expected results, monitoring progress toward the achievement of expected results, learning and integrating lessons learned into management decisions, and reporting on performance. To reinforce a culture of results-based management, a set of practices are recommended: 1) holding managers accountable; 2) empowering managers; 3) focusing on bringing about change in the lives of activity beneficiaries; 4) regularly checking the effectiveness of policies and procedures; 5) establishing a culture of learning; 6) building internal capacities; and 7) sharing and communicating results. These practices are 39 possible both in large organizations and small teams. USAID/CAM might consider providing key project￾related staff with project management training such as the certification provided by the Project Management Institute. As per ADS Chapter 200 (Guiding Principles for Development Results), and 201 (Program Cycle Principles) through its activities/interventions, USAID/CAM should aim to achieve implementation performance results that move from mere outputs (activity deliverables such as number of people trained), to outcomes (short-term results of those deliverables, such as value of smallholder incremental sales generated), and eventually to impacts (medium to long-term effects produced by a project or program that change the development situation of a country such as the poverty rate). Context indicators should be monitored and considered in decision-making, by both USAID/CAM and IPs, during the whole activity lifecycle, even if they are outside the control of the activity or program, but have the potential to affect the achievement of expected results. Implementing Independent Partners. In the spirit of the comments and recommendations of the Office of the General Auditor regarding the RTMA activity, and the experience of this evaluation trying to collaborate with IPs, it is recommendable that USAID/CAM practice a relationship with IPs, especially contractors, that is based on clear and differentiated responsibilities between USAID and the partner. For the sake of accountability, it is recommendable that: i) USAID/CAM ensures that its own implementation performance can be assessed; ii) the same degree of institutional independence and accountability that exists in Cooperative Agreements (such as the USDA/PAPA), be observed in the relationship between USAID and commercial contractors chosen by it as IPs, iii) that stakeholders are continuously able to compare the performance of institutional partnerships (e.g. USAID or IFC) vs. commercial contractors and assign resources accordingly. Most importantly, USAID/CAM’s activity decisions and performance should be clearly differentiable from IP’s decisions and performance. Consistency and Coordination. An endeavor of the magnitude of a regional strategy should continuously - throughout its lifecycle, from design to closing - make sure that its development objectives drive activities and interventions and, in turn, that interventions and activities tangibly contribute to achieving such objectives. Likewise, for the sake of greater effectiveness and efficiency, USAID/CAM should ensure a permanent effort of communication, coordination, and collaboration among activities. Recommendations in this regard include: 1) make sure that the contracts/agreements that are part of a strategy must include interventions clearly linked to the success of each other and the Strategy in general; 2) make implementing COR/AORs as well as IPs’ COPs accountable for achieving Strategy-level development results; 3) promote mutual training of IPs among each other on the subject and progress of their respective activities; and 4) get feedback from stakeholders through periodic assessments, meetings, surveys, etc., and continuously update them on the progress of both activities and the Strategy. Consultation and Collaboration. The Strategy highlights USAID/CAM extensive experience in research and consultation, for instance, in preparation of the FTF Strategic Review in 2010. Based on such experience, it is highly recommended that the consultative and collaborative approach not be limited to strategies´ or programs´ preparation, but also extended to the implementation phase and, indeed to evaluation initiatives. Greater consultations with stakeholders during the whole activity lifecycle will ensure their buy-in and commitment. Consultation and collaboration should not be a one-time opportunity. Dissemination. For dissemination of science, technology and innovation, it is recommended the USAID/CAM keep its emphasis in approaching and strengthening the communities of practice as they bring together farmers, processors, companies, universities and other research institutions, government, and other programs and projects related to a value chains. 40 Monitoring, Evaluation and Learning. A greater commitment to rigorous, practical, and continuous commitment to implementation monitoring, evaluation, and learning (MEL) is recommended. USAID/CAM should adopt mid-term evaluations of activities as regular practice and, under no circumstances a performance evaluation should be conducted after a project has ended. A full compliance of USAID Evaluation Policy would allow USAID/CAM to better track progress, facilitate performance￾based and adaptive management, remain accountable to its commitments, and learn more about effective and evidence-based approaches to food security/nutrition, and/or trade facilitation. This recommendation is extended to IPs to consistently adopt MEL practices to continually assess the performance of programs, integrate lessons learned, adapt their approaches, and make course corrections based on evidence and new findings. Gender and Social Inclusion. Full compliance by USAID/CAM with USAID own policies for gender and social inclusion, is recommended: 1) a gender and social inclusion briefing and training in gender analysis as per ADS 205 should be given periodically to USAID staff and IPs; 2) an analysis of key gender inequality gaps should be conducted during initial steps of any activity of Strategy design; 3) a gender strategy must be proposed and implemented to identify the gender gaps that can enhance activity outcomes or may impact it negatively; 4) activities dealing with food value chains should introduce the practice of conducting a gender division of labor analysis to ensure the identification of the potential impact of measures to be introduced by the intervention, and of potential negative impact on specific population groups, so it may adopt contingency measures; 5) indigenous populations should be treated as a priority target group for social inclusion and projects that include them should adopt appropriate cultural and gender measures; and 6) gender empowerment and social inclusion indicators according to the nature of the intervention, must be carefully selected for monitoring and evaluation purposes. Developing True and Effective Market Alliances. An intervention focused on market alliances for agriculture products must go beyond studies and documents. USAID/CAM should make sure that an IP working on market alliances design and implement business-oriented interventions, with performance indicators focused on demand, sales, and income, not merely in more production with not secured buyers. IPs should demonstrate through activity lifecycle, concrete work in promoting business and exports, practical actions to overcoming the local regulatory environment, hands-on activity to shaping commercial and lasting commercial connections, and initiatives to bring marketing capacities to farmers and their overall value chain to obtain results in terms of sales volume and dollar amount, so that the Strategy’s value proposition does happen: “to transform producers into exporters.” These capacities must be more focused on the “doing” than the “knowing” of business. Developing export markets, among many other things, requires a specific knowledge of non-tariff barriers and, most importantly, the ability to overcome them with a value chain focus, so that grant-supported manufacturing investments have less obstacles to access external markets. Complementarily, tariff barriers, as well as opportunities coming from trade agreements, namely CAFTA, must be detected in detail and converted into business. In parallel, USAID must persist in that its partnership with USDA brings about those market information systems that were expected, and keep working together in the institutional strengthening of Ministries of Agriculture so that they increase their institutional capacities for producing quality agriculture production and market information. A more active involvement of two key stakeholders, business people and government officers, is a must; while certainly the former remain interested and engaged whenever actual business opportunities are in sight, government officials come and go, and therefore USAID/CAM and IPs must make a permanent effort in keeping key government agencies constantly involved, engaged, and convinced that their efforts, somehow, articulate and leverage government commerce and trade policies and projects. 41 The future on USAID/CAM, Trade Barriers and Food Security. Based on its findings, this evaluation recommends to USAID/CAM: 1) include food security and nutrition actively in future programming so that FTF is given a truly regional, approach in Central America, and the RDCS achieves results (or greater ones) regarding food security; 2) for reducing barriers on trade, USAID/CAM should embrace a truly regional approach beyond circumscribing activities to mere three countries; 3) learning from the experience of the IFC activity, USAID/CAM should put an emphasis in supporting collaboration and information-sharing among governments for both food security and trade facilitation purposes; 4) in the same line, USAID/CAM should intensify its efforts in supporting policy making capacities on food security-related trade policy making process within national governments, as well as regional institutions; and 5) a greater involvement of the private sector, particularly business leaders, should be pursued not only for Strategy or activities design but, more importantly, for their implementation, and M&E. 42 ANNEX 1. ANNOTATED STATEMENT OF WORK Below, are presented the two modifications to the original SOW as requested by USAID/CAM: 1. Initially SOW included Nicaragua as part of the evaluation. Later USAID/CAM decided the evaluation to be focused only in El Salvador, Guatemala and Honduras. 2. Before starting field work, USAID/CAM requested the Evaluation Team take advantage of interviews, focus group discussion and site visits to enquire for possible effects of the Strategy activities on immigration. 43 ANNEX 2. ORIGINAL SCOPE OF WORK A. Purpose of the Evaluation The purpose of this final performance evaluation (as defined in USAID's Evaluation Policy and in Automated Directives System -ADS- 201) on the USAID Regional Integrated Trade and Food Security Strategy is to determine if the design and implementation of the four activities under the Strategy were effective in achieving the Strategy’s expected results and to inform on future programming in trade and food security based on that analysis. The four activities under the Strategy are also part of the Increasing Regional Economic Integration Project18 implemented by USAID/CAM. Per ADS 201.3.5.13 “Each Mission must conduct at least one “whole-of-project” performance evaluation within their Regional Development Cooperation Strategy (RDCS) timeframe. Whole-of-project performance evaluations examine an entire project, including all its constituent activities and progress toward the achievement of the Project Purpose”. The final performance evaluation of the Regional Integrated Trade and Food Security Strategy will count as the mid-term whole-of-project performance evaluation of the Increasing Regional Economic Integration Project. In addition, the evaluation will serve to: • Provide empirical evidence to support learning and continuous improvement for USAID/CAM’s Regional Integrated Trade and Food Security Strategy • Identify the outcomes achieved by each activity toward their expected objectives, especially on such issues as capacity building and sustainability beyond USAID support • Provide information and recommendations surrounding the issue of how to address ongoing challenges to improve trade and reduce trade barriers in the region and directly or indirectly impact food security. • Provide feedback from supported producer groups on the results of activity implementation on their income, consumption and nutrition. The principal audience of this evaluation internally will be USAID/Central America and Mexico (CAM) program, particularly the Economic Growth Office, the Strategic Development Office, and the Mission Management; USAID/ Bureau for Food Security (BFS), USAID/Bureau for Latin America and the Caribbean (LAC); USAID/Guatemala and USAID/Honduras (Guatemala and Honduras are focus countries of the U.S. Government’s Global Hunger and Food Security 18 According to the ADS 201.6 Program Cycle Operational Policy, “A project is a set of complementary activities, over an established timeline and budget, intended to achieve a discrete development result, often aligned with an Intermediate Result (IR) in the RDCS Results Framework. Taken together, a Mission’s suite of project designs provides the operational plans for achieving the objectives in its strategic plan…In most cases, multiple activities are needed to ensure the synergistic contributions necessary to achieve the project’s desired results. An activity carries out an intervention, or set of interventions, typically through a contract, grant, or agreement with another U.S. Government agency or with the partner country government.” (https://www.usaid.gov/sites/default/files/documents/1870/201.pdf). 44 Initiative “Feed the Future” -FTF-19); and other USAID offices that are expected to use the evaluation to provide a retrospective analysis of activity effectiveness to achieve the results individually and combined under the Regional Integrated Trade and Food Security Strategy. Findings, conclusions, and recommendations will be used to make decisions on new directions for the remaining period of the Strategy implementation and beyond, as well as to inform future programming priorities in trade and food security under the Development Objective 1 of the RDCS, which is “Regional economic integration increased.” Finally, the evaluation results will be used for reporting purposes to stakeholders. The evaluation will be undertaken through a collaborative, utilization-focused approach. The main participants in the evaluation will be the Strategy’s prime implementing partners -U.S. Department of Agriculture (USDA); Nathan and Associates Inc.; World Bank’s International Finance Corporation (IFC); and the United Nations Office for Project Services, through the Central American Council for Agriculture’s Regional Unit for Technical Assistance (UNOPS/RUTA). In addition, the main Strategy partners should also participate, including the General Secretariat of the Central American Integration System, especially the Secretariat for Central American Economic Integration (SIECA, for its name in Spanish), the Central American Council for Agriculture (CAC, for its name in Spanish), and the Central American Economic Integration Council (COMIECO, for its name in Spanish). Finally, the Center for Tropical Agriculture Research and Education (CATIE, for its name in Spanish) and the Zamorano Pan American Agricultural School; private sector partners involved in the Strategy; national governments of the countries involved (especially the Ministries of Agriculture); other donors, such as the Chilean International Development Cooperation Agency (AGCID); Strategy beneficiaries; USAID/CAM office; and USAID bilateral offices in Guatemala and Honduras, should also play a role in the evaluation. This final performance evaluation follows a Regional Office of Inspector General’s (RIG) performance audit conducted for one of the four activities under the strategy, the Regional Trade and Market Alliance (RTMA) Project. The audit report is currently in draft form and being reviewed by USAID and the RIG. The RIG performance audit focused on determining whether USAID/El Salvador’s RTMA project was achieving its expected results related to increased market access in value chains vital to food security, trade facilitation, and institutional capacity building. The public audit report will be available on the RIG website when completed: https://oig.usaid.gov. The final performance evaluation covers implementation from the start of the Strategy in September 2011 (even though the first activity started in May 2011) through September 30, 2016. 19 See Guatemala Feed the Future Multi-year Strategy FY 2011-2015 (http://www.feedthefuture.gov/sites/default/files/country/strategies/files/GuatemalaFeedtheFutureMultiYearStrategy.pdf), and Honduras Feed the Future Multi-year Strategy FY 2011-2015 (http://www.feedthefuture.gov/sites/default/files/country/strategies/files/HondurasFeedtheFutureMultiYearStrategy_Public_2011 -11-17_FINAL.pdf) 45 B. Background Information about the Regional Integrated Trade and Food Security Strategy 2011-2017 USAID implements five activities under its Regional Integrated Trade and Food Security Strategy; four of those activities are included under this evaluation (see the following chart). Their total funding amount is $47,268,309. Activity Implementing Partner Award Number and Dates Funding Promoting Food Security and Trade Integration through Sanitary and Phytosanitary Standards (SPS) and Other Ag￾Related Capacity Building U.S. Department of Agriculture (USDA) AID-596-T-11-00001 May 4, 2011 – July 31, 2019 $10,233,171 Regional Sustainable Agriculture Program United Nations Office for Project Services (UNOPS) / Regional Unit for Technical Assistance (RUTA) AID-596-IO-00001 September 14, 2012 – September 13, 2017 $5,000,000 Regional Trade and Market Alliances Nathan and Associates, Inc. AID-596-C-13-00001 April 22, 2013 – April 21, 2018 $34,064,039 Central America Agribusiness Trade Logistics IFC AID-AEG-G-00-04- 00013. September 30, 2013 – December 30, 2016 $1,500,000 Note: A fifth activity, the “Better Coffee Harvest Project”, which is funded with Food Security funds, does not form part of this final performance evaluation because it will have its own internal evaluation. However, the evaluation team is encouraged to review the final report of the mid-term performance evaluation of the Better Coffee Harvest Project to have a more comprehensive view of the Increasing Regional Economic Integration Project. According to a December 2010 Feed the Future (FTF) Strategic Review, USAID/CAM’s strategic focus and development hypothesis in the Central American region is: if barriers to trade are reduced and more opportunities are provided to small farmers to gain greater access to regional and international markets, so as to increase their incomes, then regional food security and nutrition will improve. Under this change theory, USAID has two focus areas: 1) Regional alliances based on product distinction and quality standards, and 2) Harmonization of policies, strategies and information. The Regional Integrated Trade and Food Security Strategy, approved on February 23, 2012, encompasses USAID/CAM’s program interventions in regional food security that support the FTF Initiative. USAID/CAM supports regional efforts to improve trade and market access for critical food security agriculture value chains; promote policy analysis and formulation among 46 Central American governments and foster sustainable agriculture practices through the validation and dissemination of innovative technologies. The Strategy’s development challenge was defined as: Food Security in Central America can be supported through targeted value chains trade development; a regional framework of effective, efficient, and sustainable policies; as well as dissemination of innovative sustainable agriculture best practices. The main goal is to help Central American countries reach higher path of inclusive economic growth to eradicate hunger and poverty, and enable expansion of intraregional trade and exports.20 Stated as a development hypothesis: If Central America’s food security is supported through targeted value chain trade development; a regional framework of effective, efficient and sustainable policies; and the dissemination of innovative sustainable agriculture best practices, then the region will have more inclusive economic growth to eradicate hunger and poverty and expand intraregional trade. The Strategy has two expected results: 1) Consolidated regional value chains and export market access for food and nutrition critical products; through increased trade facilitation, harmonization and institutional capacity; and 2) Increased effectiveness of regional food security policy and programs through the increased capacity of key decision makers in the government and private sector with the use of analytical tools, (establishment of a monitoring and evaluation system) and the promotion of best practices for sustainable agriculture.21 The Strategy and its design are regional in nature and cover the Central American countries of El Salvador, Guatemala, Honduras, Costa Rica, and Nicaragua plus the Dominican Republic. Due to this regional nature, the activities have required many levels of coordination across different local, national and international actors. Regional actors in food and nutritional security, including the Central American Agriculture Council (CAC), Regional Program for Food and Nutritional Security (PRESANCA/SICA) and the Inter American Institute for Agricultural Cooperation (IICA) play an important role in implementing the Strategy. The Strategy addresses the following FTF Second Level Objectives:22  Expanding Markets and Trade  Increased investment in agriculture related activities, and  Increased employment opportunities in targeted value chains. Within USAID/CAM's Regional Development Objective of “Regional economic integration increased,” the activities under the Regional Integrated Trade and Food Security Strategy contribute to Intermediate Results (IR) 1.1 “Regional trade expanded” and 1.2 “Regional markets and investments in key sectors increased.” In addition, the Regional Integrated Trade and Food Security Strategy contributes to the achievement of Objective 1 of the U.S. Government Strategy for Engagement in Central America (CEN strategy) to “Promote prosperity and regional economic 20 Regional Integrated Trade and Food Security Strategy, 2011-2017, USAID/Central America, September 2011, p. 11. 21 Ibid. 22 Regional Integrated Trade and Food Security Strategy, 2011-2017, USAID/Central America, September 2011, p. 12. 47 integration,” and the first Line of Action in the Plan of Alliance for Prosperity in the Northern Triangle23 to “Stimulate the productive sector to create opportunities.” This final performance evaluation will serve to assess the implementation of the Strategy as a whole with all four activities contributing directly and designed to act in coordination and in a complementary manner. This evaluation is part of the Feed the Future Learning Agenda,24 the CEN strategy Learning Agenda and USAID/CAM’s Learning Agenda. General Activity Description 1. Promoting Food Security and Trade Integration through Sanitary and Phytosanitary Standards (SPS) and Other Ag-Related Capacity Building The Promoting Food Security and Trade Integration through Sanitary and Phytosanitary Standards (SPS) and Other Ag-Related Capacity Building activity is a Participating Agency Program Agreement (PAPA) between USAID/CAM and the USDA Foreign Agricultural Service’s Office of Capacity Building and Development (FAS/OCBD), implemented from May 4, 2011, to July 31, 2019, with a Total Estimated Cost of $10,233,171. The overarching goal of the agreement is to increase incomes of the rural poor and small farmers through agriculture-led market opportunities and trade facilitation by strengthening the capacity of regional governments’ agricultural institutions and private sectors with targeted technical assistance, exchanges, and training activities. The objective of the PAPA is to enhance food security and promote regional agricultural trade in El Salvador, Guatemala, Honduras, Nicaragua, Costa Rica, Panama, and the Dominican Republic. The scope of the PAPA emphasizes technical assistance and training to: a) build public and private sector sanitary and phytosanitary (SPS) capacity; b) improve access to regional and international markets, including market information; c) improve statistical systems to better inform policy makers on matters of food security, and d) address other agricultural products under the Dominican Republic-Central America-Unites States Free Trade Agreement (CAFTA-DR). Technical assistance is provided at both the regional level and the cooperating country level. At the regional level the technical assistance focuses on cross-cutting technical themes and issues common to more than one country in the region that will further regional integration and harmonization goals. In addition, issues related to the harmonization of regional regulations are to be addressed via regional consultations and workshops and together with relevant regional institutions and key stakeholders. Technical assistance provided to individual cooperating countries under the PAPA addresses country-specific technical issues, which were identified as key barriers to trade, economic growth, or food security between USDA and USAID. Assistance is provided through Government-to￾Government training and technical advice programs and includes private industry and smallholder 23 http://idbdocs.iadb.org/wsdocs/getdocument.aspx?docnum=39224238 24 Feed the Future Learning Agenda http://agrilinks.org/sites/default/files/resource/files/Feed%20the%20Future%20Learning%20Agenda.pdf 48 producer participation, as appropriate. This assistance strengthens plant health, animal health, food safety inspections services and assists the respective country governments and the private sector in improving quality and productivity to meet international standards. While much of the work is with government authorities, special emphasis is placed on enhancing the capabilities of the private sector (especially food exporters) and smaller, food-insecure producers. Regional efforts aim at improving the enabling environment for regional trade. Finally, under this activity, USAID helps CAC and other relevant regional organizations use timely and accurate information to examine key issues and recommend policy changes to improve rural economic competitiveness in the region. 2. Regional Sustainable Agriculture Program The Regional Sustainable Agriculture and Food Security Program is a Public International Organization grant to the United Nations Office for Project Services (UNOPS), implemented through the technical execution of the CAC’s Regional Unit for Technical Assistance (RUTA) from September 14, 2012, to September 13, 2017, with a Total Estimated Cost of $5,000,000. The purpose of this program is to promote inclusive economic growth, improve food security and facilitate trade in the region. USAID/CAM focuses on two specific objectives/components: a). Regional food security policy effectiveness; and b). Promotion of sustainable agricultural technologies in Central America and the Dominican Republic. The first component supports a regional agenda for food security with the political commitment and mandate of Central American governments through the CAC to strategically engage with multilateral, regional organizations and donors, to promote dialogue, accountability and partnership in policy. Specifically, the objective is to improve the effectiveness of policy implementation and the relevance of food security in the Central American Integration system’s policy agenda. Interventions focus on developing the capacity of the Central American countries’ national governments to support analysis, policy formulation, monitoring, evaluation, and learning (a monitoring, evaluation, and learning system of food and nutritional security public policy agendas and instruments that have greater impact on food availability and food prices). The expected results under this component are increased capacity of governments, producers and other civil society organizations for analysis, formulation and assessing the effects of policies on food security. Efforts concentrate at the national level for Central American governments (Guatemala, Honduras, and El Salvador). The program also supports the implementation of the food security agenda developed and consulted with governments, donors and the private sector and at the regional level within the CAC and its technical groups. Finally, the program includes the development of at least four policies, regulatory or programmatic actions to improve the regional agri-business environment. The second component seeks to develop a regional platform for research and innovation of sustainable agricultural practices. This platform will support and be a resource for farm level application and inclusive regional, national, and local level efforts, including international organizations and centers of excellence. There are two sub components: 49 a) Sustainable agriculture technologies: with the objective to increase the number of new technologies for sustainable agriculture available to small producers throughout the region; and b) Networks and communities of practice: with the objective to establish agriculture networks, partnerships, and communities of practice, including private sector participation, which will lead to the advancement of agriculture tools and information dissemination. The expected results of this component are: 1) at least six Memorandums of Understanding signed by CAC/RUTA/UNOPS with regional agriculture research centers for identification, validation, and dissemination of sustainable agriculture practices relevant to food security and applicable to small farmers; and 2) at least six new sustainable agriculture technologies adapted for small farmers, and disseminated through government extension services and relevant food security small farmers’ technical assistance programs. 3. Regional Trade and Market Alliances Project The Regional Trade and Market Alliances Project is a contract implemented by Nathan Associates, Inc. The initial base period extended from April 22, 2013, to April 21, 2016, with a Total Estimated Cost of $15,164,039. The contract was just extended to the option period beginning on April 22, 2016, and will now operate through April 21, 2018, with an additional TEC of $18.9 million. The purpose of this activity is to promote inclusive economic growth, improve food security and facilitate trade in the region. The activity has two major components: 1) market access for critical food security value chains; and 2) increased trade facilitation and improved trade and institutional capacity. The first component supports critical agriculture value chains for food security, linking small and medium producers to regional and export markets (to and from Central America, and also to other export markets). It also consolidates intra-regional value chains to leverage underutilized market opportunities. The main purposes of this component are to link producers to buyers, servicing markets in the region, as well as in the United States, and other export markets; promote compliance of quality and environmental sustainability standards; and assess investment and trade provisions with an emphasis on creating private sector alliances. The expected results of this component are: 1) at least 15 percent increase in exports sales value of participating producer groups and buyers; 2) at least 1,500 small producers units (up to 1.05 hectares associated in small producer groups, associations or cooperatives) participating in regional and export value chains; 3) at least three Memorandums of Understanding with regional or national institutions signed to support implementation of regional value chain development; 4) at least four special regional trade mechanisms implemented; and 5) at least 20 regional buyer￾seller alliances established to link small producers and small enterprises through a value-chain development approach with at least five resulting in sustainable market driven contract relationships. The second component focuses on reducing obstacles and costs of trade in the region, as well as continued support for the Central American Economic Integration Subsystem to improve trade facilitation, harmonization of trade and investment laws, regulations and procedures. The goal is to reduce the burden of costs of doing business and crossing borders, focusing on mechanisms 50 that can have a significant impact for small and medium enterprises. This component supports the implementation of and compliance with trade agreements and regional integration schemes in those areas that promote inclusive economic growth in the agricultural sector. The activity supports the implementation of USAID’s Partnership for Trade Facilitation and contributes to the adoption of best practices for trade facilitation in intra-regional customs administration offices and other administrative procedures at borders by promoting the decentralization of procedures and strengthening systems. The expected results of this component are: 1) reduce to two days the average time to trade goods across borders in the region; 2) trade facilitation monitoring system implemented and adopted by a regional private sector organization or national organizations; 3) a trade and customs specialist certification program implemented and adopted through partnership(s) with academic center(s); 4) one regional or five national Memorandums of Understanding with private sector organizations to promote the Central American Trade Facilitation Agenda; and 5) at least four policies, regulations and administrative procedures consulted among stakeholders and presented to COMIECO. 4. Central America Agribusiness Trade Logistics The Central America Agribusiness Trade Logistics Activity is an activity under the global USAID- World Bank International Finance Corporation (IFC) grant agreement to be implemented from September 30, 2013, to December 30, 2016, with a Total Estimated Cost of $1,500,000. The activity provides technical assistance/advisory services for regulatory and technological reforms to the Governments of the region, focusing on the agribusiness sectors of Guatemala, El Salvador, Honduras, and Costa Rica. Its objective is to streamline, harmonize and automate procedures for mutual recognition of sanitary registration for processed food and beverages in Costa Rica, El Salvador, Guatemala and Honduras. These procedures will include first time registry and mutual recognition of registries among Central American Customs Union Ministries of Health. The technical assistance involves process redesigning and harmonization, as well as the design, development, and support of implementation of a system to facilitate mutual recognition of sanitary registries in Central America and the improvement of national registries. The work includes process mapping, identification of unnecessary requirements and redesigning the process for sanitary registration for the first time and for mutual recognition. The activity has three components: 1) Regulatory reforms for process redesigning and harmonization. This component includes sanitary registration process redesigning and harmonization, legal analysis and a proposal for system sustainability. The latter includes an assessment of the options for the central entity managing the system and financial options for system maintenance after project end; 2) Technological reforms in Information, Communication, and Technology (ICT) systematization. This component includes activities for the design and development of an ICT system for two national registries for processed food and beverages, as well as for the automation of mutual recognition registration. The system will be phased in by modules, which will be implemented simultaneously in the four countries, in order to test them, 51 get early results and demonstrate the benefits of regional software; 3) Capacity Building. This component includes training for public sector officials and private sector users on new procedures/systems, as well as their implementation in four countries. Expected results at the end of the activity are: 1) average number of days to register a processed food product for the first time reduced from 80 days to 55 days; 2) average number of days to register a processed food product for mutual recognition reduced from 8 days to 1 day; 3) average cost to register a processed food product for the first time reduced from $400 to $300; 4) average cost to register a processed food product for mutual recognition reduced from $145 to $120; 5) number of documents required to trade for mutual recognition registration reduced from 4 to 2; 6) number of product registries mutually recognized increased from 4633 to 5096. C. Evaluation Questions The following evaluation questions, in priority order, have been identified by USAID/CAM. They should be answered by the Evaluation Team and clearly presented in the Final Report in terms of how they relate to the evaluation purpose. 1. To what extent have the activities individually and collectively led to the strengthening of local institutions? Specifically: 1.1 To what extent did the activities strengthen national governments’ and regional entities’ capacity for analysis and policy formulation related to trade facilitation and food security? 1.2 To what extent did technical assistance provided to SIECA result in improved organizational capacity to support the Central American economic integration process? 1.3 To what extent did the grants program and technical assistance build the capacity of agriculture local partners and small producer organizations? 1.4 To what extent have activities contributed to market alliances and mechanisms that will likely last beyond USAID activity implementation? 2. How has collaboration between the four activities affected Strategy and Project results? 3. To what extent have the activities contributed to the dissemination and use of science, technology and innovation for: a) Sustainable agricultural practices among small farmers, b) Trade facilitation; c) A reduction in times and costs to cross borders? 4. How did each activity promote gender integration and social inclusion approaches? 5. Activity specific questions: 5.1 (IFC) How has the Regional System of Mutual Recognition of Sanitary Registries facilitated, or not, trade of food and beverages in Central America? 5.2 (RUTA) How did RUTA’s activity change the capacity of communities of practice to perform as networks disseminating best practices? 5.3 (RTMA) To what extent have the activities under the RTMA supported producers of selected agricultural value chains to improve access to regional and export markets? 52 D. Evaluation Methodology, Data Collection, and Analysis For the USAID Regional Integrated Trade and Food Security Strategy final evaluation, a non￾experimental mixed-methods design that combines a comprehensive, rigorous analysis of existing quantitative data with customized qualitative techniques designed to elicit primary data from a wide range of counterparts, partners, beneficiaries, and stakeholders is recommended. This approach allows for triangulation of complementary data to elucidate linkages between activity inputs, outputs, and outcomes. The Evaluation Team should consider a range of possible methods and approaches for collecting and analyzing the data to address the evaluation questions thoroughly. The use of participatory methods and activities that will enhance collaboration and dialogue among counterparts, particularly partners, is required. The finalized evaluation methodology, approaches, data collection plan and analysis will be included in the Evaluation Plan submitted to USAID/CAM for revision and approval before field visits and data collection begin (see Deliverables section below). The data collection plan for this evaluation will include, at a minimum: a desk review of relevant documents; review of Strategy and activity performance monitoring data; key informant interviews and/or focus group discussions; surveys with activity beneficiaries; and direct observation through site visits. USAID/CAM expects both qualitative and quantitative data to be collected; and the results will be coded, triangulated and analyzed for content. The Evaluation Team is encouraged to propose additional/alternate data collection and analysis methods in the Evaluation Plan that they consider can yield stimulating, robust evidence in answering the evaluation questions. The analysis of the data collected is just as important as the actual collection. The Evaluation Team must triangulate data collected to have sound evidence for the findings and conclusions in the final Evaluation Report based on the data presented. All data collected in response to the evaluation questions must have as much level of disaggregation as possible, which may allow having a better level of analysis. As minimum, and per USAID Gender Equality and Female Empowerment Policy25, all data must be disaggregated and analyzed by sex, as well as analyzed for any differences between effects on men and women or male and female participation. Specific interview, survey and/or focus group protocols will be appended to the Evaluation Plan and finalized with approval from USAID/CAM; the questions should be used to answer the evaluation questions listed in this document and address the purpose of this evaluation. • Desk review of relevant documents USAID/CAM will provide the Evaluation Team with all relevant available strategy and activity specific documents, such as statements of work/program descriptions, performance reports stating the results achieved, prior assessments, etc. The Evaluation Team must review these documents and other existing literature to be provided by USAID and others in preparation for the initial team 25 USAID Gender Equality and Female Empowerment Policy (https://www.usaid.gov/sites/default/files/documents/1865/GenderEqualityPolicy_0.pdf) 53 planning meetings and before meeting with local stakeholders for interviews. The Evaluation Team is expected to review these, make their own literature research and review, and create a Review Matrix to be delivered to USAID/CAM as part of the final Evaluation Report, which indicates how reviewed documents and other methodologies used link to each evaluation question. • Review of performance indicators A Project Monitoring and Evaluation Plan is part of the Regional Integrated Trade and Food Security Strategy; additionally, each Implementing Partner prepared an Activity-specific Monitoring and Evaluation Plan that, taken together, addresses all the points in the Project M&E plan. Through the implementation of these plans, data on a number of standard and custom indicators were collected during activity implementation. USAID/CAM has used a number of performance indicators to monitor progress of the activities, as well as regular meetings with each implementing partner, site visits and other means of communications. This monitoring data collected from USAID/CAM, Annual and Quarterly reports, and the Implementing Partners will provide one source of data on progress toward objectives and outcomes. The Evaluation Team may use monitoring data on performance indicators from all the Implementing Partners as part of the evaluation analysis and should report on it in the Final Evaluation Report as much as it relates to the evaluation questions stated in this Statement of Work. • Key informant interviews, surveys, focus group discussions The Evaluation Team will interview people, through key informant interviews, group interviews, short surveys, and/or focus groups, from the institutions listed below at a minimum, as well as any from others deemed relevant to this evaluation. The Evaluation Team will include both men and women in the stakeholder consultation processes. USAID/CAM will provide the list of contact information once the evaluation task order is awarded. Preliminary list of contacts: 1. USAID/CAM (including Contracting/Agreement Officer Representatives) 2. USAID Mission offices in Guatemala, Honduras and/or Nicaragua 3. USAID Bureau of Food Security 4. USAID Bureau of Latin America and Caribbean 5. USDA (prime implementing partner of “Promoting Food Security and Trade Integration through Sanitary and Phytosanitary Standards (SPS) and Other Ag-Related Capacity Building” activity) 6. UNOPS/RUTA (prime implementing partner of “Regional Sustainable Agriculture Program”) 7. Nathan Associates, Inc. (prime implementing partner of “Regional Trade and Market Alliances Project”) 8. World Bank’s International Finance Corporation (prime implementing partner of the “Central America Agribusiness Trade Logistics” activity) 9. Development Alternatives Inc. (sub-contractor of “Regional Trade and Market Alliances Project”) 10. Trusted Trade Alliances (sub-contractor of “Regional Trade and Market Alliances Project”) 11. Zamorano Pan American Agricultural School (sub-contractor of “Regional Trade and Market Alliances Project”) 54 12. CATIE, (key partner of the “Regional Trade and Market Alliances Project” and the “Regional Sustainable Agriculture Program”) 13. Universidad Rafael Landívar (key partner of the “Regional Sustainable Agriculture Program”) 14. RTMA Local Technical Partners for implementation of Component 1: AGEXPORT, FUSADES/PROINOVA, APEN, FUNDER. 15. RTMA Grantees under Contract (a representative sample) 16. SIECA (key partner of the “Regional Sustainable Agriculture Program” and the “Promoting Food Security and Trade Integration through Sanitary and Phytosanitary Standards (SPS) and Other Ag-Related Capacity Building” activity) 17. CAC (key partner of the “Regional Sustainable Agriculture Program”, and the “Central America Agribusiness Trade Logistics” activity) 18. AGCID (key partner of the “Promoting Food Security and Trade Integration through Sanitary and Phytosanitary Standards (SPS) and Other Ag-Related Capacity Building” activity) 19. Ministries of Agriculture (key partners of the “Regional Sustainable Agriculture Program” and the “Promoting Food Security and Trade Integration through Sanitary and Phytosanitary Standards (SPS) and Other Ag-Related Capacity Building” activity) 20. Ministries of Trade and Commerce 21. Customs Administrations of El Salvador, Guatemala and Honduras 22. Trade Facilitation Commissioners and/or Coordinators 23. Ministries of Health of El Salvador, Guatemala, Honduras, and Costa Rica 24. Private companies that participated in the IFC Regional Central America Trade Logistics and Agribusiness Project pilot project for the use of the Regional Mutual Recognition system. 25. Federation of Chambers of Commerce of Central America and national partners; Export chambers of the region 26. Inter American Institute for Agricultural Cooperation (IICA) 27. Producers and exporters, especially RTMA supported producer groups 28. RUTA’s small grants beneficiary organizations (4) 29. RUTA’s communities of practice representatives (6) A sampling plan describing the selection process26 for organizations and stakeholders for key informant interviews, surveys and focus group discussions (including sex disaggregation) is expected to be included in the Evaluation Plan and Final Evaluation Report. In addition to the interviews, surveys and focus group discussions designed and implemented to answer the evaluation questions directly, the evaluation team must also design focus group protocols to conduct a series of additional focus groups with producer groups and farmers supported by the Regional Trade and Market Alliances (RTMA) Project. The focus groups will be used to identify how trade facilitation support via the RTMA contributed at the farm level to changes in income, consumption and nutrition. USAID/CAM would like to understand to what 26 Such as purposeful, random or a combination of approaches. 55 extent support from RTMA to producer groups has contributed to improved farm level production, income, consumption and nutrition in order to respond to a RIG finding in the RTMA audit. Focus group questions may include the following: • What was your participation in the RTMA project? What support did you receive? What production changes did you implement? • How has your income changed since implementing the RTMA support mechanism? • If your income has increased, what types of purchases do you make with that added income? • Have your consumption patterns changed because of that change in income? • Do you purchase different foods now? If so, what are they? • What changes have you noticed in your family/living unit since implementing the RTMA production changes? In addition, the evaluation team may use questionnaires designed by FTF in Honduras and Guatemala that would be relevant to the focus group discussions to pull tested and approved questions from those surveys. Information collected in these focus group discussions may be used to complement responses to the evaluation questions, if deemed relevant by the evaluation team. • Site visits and direct observation The Evaluation Team will arrange to visit countries in the region and in-country sites in consultation with USAID/CAM, as appropriate. Site visit selection should be based on a sampling plan developed for the Evaluation Plan and included in the Final Report. This could include purposeful sampling methods, random sampling, or some combination of approaches. Sites should include at a minimum: • Guatemala • Honduras • El Salvador • Nicaragua Other countries in the region involved in one or more of the included activities may be visited at the discretion of the evaluation team. The Evaluation Team should plan to meet with USAID Bilateral Missions (USAID/Guatemala, USAID/Honduras and USAID/Nicaragua) after being introduced by USAID/CAM once the evaluation task order is awarded. When meeting key in-country representatives, the Evaluation Team may be accompanied by a member of USAID Bilateral Missions if the Missions consider it appropriate. The Evaluation Team may be required to brief and debrief USAID Bilateral Missions on site visits, if deemed appropriate. The purpose of the briefing will be to share contacts and initial information and plans, and the debriefing will be used to share findings and receive comments or feedback prior to the preparation of the draft Evaluation Report. The Evaluation Team may attend events hosted or sponsored by the four activities during the field work period of the evaluation to conduct direct observation. The Evaluation Team can use these 56 events to talk with stakeholders, conduct interviews and collect additional data as evidence to answer the evaluation questions. USAID/CAM and the Implementing Partners will provide the Evaluation Team with a list of events once the evaluation contract is awarded. Team planning meetings An initial team-planning/ kick off meeting will be held in El Salvador between USAID/CAM and the Evaluation Team before the submission of the Evaluation Plan so that USAID can clarify any questions from the Evaluation Team, expectations, and guidelines. The expected results of this meeting are to:  Clarify each team member's role and responsibilities  Confirm the anticipated timeline and deliverables  Discuss data collection tools and methodologies by evaluation question to be presented in the Evaluation Plan  Identify communications logistics and how the Evaluation Team, USAID/CAM and the Implementing Partners will communicate with each other. Additional meetings may be held as deemed necessary by USAID/CAM and/or the Evaluation Team. E. Deliverables The evaluation team shall submit the following deliverables: 1. An Evaluation Plan, in Microsoft Word font Times new Roman, size 12, to be completed by the Evaluation Team after the Team Planning Meetings, no later than 28 calendar days after the award. USAID/CAM will receive the Evaluation Plan via electronic mail and review it to provide comments no later than five working days after receiving the document. The Evaluation Plan must include at least:  Strategy and Activity descriptions and logic (change theory/development hypothesis);  Evaluation design, and the justification of why one design or mix of designs is selected, its limitations, and how these limitations will be addressed;  A Matrix summarizing the following information per each Evaluation question: a. Methodology for data collection, sampling plan, data for each evaluation question, the justification of why one method or mix of methods is selected, its limitations and the ways to address them; b. Methodology for data analysis, how quantitative and qualitative data will be analyzed, the justification of why one analysis or mix of analyses is selected, its limitations and the ways to address them.27  Timeline and/or Milestone Plan;  Drafts of data collection protocols, such as questionnaires or focus group moderator guide(s);  Evaluation Team composition and roles; 27 Some examples of matrix formats are available in https://usaidlearninglab.org/library/evaluation-design-matrix￾templates 57  Site visit plan. For key informant interviews, surveys, and/or focus groups, the Evaluation Plan also should include the following information:28  How the interviews/surveys will help to answer the evaluation questions  Who will conduct the interviews/surveys and why they are qualified to do so  What the rationale and methods are for deciding the number, timing, and location of the interviews/surveys  How the participants will be selected and recruited  How the interviews/surveys will be recorded  How the interview/survey data will be analyzed and presented The Evaluation Plan, particularly the data collection and analysis protocols, as well as interview and focus group guides must be approved by USAID/CAM prior to start the data collection and the field work. All interview protocols must be submitted in English and Spanish. The Evaluation Team will have an additional five days to make requested changes and resubmit for final approval. Once the Evaluation Plan is approved, the Evaluation Team will submit to USAID/CAM an electronic copy of it in PDF. Any subsequent change to the Evaluation Plan must be approved by USAID/CAM. 2. Brief weekly bullet reports of activities, submitted by electronic mail due every Monday by the close of business. 3. A draft PowerPoint Out-Briefing to USAID/CAM, other USAID Missions and Implementing Partners at the end of the field work, with preliminary data and findings prior to submission of the Draft Final Report and team’s departure from the country. Comments generated during this presentation will be incorporated in the draft of the Final Report. 4. A draft of the Final Report in English in Microsoft Word, font Times New Roman size 12, submitted for review due no later than 100 calendar days after the award via electronic mail. USAID/CAM will be responsible for distributing to the Implementing Partners and other stakeholders for comments. USAID/CAM will consolidate all comments and send the draft back to the Evaluation Team within 15 working days. At a minimum, and in accordance with the USAID Evaluation Policy and ADS 201, the draft report must include the following sections:  Executive Summary;  Evaluation purpose and questions;  Strategy/Activity Background;  Evaluation design and any challenges/limitations29, with emphasis on the timeliness 28 For additional information on Focus Group Interviews, see “USAID’s Technical Note: Focus Groups Interviews” (http://usaidlearninglab.org/sites/default/files/resource/files/Focus_Group_Interviews_Tech_Note_FINAL_2013_1 119.pdf) 29 The evaluation team must identify (a) steps to mitigate any limitations, and (b) how/whether the limitations affect any particular findings, conclusions, or recommendations. 58 and methods for data collection and data analysis;  Relevant data analysis tables;  Findings and Conclusions;  Recommendations; and,  Appendices: a. Original SOW, annotated with any changes approved by USAID b. Evaluation and data collection team composition and roles, with conflict of interest disclosures for all real or perceived conflicts of interest, if necessary c. Data collection protocols and instruments d. Review matrix In addition, the protocols designed and used for the RTMA focus group discussions, and the findings and conclusions from those focus groups, must be included in the Draft Final Report as a separate Annex. Results from these focus groups may also be incorporated in the main body of the evaluation, in the findings, conclusions and recommendations, if deemed relevant to answering the evaluation questions by the evaluation team. 5. A Final Report in PDF, font Times New Roman size 12, no longer than 40 pages in its body, excluding the cover page; Table of Contents; List of Acronyms; tables, graphs, charts, or pictures taking during site visits; and/or any material deemed important and included as appendices. The approved Final Report must adhere to USAID’s Evaluation Policy and ADS 201 and have incorporated USAID's comments, as appropriate. The Evaluation Team should turn in one copy in print in each language (English and Spanish) and one copy in electronic version with both languages (DVD and electronic mail with electronic files). The Final Report will be due to USAID/CAM 10 calendar days after the Evaluation Team receives comments on the draft and no later than 154 calendar days after the award. 6. A One-Page summary of the evaluation purpose, findings, conclusions and recommendations. The One-Page summary will be prepared in English and Spanish in PDF. 7. Any raw data (qualitative or quantitative) collected in electronic form (DVD , email or flash drive, in original format of Word, Excel, etc.) is due no later than 161 calendar days after the award. As per Automated Directives System 540, the Evaluation Team must submit to the Development Data Library (DDL) at www.usaid.gov/data, in a machine￾readable, non-proprietary format, a copy of any datasets that are used (or of sufficient quality) to produce an Intellectual Work. 8. A Final Presentation with PowerPoint slides to USAID (and potentially to any other stakeholders that USAID considers relevant) as the Final Report is being finalized no later than 161 calendar days after the award. Only the Team Leader needs to be present for the Final Presentation; however, local/regional Evaluation Team members may also attend. The evaluation team will upload the final presentation to the DEC and submit an electronic copy of the final presentation to the COR of this Task Order. 59 9. Other deliverables as identified during the Team Meeting and agreed to by USAID/CAM and the Evaluation Team. All reports and papers will be considered draft versions (they must be labeled with the word “DRAFT” in watermark) until they are approved by USAID/CAM. To ensure unbiased findings, there is no guarantee that findings will be modified based on Implementing Partner suggestions. The Evaluation Team will research, investigate, and corroborate as objective any suggestion before it is incorporated in the findings, and the change will be noted in the Draft document so as to have a record of the change. All submitted reports and presentations must be a thoughtful, well-researched, well-organized document, and objectively answer the evaluation questions (what worked, what did not, and why). When writing the report, the Evaluation Team must remember the different audiences. The style of writing should be easy to understand and concise, while making sure to address the evaluation questions and issues with accurate and data-driven findings, justifiable conclusions, and practical recommendations. The Evaluation Team should clearly list any biases or limitations that exist during both data collection and analysis. In addition, all real or perceived conflicts of interest must be disclosed by each member of the evaluation team in writing. When quoting an individual in any report, the Evaluation Team must always give the context or circumstances of the quote. Correcting a grammatical error in the quote may be valid, but not rewording an entire phrase. When translating quotes from one language to another, the Evaluation Team should do so in an idiomatic way and care must be taken to ensure that the tone of the translation is equivalent to the tone of the original. Quotes should be presented in their original language in report texts. Both versions must be proofread before sending to USAID/CAM. All report must comply with the USAID Graphic Standards Manual (http://pdf.usaid.gov/pdf_docs/PNADB334.pdf) and the ADS Style and Format Guide (https://www.usaid.gov/sites/default/files/documents/1868/501mac.pdf)30. Once the Final Report has been approved by USAID/CAM, the Evaluation Team will make it compliant and submit it to the Development Experience Clearinghouse (DEC)31. The Evaluation Team will send by electronic mail to USAID/CAM the link to the DEC where the Evaluation Reports are available. USAID/CAM will assess the quality of the evaluation reports using the Evaluation Report and Review Template (http://usaidlearninglab.org/sites/default/files/resource/files/template- 30 Evaluation Team may use the Evaluation Report Template available in http://usaidlearninglab.org/library/evaluation-report-template 31 Per ADS 540, documents and development assistance projects materials produced or funded by USAID must be submitted for inclusion in the DEC: https://dec.usaid.gov/dec/content/Create.aspx?ctID=ODVhZjk4NWQtM2YyMi00YjRmLTkxNjktZTcxMjM2NDB mY2Uy 60 evaluation_report_checklist_and_review_aug2015.pdf). USAID/CAM may attach a Statement of Differences as an Annex to any the Final Report if any differences remain in the final version. F. Evaluation Team Composition To minimize costs, this performance evaluation will use a combination of multidisciplinary international, regional, and local experts. The Evaluation Team must include at a minimum: 1. Evaluation Team Leader (Key personnel) Minimum qualifications Education: Master’s degree in development or related development fields, such as Economics, Political Science, Public Administration, Business Administration, or other disciplines related to development assistance is required. Advanced degree is a plus. Formal training in monitoring and evaluation is preferred. Language Proficiency: English Level IV and Spanish Level III Work Experience: At least 10 years of relevant prior experience conducting rigorous external evaluations using both quantitative and qualitative methods for development objectives and monitoring projects and programs, preferably in Latin America. Experience in Central America is a plus. At least eight years of project management experience. Experience with management of multidisciplinary teams is a plus. Some familiarity with USAID’s objectives, approaches, operations, and policies, particularly as they relate to evaluations is a plus. Position Description: The Evaluation Team Leader will be responsible for overseeing and coordinating all activities related to this performance evaluation and for ensuring the production and completion of quality deliverables in a professional manner, in conformance with this SOW. 2. Regional Trade Specialist (Key Personnel) Minimum qualifications Education: Bachelor’s degree in Economics or a field related to development such as International Relations, Public Administration, Public Policy, Political Sciences, and other Economic Development fields. Advanced degree is preferred. Formal training in monitoring and evaluation is preferred. Language Proficiency: Spanish Level IV and English Level IV. Work Experience: At least 8 years of progressively responsible, professional-level experience in agricultural trade facilitation, value chains, development program management, and/or public policy analysis is required, preferably in Central or Latin America. Familiarity with USAID’s objectives, approaches and operations, particularly as they relate to evaluations is a plus. Position Description: The Specialist will provide guidance to the Evaluation Team Leader on regional trade integration issues. S/he should be familiar with major trade agreements and policy frameworks signed by the countries in the region. 3. Agriculture Specialist (Key Personnel) Minimum qualifications 61 Education: Bachelor’s degree in a discipline that is directly relevant to food security analysis and the analysis of staple food markets such as Agricultural Economics, Economics, Rural Development, International Trade, or other similar. Advanced degree is preferred. Formal training in monitoring and evaluation is preferred. Language Proficiency: Spanish Level IV and English Level IV. Work Experience: At least 8 years of progressively responsible, professional-level experience in agricultural business development, development program management, and/or public policy analysis is required, preferably in Central or Latin America. Familiarity with USAID’s objectives, approaches and operations, particularly as they relate to evaluation is a plus. Position Description: The Specialist will provide technical advice to the Evaluation Team Leader on agriculture and food security analysis. S/he should be familiar with the U.S. Government’s Feed the Future Presidential Initiative. 4. Logistics Coordinator (Support Personnel) Minimum qualifications Education: High school or technical degree in administration or a related development field, such as Economics, Political Science, Public Administration, Business Administration, or other related discipline related to development assistance is required. Language Proficiency: Spanish Level IV and English Level II. Work Experience: At least five years of relevant work experience. Participation in or knowledge of evaluation of development projects is a plus. Position Description: The Logistics Coordinator will be responsible for coordinating with USAID/CAM, the Implementing Partners, and the Evaluation Team to schedule meetings, transportation, and other administrative logistics. S/he will be involved in planning and may help with data collection and analysis. In addition, each team member should have, at minimum, the following skills and experience:  Understanding of the regional context  Demonstrated skill in written and oral communications  Ability to work effectively and communicate with a diverse set of senior governmental officials and professionals This Performance Evaluation is an external evaluation; therefore, the Team Leader must be someone external to USAID. Anyone who has worked directly with any of the four activities, or with the Implementing Partners in the last ten years, must not be considered as part of the Evaluation Team. Key Personnel with gender skills will be a plus. G.Logistics The Contractor will be responsible for all logistical support under this Task Order, including field office administration, all travel arrangements (with required USAID/CAM clearances), team planning facilitation and appointment scheduling, coordination with all partners and stakeholders involved in the contracted services, administrative services (computer support, printing and copying), report editing and dissemination, and for complying with provisions set forth in this SOW. 62 H.Annexes: 1. Regional Integrated Trade and Food Security Strategy 2011-2017 2. Feed the Future Learning Agenda 3. Regional Development Cooperation Strategy 2015-2020 4. Project Appraisal Document of the Increasing Regional Economic Integration Project 5. Program Description of the Promoting Food Security and Trade Integration through Sanitary and Phytosanitary Standards (SPS) and Other Ag-Related Capacity Building 6. Program Description of the Regional Sustainable Agriculture Program 7. Program Description of the Central America Agribusiness Trade Logistics activity 8. Scope of Work of the Regional Trade and Market Alliances Project 9. The most recent Work Plans for each activity 10. Monitoring and Evaluation Plan of each activity 11. Quarterly and/or Annual Performance Report of each activity 12. U.S. Government Strategy for Engagement in Central America 13. Plan of Alliance for Prosperity in the Northern Triangle 14. Disclosure of Conflict of Interest Form 63 ANNEX 3. DETAILED EVALUATION METHODOLOGY The scope of work of this evaluation asks for a wide range of evaluation questions that were expected to be clearly answered by the evaluation team, and presented in the final report in terms of how the design and implementation of the four activities under the Strategy were effective in achieving the Strategy’s expected results and to inform on future programming in trade and food security based on that analysis. In other words, it calls for a performance evaluation of the activities. Performance evaluations, as literature such as FTF M&E Guides32 suggests, are better carried out under a non-experimental design, which is meant to address a wide range of questions, both descriptive and normative, to compare a “before and after” comparison without a “rigorously defined counterfactual or a focus on causal effects and the evidence to attribute results to certain activities or programs” Therefore, this evaluation adopts a non-experimental design. It focuses on the description of activities’ actions, their performance, and whether they have made any differences resulting from before-after comparisons. As a performance evaluation, it is designed to analyze the activities to identify results, accomplishments; recognize performance issues, and constraints in their implementation and determine how the activities are perceived and valued. All of this with the purpose of present relevant findings, make conclusions and lessons to be learned, and present recommendations to informing future USAID programming on trade and food security This evaluation takes a mixed-methods design, combining a comprehensive analysis of existing quantitative data with qualitative techniques designed to capture data from a wide range of Strategy and activities’ stakeholders. This approach allows for triangulation of complementary data to determine linkages between activity inputs, outputs, and outcomes. As explained later, the evaluation considered a variety of methods for collecting and analyzing the data to address the evaluation questions thoroughly. On the qualitative side, the technique this evaluation will use is mostly content analysis. Content Analysis is a research option used to reduce large amounts of unstructured textual content into manageable data relevant to the evaluation questions. Texts refer to any way of communication, including websites, books, essays, interviews, focus groups, articles, speeches, conversations, and so on. To conduct a content analysis on any such text, the text is broken down into manageable categories on a variety of levels (ie, key words, word sense, phrase, sentence, or theme) and coded. The coded content can then be quantitatively analyzed for trends, patterns, relationships, similarities, differences etc., from which researchers can get insights and make inferences about the messages within the texts, the writer(s) and the context. 32 Feed the Future. M&E Guidance Series. Vol 10. Performance Evaluation and Feed the Future. Undated. 64 The evaluation of the Strategy combined an assessment of its four activities in a bottom-up approach (activity to Strategy), combined with a top-down (Strategy to activity) analysis of the Strategy as a whole, so that the evaluation purposes are achieved. In this evaluation, Google Forms, as it is integrated with Google docs, served a dual purpose; the first one is allowing for entering data collection during field work. The data from interviews, focus group discussions, site visits, and even document reviews, was collected through Google Forms so that a common repository of field work information was built by the evaluators and made available later for data analysis. The second use is one online surveys which was easily deployed, captured, managed, and processed. After entry, the data was transferred to a worksheet format from where it was cleaned and all logical data checks performed. Google Forms creates a database that is easy to use and makes real time monitoring, quality checks and improvement possible. Google Docs and Google Forms are free online Office applications. No special software is required; all that is needed to create documents or surveys is an Internet connection and a Google account. The evaluation was conducted in four successive stages:  Review and analysis of secondary activity information, and activity-related information  Primary information data capture (field work)  Data analysis  Evaluation reporting To choose whom among the many key informants would be interviewed, made part of a focus group discussions, sites to visit, or surveyed online, this evaluation adopted the following criteria:  Major priority was given to those stakeholders prioritized by USAID in the evaluation SOW.  The next level of priority was given to beneficiaries (direct and indirect) of activities’ activities.  Next are other stakeholders with interest in activities’ results.  Likewise, constraints of time, cost, distance were considered to define final lists of informants. Data collection methods were selected to seek the highest quality and rigor in answering the performance evaluation questions which certainly are the first step in deciding proper data collection methods. To ensure relevance of such questions, a distinction between key questions and key sub-questions was established so that they are focused in what is important for evaluation purposes. In this regard, high consideration must be given to the fact that this is the evaluation of four activities, implemented in three countries, during a period of five years, therefore the number of questions is necessarily high. In other words, the first step in ensuring that data collection methods are appropriate is making sure that the right key evaluation questions are defined and presented in interviews, focus group discussions, or surveys. 65 Document Review The evaluation team met the COPs of all four activities and asked to be provided with the following documents: 1. Activity SOW 2. Activity contract/agreement with USAID 3. Initial and annual work plans 4. M & E plans 5. Budget and financial plan 6. Stakeholders list with detailed contact information 7. Activity geographic intervention area and sites to visit 8. Logical Framework + Performance Indicators 9. Activity performance reports 10. Activity deliverables (documents) 11. Working documents (training plans, policy proposals, systematization of experiences, methodological guides, etc. 12. Activity gender equality goals/performance/results 13. USAID reviews of the activity if any 14. Activity evaluations or audits if any 15. External publications related to the activity These documents, along with others found during field work, were reviewed by the Evaluation Team on continued basis, and entered in a Google forms database to be used for data analysis and evaluation report writing. The complete list of 189 documents reviewed is presented as Annex 6. Performance Indicators Review Both the Strategy and the four activities have Monitoring & Evaluation Plans including key performance indicators expected to be monitored throughout implementation period. The Evaluation Team used monitoring data on performance indicators from all the Implementing Partners as part of the evaluation analysis as much as it relates to the evaluation questions, as well as to the evaluation criteria with emphasis on effectiveness of meeting objectives and bringing about activity outcomes. Key Informant Interviews (KII) Using the sampling approach explained above, this evaluation conducted 127 interviews of 192 interviewees (who are listed in Annex 6) based on the key evaluation questions and sub-questions, presented later in this Annex. Likewise, a KII protocol is presented in Annex 8. Focus Group Discussions (FGD) Focus group discussions typically offer greater descriptive depth than, for instance, surveys and a broader base of perspectives than often is obtained through individual interviews. The criteria used to choose these focus groups included: i) a specific request by the evaluation SOW in the case of RTMA, ii) representativeness of the product or value chain, iii) priority has been given to focus groups to be applied mainly to small farmers and their organizations, and iv) geographic proximity and logistical convenience has been considered. 66 The Evaluation Team convened nine focus group discussions related to the RUTA or RTMA activities in the three countries, with a total of 88 participants, including 33 women.  RUTA, National Coffee Working Group, Santa Tecla, El Salvador  RTMA, plantain packing and dried fruit processing, San Carlos Cooperative Association (ACOPASCA), El Paisnal, Aguilares, El Salvador (SOW requirement)  RUTA, cocoa producers assisted by FundaSistemas, Coban, Guatemala  RUTA, cocoa producers and processors, assisted by Fundación Pro-Peten, Guatemala  RTMA, honey producers and processors, COPIASURO, San Pedro La Laguna, Guatemala  RUTA, Women in Cocoa in Honduras Network (REDMUCH), in La Lima, Honduras  RUTA, food security policy course participants, Universidad Nacional de Honduras, Tegucigalpa, Honduras  RTMA, potato packing, Empresa Campesina Agroindustrial de la Reforma Agraria de Intibucá ECARAI, La Esperanza, Intibucá, Honduras  RTMA, bean packing, ARSAGRO, Danlí, Honduras (SOW requirement)  RTMA, Lepaterique, Honduras Site Visits Site visits are meant to allow for the capture of primary information by direct observation. Preference for site visits has been given to farmers’ agriculture-related processes or organizations. In some cases, site visits coincide with focus group discussions and/or with key informant interviews of same stakeholders. Priority has been given to key value chains and the possibility to compare processes among countries. All five of the RTMA focus group discussions and the RUTA FGD in ProPetén were complemented by site visits to the packing and/or processing facilities of the respective organizations. The additional visited as part of this evaluation were:  RUTA, Ecological Fraternity Coffee Cooperative (CAFEL), Copan Ruinas, Honduras  RUTA, Cooperativa Association of Cocoa Agricultural Production (ACPACI), Los Izalcos, El Salvador  RTMA, border crossing facilities at El Amatillo, between El Salvador and Honduras Online Survey One online survey was used in this evaluation to complement individual interviews with higher level perspectives, while looking for more accurate answers than can arise from the isolation that comes from use a personal computer. Since online surveys rely on computer access and savvy, they target more educated people. The evaluation of RITFS used Google Forms for the hosting of the survey. 67 ANNEX 4. EVALUATION MATRICES EVALUATION MATRIX FOR THE SCOPE OF WORK EVALUATION QUESTIONS SOW Questions Indicator Data Source Data Collection Method Data Analysis 1. To what extent have the activities individually and collectively led to the strengthening of local institutions? Ministries of Agriculture Ministries of Commerce CAC-SIECA RITFS Activities Document review KII FGD On line survey Data synthesis, Descriptive statistics, Cross tabulation, Triangulation 1.1. To what extent did the activities strengthen national governments’ and regional entities’ capacity for analysis and policy formulation related to trade facilitation and food security? Ministries of Agriculture CAC RITFS Activities Document review KII FGD On line survey Data synthesis, Triangulation 1.2. To what extent did technical assistance provided to SIECA result in improved organizational capacity to support the Central American economic integration process? Ministries of Commerce SIECA RITFS activities Document review KII Data synthesis, Triangulation 1.3. To what extent did the grants program and technical assistance build the capacity of agriculture local partners and small producer organizations? Ministries of Agriculture RITFS Activities Small farmers and their organizations Document review KII FGD Data synthesis, Descriptive statistics, Cross tabulation, Triangulation 1.4. To what extent have activities contributed to market alliances and mechanisms that will likely last beyond USAID activity implementation? Document review KII Data synthesis, Descriptive statistics, Cross tabulation, Triangulation 68 SOW Questions Indicator Data Source Data Collection Method Data Analysis 2. How has collaboration between the four activities affected Strategy and Activity results? USAID CAM USDA RITFS activities Document review KII | 3. To what extent have the activities contributed to the dissemination and use of science, technology and innovation? Ministries of Agriculture Ministries of Commerce CAC-SIECA RITFS Activities Document review KII Data synthesis, Descriptive statistics, Cross tabulation, Triangulation 3.1. To what extent have the activities contributed to the dissemination and use of science, technology and innovation for sustainable agricultural practices among small farmers? RITFS activities Small farmer organizations Agroprocessors Extension services Document review KII FGD Online survey 3.2. To what extent have the activities contributed to the dissemination and use of science, technology and innovation for trade facilitation? Ministries of Commerce Customs services Sanitary registries SIECA RITFS activities Document review KII 3.3. To what extent have the activities contributed to the dissemination and use of science, technology and innovation for reduction in times and cross borders? Ministries of Commerce Customs services SIECA RITFS activities Exporters Document review KII 4. How did each activity promote gender integration and social inclusion approaches? RITFS activities Beneficiaries and their organizations Document review KII FGD Data synthesis, Descriptive statistics, Cross tabulation, Triangulation 5. Activity specific questions: Data synthesis, 69 SOW Questions Indicator Data Source Data Collection Method Data Analysis 5.1.(IFC) How has the Regional System of Mutual Recognition of Sanitary Registries facilitated, or not, trade of food and beverages in Central America? Time and cost of registration IFC Sanitary registries AMCHAM Exporters Document review KII Descriptive statistics, Cross tabulation, Triangulation 5.2.(RUTA) How did RUTA’s activity change the capacity of communities of practice to perform as networks disseminating best practices? Implementing partners Small farmers and their organizations Document review KII Descriptive statistics, Cross tabulation, Triangulation 5.3.(RTMA) To what extent have the activities under the RTMA supported producers of selected agricultural value chains to improve access to regional and export markets? Implementing partners Small farmers and their organizations Document review KII Descriptive statistics, Cross tabulation, Triangulation 70 Regional Integrated Trade and Food Security Strategy - RITFS OVERALL EVALUATION MATRIX FOR THE STRATEGY Question Sub-question Indicator Data Source Data Collection Method Data Analysis EVALUATION CRITERIA: RELEVANCE KQ 1. Has the Strategy responded to your needs, expectations, or policies? Document review KII FGD On line survey Data synthesis, Descriptive statistics, Cross tabulation, Triangulation KSQ 1.1 To what extent is the Strategy important for USAID policies? Ministries of Agriculture Ministries of Commerce CAC-SIECA Other RITFS Activities Document review KII KSQ 1.2 To what extent is the Strategy important for your country’s government? Ministries of Agriculture Ministries of Commerce CAC-SIECA Other RITFS Activities Document review KII 71 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 1.3 Has USG assistance enhanced food security as expected? Ministries of Agriculture Ministries of Commerce CAC-SIECA Other RITFS Activities Small farmers Document review KII FGD KSQ 1.4 Has USG assistance promoted regional trade as expected? Ministries of Agriculture Ministries of Commerce CAC-SIECA Other RITFS Activities Small farmers Document review KII FGD KSQ 1.5 Has USG assistance promoted capacity building as expected? Ministries of Agriculture Ministries of Commerce National Statistics Offices CAC-SIECA officers Other RITFS Activities Document review KII FGD EVALUACION CRITERIA: EFFECTIVENESS KQ 2. In terms of food security and/or sustainable agriculture, what results has RITFS achieved so far? Document review KII FGD On line survey Data synthesis, Descriptive statistics, Cross tabulation, Triangulation 72 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 2.1 How has the RITFS contributed to strengthening value chains that are critical for food security? Ministries of Agriculture CAC-SIECA officers Other RITFS Activities Small farmers´ associations Document review KII FGD KSQ 2.3 How has the Strategy linked small farmers´ groups to buyers? Ministries of Agriculture CAC-SIECA officers Small farmers´ associations Document review KII FGD KSQ 2.4 How has the Strategy promoted alliances to build buyer-producer networks? Ministries of Agriculture CAC-SIECA officers Small farmers´ associations Document review KII FGD KSQ 2.5 How has the Strategy contributed to eradicate hunger and poverty? Ministries of Agriculture CAC-SIECA officers Small farmers´ associations Document review KII FGD KSQ 2.6 How has the Strategy contributed to increase public and private investment in agriculture? Ministries of Agriculture CAC-SIECA officers Small farmers´ associations Private sector Document review KII FGD KSQ 2.7 How has the Strategy contributed to increase public and private investment in nutrition? Ministries of Agriculture CAC-SIECA officers Document review KII 73 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 2.8 How has the Strategy contributed to increase on and off-farm jobs in agriculture value chains? Ministries of Agriculture CAC-SIECA officers Document review KII KQ 3. In terms of regional and international trade, what results does the RITFS have so far? Document review KII FGD On line survey Data synthesis, Descriptive statistics, Cross tabulation, Triangulation KSQ 3.1 How has the Strategy increased export market access for nutritionally critical foods? Ministries of Commerce Ministries of Agriculture CAC-SIECA officers Document review KII KSQ 3.2 How has the Strategy contributed to expand markets and trade? Ministries of Commerce Ministries of Agriculture CAC-SIECA officers Exporters’ associations Document review KII KSQ 3.3 What percentage change in the value of intra￾regional trade in targeted agricultural commodities can be attributed to the RITFS? Ministries of Commerce Ministries of Agriculture CAC-SIECA officers Exporters’ associations Document review KII 74 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 3.4 How has the RITFS increased trade facilitation? Ministries of Commerce Ministries of Agriculture CAC-SIECA officers Exporters’ associations Document review KII KSQ 3.5 How has the Strategy increased harmonization of trade regulations? Ministries of Commerce Ministries of Agriculture Customs CAC-SIECA officers Exporters’ associations Document review KII KSQ 3.6 How has the RITFS contributed to reduce the average number of days required to trade goods across borders? Ministries of Commerce Ministries of Agriculture Customs CAC-SIECA officers Exporters’ associations Document review KII KSQ 3.7 How has the Strategy contributed to a better trade enabling environment? Ministries of Commerce Ministries of Agriculture Customs CAC-SIECA officers Exporters’ associations Document review KII 75 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KQ 4. In terms of capacity building, what results does the RITFS have so far? Document review KII FGD On line survey Data synthesis, Descriptive statistics, Cross tabulation, Triangulation KSQ 4.1 How has the RITFS improved institutional capacity in the region? Ministries of Commerce Ministries of Agriculture Customs CAC-SIECA officers Document review KII KSQ 4.2 How has the RITFS strengthened regional trade institutions? Ministries of Commerce Ministries of Agriculture Customs CAC-SIECA officers Exporters associations Document review KII KSQ 4.3 How has the RITFS strengthened food security￾related institutions? Ministries of Commerce Ministries of Agriculture CAC-SIECA officers Document review KII KSQ 4.4 How has the RITFS improved food-security policy making? Ministries of Commerce Ministries of Agriculture CAC-SIECA officers Document review KII 76 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 4.5 How has the RITFS improved trade policy making? Ministries of Commerce Ministries of Agriculture CAC-SIECA officers Document review KII KSQ 4.6 What public-private partnerships have been formed due to RITFS assistance? Ministries of Commerce Ministries of Agriculture CAC-SIECA officers Exporters associations Farmers associations Buyers Document review KII KSQ 4.7 What technologies or management practices have been adopted due to USG assistance? Ministries of Commerce Ministries of Agriculture Customs CAC-SIECA officers Exporters associations Document review KII KSQ 4.8 What policies/regulations/admin￾istrative procedures have been drafted and presented for public/stakeholders consultation? Ministries of Commerce Ministries of Agriculture CAC-SIECA officers Document review KII 77 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 4.9 Has the RITFS helped the Central American Council of Agriculture -CAC- to establish a framework for food security policies’ dialogue, analysis, and peer review? Ministries of Commerce Ministries of Agriculture CAC-SIECA officers Document review KII KSQ 4.10 Has the RITFS sought to establish public-private partnerships for market access purposes with agro-industrial business, agro-exporters, brokers, and importers? Ministries of Commerce Ministries of Agriculture CAC-SIECA officers Document review KII KSQ 4.11 Has the RITFS sought to establish public-private partnerships with technology firms to improve customs and other trade-related administrative procedures? Ministries of Commerce Ministries of Agriculture CAC-SIECA officers Customs Business associations Document review KII KSQ 4.12 Has RITFS sought to establish public-private partnerships with centers of excellence to promote a platform to disseminate best practices for sustainable agriculture? Ministries of Commerce Ministries of Agriculture CAC-SIECA officers Academic sector Document review KII KQS 4.13 Has the RITFS complemented the region’s different levels of intervention which are strategically related to FTF objectives? Ministries of Commerce Ministries of Agriculture CAC-SIECA officers Document review KII 78 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KQS 4.14 How are SICA’s PRESANCA and ECADERT involved in RITFS programs? Ministries of Commerce Ministries of Agriculture CAC-SIECA officers PRESANCA officers ECADERT officers Document review KII KQS 4.15 How has the RITFS coordinated its programs with programs of other donors Ministries of Commerce Ministries of Agriculture CAC-SIECA officers Document review KII KQS 4.16 How has RITFS interacted with the relevant private sector and producer associations in the region? Small farmers associations Exporters associations Document review KII FGD KQS 4.17 Were the RITFS priorities met? (First greater investment in building market access for value chains critical to food security, then harmonization and capacity building of the regional SPS system, and then, the capacity to implement and manage the regional framework through SIECA) USAID officers Implementing partners SIECA officers Document review KII 79 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KQ 5. In terms of collaboration between the RITFS activities, what results does the Strategy have so far? Document review KII FGD On line survey Data synthesis, Descriptive statistics, Cross tabulation, Triangulation KSQ 5.1 Could you mention examples of collaboration among RITFS programs? Ministries of Agriculture Ministries of Commerce USAID AORs/COR Implementing partners Document review KII KSQ 5.2 In terms of RITFS activities’ collaboration, what missed or potential opportunities are perceived? Ministries of Agriculture Ministries of Commerce USAID AORs/COR Implementing partners Document review KII KSA 5.3 KQ 6. What are the RITFS, as a strategy, major strengths? Ministries of Agriculture Ministries of Commerce USAID AORs/COR Implementing partners Document review KII 80 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KQ 7. What are the RITFS, as a strategy, major weaknesses? Ministries of Agriculture Ministries of Commerce USAID AORs/COR Implementing partners Document review KII EVALUATION CRITERIA: EFFICIENCY KQ 8. Has the RITFS managed its resources efficiently? Document review KII FGD On line survey Data synthesis, Descriptive statistics, Cross tabulation, Triangulation KSQ 8.1 Have the RITFS activities costs been as expected? USAID AORs/COR Implementing partners Document review KII KSQ 8.2 Have the RITFS activities performed per schedule? USAID AORs/COR Implementing partners Document review KII KSQ 8.3 Were the RITFS activities carried out within budget? USAID AORs/COR Implementing partners Document review KII KSQ 8.4 Have the RITFS activities made good use of human resources? USAID AORs/COR Implementing partners Document review KII KSQ 8.5 Does the activity have regional/by country work plans? USAID AORs/COR Implementing partners Document review KII KSQ 8.6 Does the RITFS have a Performance Monitoring Plan? USAID AORs/COR Implementing partners Document review KII 81 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 8.7 Have the RITFS’ indicators been monitored? USAID AORs/COR Implementing partners Document review KII KSQ 8.8 Were quarterly and annual execution reports presented? USAID AORs/COR Implementing partners Document review KII KSQ 8.9 Did the RITFS commission a mid-term external evaluation? USAID AORs/COR Implementing partners Document review KII EVALUATION CRITERIA: SUSTAINABILITY KQ 9. How will advances developed through RITFS activities continue after USG support ends? Document review KII FGD On line survey Data synthesis, Descriptive statistics, Cross tabulation, Triangulation KSQ 9.1 How will improvement on food security continue after the RITFS ends? USAID AORs/COR Implementing partners Document review KII KSQ 9.2 How will strengthening of regional trade continue after the RITFS ends? USAID AORs/COR Implementing partners Document review KII KSQ 9.3 How will capacity building continue after the RITFS ends? USAID AORs/COR Implementing partners Document review KII CROSS CUTTING ISSUES EVALUATION KQ 10. Does the RITFS have a gender strategy? Document review KII FGD Data synthesis, Descriptive statistics, Cross tabulation, Triangulation 82 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 10.1 Did the RITFS design include gender oriented considerations, information, indicators, and deliverables? USAID AORs/COR Implementing partners Document review KII KSQ 10.2 Were gender considerations included in the RITFS M&E Plan? USAID AORs/ COR Implementing partners Document review KII KSQ 10.3 Does the activity include any gender-oriented deliverables? USAID AORs/COR Implementing partners Document review KII KSQ 10.4 What gender best practices or lessons learned have derived from the RITFS? USAID AORs/COR Implementing partners Document review KII KQ 11. How has the activity promoted social inclusion? Document review KII FGD Data synthesis, Descriptive statistics, Cross tabulation, Triangulation KSQ 11.1 How has the RITFS encouraged the involvement of youth in its activities? USAID AORs/COR Implementing partners Document review KII KSQ 11.2 How did the RITFS encourage the involvement of different ethnicities? USAID AORs/COR Implementing partners Document review KII KSQ 11.3 Are the RITFS indicators disaggregated by income level/age/race, if relevant? USAID AORs/COR Implementing partners Document review KII 83 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 11.4 Are there any good practices / lessons learned related to social inclusion derived from the RITFS experience? USAID AORs/CORs Implementing partners Document review KII KQ 12. Does the activity include any environmental sustainability considerations? Document review KII FGD Data synthesis, Descriptive statistics, Cross tabulation, Triangulation KSQ 12.1 Does the RITFS promote environmental sustainability? How? Implementing partners USAID AORs/CORs Environment Ministries Document review KII KSQ 12.2 Does the RITFS promote environmental sustainability as part of capacity building? How? Implementing partners USAID AORs/COR Ministries of Agriculture Ministries of Environment Document review KII KSQ 12.3 Are there any good practices related to environmental sustainability and sustainable agriculture derived from the RITFS experience? Implementing partners USAID AORs/COR Document review KII KQ 13. Has the activity had an impact on the number of young people and families leaving or returning to the area? Small farmers Small scale FGD 84 USAID-USDA PARTICIPATING AGENCY PROGRAM AGREEMENT-PAPA EVALUATION MATRIX Question Sub-question Indicator Data Source Data Collection Method Data Analysis EVALUATION CRITERIA: RELEVANCE KQ 1. Has the Activity provided assistance that responds to your needs, expectations, and/or policies? Document review KII FGD Data synthesis, Descriptive statistics, Crosstabulation, Triangulation KSQ 1.1 Has USAID assistance enhanced food security as you expected? Ministries of Agriculture Ministries of Commerce CAC-SIECA Other RITFS Activities Small farmers Document review KII FGD KSQ 1.2 Has USAID assistance promoted regional trade as you expected? Ministries of Agriculture Ministries of Commerce CAC-SIECA Other RITFS Activities Small farmers Document review KII FGD 85 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 1.3 Has USAID assistance promoted capacity building as you expected? Ministries of Agriculture Ministries of Commerce Other RITFS Activities CAC-SIECA officers Other RITFS Activities Document review KII EVALUATION CRITERIA: EFFECTIVENESS Question Sub-question Indicator Data Source Data Collection Data Analysis KQ2. In terms of food security/sustainable agriculture, what results has the activity achieved so far? Document review KII FGD Data synthesis, Descriptive statistics, Crosstabulation, Triangulation KSQ 2.1 Has USAID assistance contributed to increase small farmers’ income? How? Small farmers Ministries of Agriculture Ministries of Commerce USDA officers SIECA officers Document review KII KSQ 2.2 Has USAID assistance strengthened plant health, animal health, and food safety inspection services? How? Ministries of Commerce Ministries of Agriculture USDA officers Document review KII 86 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 2.3 How many individuals and institutions have received USG supported short-term training in agricultural sector productivity or food security? Small farmers KSQ 2.4 How has USAID assisted your country’s private sector in improving quality and productivity to meet international standards? Ministries of Commerce Ministries of Agriculture USDA officers Small farmers Food Industry Small farmer’s associations Document review KII FGD KQ3. In terms of regional trade, what results has the activity achieved so far? Document review KII FGD Data synthesis, Descriptive statistics, Crosstabulation, Triangulation KSQ 3.1 Did PAPA do a gap assessment on the status of regional harmonization and integration processes in respective member countries? Ministries of Commerce Ministries of Agriculture USDA officers Document review KII 87 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 3.2 Has USAID assistance contributed to increased trade within the CA region? How? Ministers of Commerce Ministers of Agriculture USDA officers Exporters’ associations Document review KII KSQ 3.3 How much of the value of intra￾regional exports of targeted agricultural commodities can be attributed to USG assistance? Small farmers’ associations Exporters associations USDA officers Ministries of Agriculture Ministries of Commerce Document review KII FGD KQ4. In terms of capacity building, what results has the activity achieved so far? Document review KII FGD Data synthesis, Descriptive statistics, Crosstabulation, Triangulation KSQ 4.1 What workshops, seminars, and conferences have been carried out as part of the program? USDA officers Document review KII 88 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 4.2 What scientific exchange programs have been carried out as part of the program? USDA officers Ministries of Agriculture Ministries of Commerce CAC-SIECA officers Technical institutions Document review KII KSQ 4.3 How did the activity strengthen national governments’ capacity for analysis and policy formulation related to trade facilitation and food security? USDA officers Ministries of Agriculture Ministries of Commerce CAC SIECA Document review KII KSQ 4.4 How did the activity strengthen regional entities’ capacity for analysis and policy formulation related to trade facilitation and food security? USDA officers Ministries of Agriculture Ministries of Commerce CAC-SIECA officers Document review KII KSQ 4.5 What policies, regulations, and administrative procedures have been analyzed through USG assistance? USDA officers Ministries of Agriculture Ministries of Commerce SIECA officers Document review KII 89 Question Sub -question Indicator Data Source Data Collection Method Data Analysis KSQ 4.6 What policy reforms, regulations, and administrative procedures have been presented / passed for legislation/decree due to USG assistance? USDA officers Small farmer’s organizations Document review KII KSQ 4.7 What institutions or organizations have undertaken capacity/competency strengthening as a result of USG assistance? USDA officers Ministries of Agriculture Ministries of Commerce SIECA Technical institutions Document review KII KSQ 4.8 How has the institutional capacity of relevant national statistical offices improved to better collect high -quality agricultural data? USDA officers Ministries of Agriculture Ministries of Commerce Technical institutions Document review KII KSQ 4.9 How has USG assistance provided to SIECA resulted in improved organizational capacity to support the Central American economic integration process? USDA officers Ministries of Agriculture Ministries of Commerce Technical institutions Document review KII 90 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 4.10 How has USAID helped the Central American Agricultural Council (Consejo Agropecuario Centroamericano, CAC) use agricultural information for policy-making and rural economic competitiveness improvement in your country/ region? USDA officers Ministries of Agriculture Ministries of Commerce National Statistics Offices SIECA officers CAC officers Technical institutions Document review KII KQ5. Has the activity promoted institutional collaboration among RITFS implementing partners and with USAID Honduras and Guatemala? Document review KII FGD Data synthesis, Descriptive statistics, Crosstabulation, Triangulation KSQ 5.1 How has the PAPA collaborated with UNOPS/RUTA’s Regional Sustainable Agriculture Program? USDA officers Government Officers UNOPS/RUTA Officers Document review KII 91 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 5.2 How has the PAPA collaborated with Nathan Associates’ Regional Trade and Market Alliances (RTMA) Activity? USDA Government Officers Nathan Associate Officers Document review KII KSQ 5.3 How has the PAPA collaborated with CATL’s Central America Agribusiness Trade Logistics Program? USDA Government Officers CATL Officers Document review KII KSQ 5.4 How has the activity collaborated with USAID missions in Honduras and Guatemala? USDA officers Government Officers USAID Honduras and Guatemala Document review KII KQ 6. What are the RITFS, as a strategy, major strengths? USDA officers Ministries of Agriculture Ministries of Commerce SIECA officers CAC officers Technical institutions Document review KII 92 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KQ 7. What are the RITFS, as a strategy, major weaknesses? USDA officers Ministries of Agriculture Ministries of Commerce SIECA officers CAC officers Technical institutions Document review KII EVALUATION CRITERIA: EFFICIENCY KQ8. Did the activity use its resources efficiently? Document review KII FGD Data synthesis, Descriptive statistics, Crosstabulation, Triangulation KSQ 8.1 Have activities’ activities cost as expected? USDA officers USAID AOR Document review KII KSQ 8.2 Have activities been performed on schedule? USDA officers USAID AOR Document review KII KSQ 8.3 Were activity activities carried out within budget? USDA officers USAID AOR Document review KII KSQ 8.4 Has the activity made good use of human resources? USDA officers USAID AOR Document review KII KSQ 8.5 Does the activity have regional/by country work plans? USDA officers USAID AOR Document review KII 93 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 8.6 Does the activity have a Performance Monitoring Plan? USDA officers USAID AOR Document review KII KSQ 8.7 Have activity indicators been monitored? USDA officers USAID AOR Document review KII KSQ 8.8 Were quarterly and annual execution reports presented by PAPA? USDA officers USAID AOR Document review KII KSQ 8.9 Did PAPA commission a mid￾term evaluation? USDA officers USAID AOR Document review KII EVALUATION CRITERIA: SUSTAINABILITY KQ 9. How will activity- supported activities continue after USG support ends? Document review KII FGD Data synthesis, Descriptive statistics, Crosstabulation, Triangulation KSQ 9.1 How will technical assistance on food security continue after the PAPA ends? USDA officers USAID AOR Ministries of Agriculture Ministries of Commerce Document review KII FGD Data synthesis, Descriptive statistics, Crosstabulation, Triangulation KSQ 9.2 How will technical assistance on regional trade continue beyond the PAPA? USDA officers USAID AOR Ministries of Agriculture Ministries of Commerce Document review KII 94 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 9.3 How will technical assistance in capacity building continue after the PAPA ends? USDA officers USAID AOR Ministries of Agriculture Ministries of Commerce Document review KII CROSS CUTTING ISSUES EVALUATION KQ 10. Does the activity have a gender strategy? Document review KII FGD Data synthesis, Descriptive statistics, Crosstabulation, Triangulation KSQ 10.1 Did the PAPA’s SOW include gender oriented considerations, information, indicators, and deliverables? USDA officers USAID AOR Document review KII KSQ 10.2 Were gender considerations included in the PAPA’s M&E Plan? USDA officers USAID AOR Document review KII KSQ 10.3 Does the PAPA include any gender-oriented deliverables? USDA officers USAID AOR Document review KII KSQ 10.4 What gender good practices or lessons learned were found in the PAPA? USDA officers USAID AOR Document review KII 95 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KQ 11. How has the activity promoted social inclusion? Document review KII FGD Data synthesis, Descriptive statistics, Crosstabulation, Triangulation KSQ 11.1 How has the activity encouraged the involvement of youth in its activities? USDA officers USAID AOR Document review KII KSQ 11.2 How has the activity encouraged the involvement of different ethnicities? USDA officers USAID AOR Document review KII KSQ 11.3 Are activity indicators disaggregated by income level/age/race, if relevant? USDA officers USAID AOR Document review KII KSQ 11.4 Are there any good practices / lessons learned related to social inclusion that could be useful to share beyond the activity context? USDA officers USAID AOR Document review KII 96 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KQ 12. Does the activity include any environmental sustainability considerations? Document review KII FGD Data synthesis, Descriptive statistics, Crosstabulation, Triangulation KSQ 12.1 Does the PAPA assistance promote environmental sustainability? How? USDA officers USAID AOR Document review KII KSQ 12.2 Are there any good practices related to environmental sustainability and sustainable agriculture that could be useful to share beyond the activity context? USDA officers USAID AOR Document review KII KQ 13. Has the activity had an impact on the number of young people and families leaving or returning to the area? Small farmers Small scale farmers FGD 97 Regional Food Security Policy Effectiveness and Sustainable Agriculture Program – RUTA USAID- UNOPS/RUTA Evaluation Matrix Question Sub-question Indicator Data Source Data Collection Method Data Analysis EVALUATION CRITERIA: RELEVANCE KQ1. Has the activity provided assistance that responds to your needs, expectations or policies? Document review KII FGD Data synthesis, Descriptive statistics, Crosstabulation, Triangulation KSQ 1.1 Has the activity supported food security as expected? Activity documents IP Ministries of Agriculture CAC SICA Research Institutes / Universities Small farmers (individually and organized) Donors NGOs Document review KII FGD KSQ 1.2 Has the activity promoted regional trade as expected? Activity documents IP Ministries of Agriculture CAC SIECA Small farmers Organizations Donors NGOs Document review KII 98 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 1.3 Has the activity promoted capacity building as expected? Activity documents IP Ministries of Agriculture CAC SICA Research Institutes / Universities Communities of Practices Document review KII EVALUATION CRITERIA: EFFECTIVENESS KQ 2. In terms of food security/sustainable agriculture, what results has the activity achieved so far? KSQ 2.1 What were the most tangible impacts of the Policy Effectiveness Program on food security in the region? Activity documents UNOPS/RUTA CAC Ministries of Agriculture SICA Document review KII KSQ 2.2 How was the implementation of the Central American Food Security policy agenda affected by the program? Activity documents IP CAC Ministries of Agriculture Document review KII 99 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 2.3 Did the program establish an inventory of research and development activities for climate resilient and environmentally sustainable practices, reduction of post￾harvest losses and improvement of energy efficiency along agriculture value chains? At least 4 studies done on public expenditure in food and nutritional security; subsidy and protection analysis; and markets per year; and at least 2 studies and research per year arising from the analysis of the research agenda. Activity documents IP Research Institutes / Universities Document review KII KSQ 2.4 How did the activity establish coordination with/among regional centers of excellence for agriculture research? Which ones? A regional platform for research and innovation of sustainable agricultural practices created Activity documents IP CAC Regional Agricultural Research Centers Document review KII KSQ 2.5 How did the activity contribute to establishing agriculture networks, partnerships, and “communities of practice”, including the private sector, for the advancement of agriculture tools and information dissemination? Activity documents IP CAC Ministries of Agriculture Research institutes Communities of Practice Extension Services NGOs Small farmers organizations Document review KII 100 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 2.6 How did the activity contribute to adaptation of sustainable agriculture best practices and technologies? Activity documents IP Ministries of Agriculture Research institutes Extension Services NGOs Small farmers Document review KII FGD KSQ 2.7 How did the activity contribute to systematization and communication of best practices and technologies for sustainable agriculture? KSQ 2.8 What training programs did the activity implement? KSQ 2.9 How many and what memorandums of understanding among agriculture research centers on sustainable agriculture were established? At least 6 Memorandums of Understanding signed by CAC/RUTA/UNOPS with regional agriculture research centers. Activity documents IP CAC Regional Agricultural Centers Document review KII 101 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 2.10 How many and what agriculture sustainable technologies were adopted by small farmers due to the activity? How were they disseminated? At least six new sustainable agriculture technologies identified and adopted by small farmers Activity documents Small Farmers and their organizations Extension Services Communities of Practice NGOs Document review KII FGD KQ3. In terms of regional trade, what results has the activity achieved so far? KSQ 3.1 Has the activity contributed to the increase of regional trade of agricultural products key for food and nutrition security? How? Activity documents IP CAC SIECA Document review KII KQ4. In terms of policy effectiveness on food security (capacity building), what results has the activity achieved so far? Document review KII FGD Data synthesis, Descriptive statistics, Crosstabulation, Triangulation KSQ 4.1 What results did the activity have in the identification of the state of the art in public policies on food and nutritional security? Activity documents IP Ministries of Agriculture CAC Document review KII 102 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 4.2 What is the status of the database for registration of public policies on food and nutritional security? Activity documents IP Ministries of Agriculture CAC Document review KII KSQ 4.3 What is the status of the proposed regional policy analysis model? Activity documents IP Ministries of Agriculture CAC Document review KII KSQ 4.4 How effective was the technical assistance and training on the qualitative policy analysis model at the national level and capacity building for the analysis? Activity documents IP Ministries of Agriculture CAC Document review KII KSQ 4.5 How did the activity perform regarding the establishment of the Regional Monitoring and Evaluation System of policies that affect food and nutritional security? A unified, participatory regional monitoring, evaluation and learning system established. Activity documents IP Ministries of Agriculture CAC PRESISAN Document review KII 103 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 4.6 What is the research and studies agenda on food and nutritional security, and how it has been developed? Activity documents IP Ministries of Agriculture CAC Document review KII KSQ 4.7 What kind of support has the activity provided in the formulation of policies, programmatic or regulatory actions that affect food and nutritional security? IP CAC SICA PRESANCA/PRESISAN Document review KII KSQ 4.8 Due to program support, how many policies, regulatory or programmatic actions to improve the regional agribusiness environment, were analyzed, drafted, consulted and adopted by the CAC? At least 4 policies, regulatory or programmatic actions to improve the regional agri-business environment developed. Activity documents IP Ministries of Agriculture CAC Document review KII 104 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 4.9 What interventions (policy and regulatory reform, institutional strengthening, market development, sustainable agriculture platform) in this activity have contributed to improve the enabling environment for agriculture businesses? Activity documents IP SIECA CAC Ministries of Agriculture Ministry of Economics Document review KII KQ5. Has the activity promoted institutional collaboration among RITFS implementing partners and with USAID Honduras and Guatemala? Document review KII FGD Data synthesis, Descriptive statistics, Crosstabulation, Triangulation KSQ 5.1 How has the activity collaborated with the USAID/USDA PAPA program? Activity documents UNOPS/RUTA Officers Government Officers USDA officers Document review KII KSQ 5.2 How has activity collaborated with USAID-CATL Central America Regional Agribusiness Trade Logistics Activity? Activity documents UNOPS/RUTA Officers Government Officers RUTA Officers Document review KII 105 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 5.3 How has activity collaborated with USAID-Nathan Associates’ Regional Trade and Market Alliances (RTMA) Activity? Activity documents UNOPS/RUTA Officers Government Officers RUTA Officers Document review KII KSQ 5.4 How has the activity collaborated with USAID missions in Honduras and Guatemala? USDA officers Government Officers USAID Honduras and Guatemala Document review KII KQ 6. What are the RITFS, as a strategy, major strengths? Activity documents IP SIECA CAC Ministries of Agriculture Ministry of Economics Document review KII KQ 7. What are the RITFS, as a strategy, major weaknesses? Activity documents IP SIECA CAC Ministries of Agriculture Ministry of Economics Document review KII EVALUATION CRITERIA: EFFICIENCY KQ 8. Did the activity use its resources efficiently? Document review KII FGD Data synthesis, Descriptive statistics, Crosstabulation, Triangulation 106 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 8.1 Have the activities’ activities cost as expected? Activity documents IP USAID AOR Document review KII KSQ 8.2 Have activity’s activities performed on schedule? Activity documents IP USAID AOR Document review KII KSQ 8.3 Were activity activities carried out within budget? Activity documents IP USAID AOR Document review KII KSQ 8.4 Has the activity made good use of human resources? Activity documents IP USAID AOR Document review KII KSQ 8.5 Does the activity have regional/by country work plans? Activity documents IP USAID AOR Document review KII KSQ 8.6 Does the activity have a Performance Monitoring Plan? Activity documents IP USAID AOR Document review KII KSQ 8.7 Have activity indicators been monitored? Activity documents IP USAID AOR Document review KII KSQ 8.8 Has the activity presented quarterly and annual execution reports? Activity documents IP USAID AOR Document review KII KSQ 8.9 Did the activity commission a mid-term evaluation? Activity documents IP USAID AOR Document review KII 107 Question Sub-question Indicator Data Source Data Collection Method Data Analysis EVALUATION CRITERIA: SUSTAINABILITY KQ 9. How will activity- supported activities continue after USG support ends? Document review KII FGD Data synthesis, Descriptive statistics, Crosstabulation, Triangulation KSQ 9.1 How will assistance in policy making effectiveness continue after the activity ends? Activity documents IP USAID AOR CAC Ministries of Agriculture Document review KII FGD Data synthesis, Descriptive statistics, Crosstabulation, Triangulation KSQ 9.2 How will assistance in developing sustainable agriculture in key value chains for food security continue after the activity ends? Activity documents IP USAID AOR CAC Ministries of Agriculture Research Institutes Document review KII KSQ 9.3 How will technical assistance in capacity building continue after the activity ends? Activity documents IP USAID AOR CAC Ministries of Agriculture Document review KII CROSS CUTTING ISSUES EVALUATION KQ10. Does the activity have a gender strategy? Document review KII FGD Data synthesis, Descriptive statistics, Crosstabulation, Triangulation 108 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 10.1 Did the activity SOW include gender oriented considerations, information, indicators, and deliverables? Activity documents IP USAID AOR Document review KII KSQ 10.2 Were gender considerations included in the activity’s M&E Plan? Activity documents IP USAID AOR Document review KII KSQ 10.3 Does the activity include any gender-oriented deliverables? Activity documents IP USAID AOR Document review KII KSQ 10.4 What gender good practices can be found or lessons learned be derived from the RUTA activity? Activity documents IP USAID AOR Document review KII KQ 11. How has the activity promoted social inclusion? Document review KII FGD Data synthesis, Descriptive statistics, Crosstabulation, Triangulation KSQ 11.1 How has the activity encouraged the involvement of youth in its activities? Activity documents IP USAID AOR Document review KII 109 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 11.2 How has the activity encouraged the involvement of different ethnicities? Activity documents IP USAID AOR Document review KII KSQ 11.3 Are activity indicators disaggregated by income level/age/race, if relevant? Activity documents Document review KSQ 11.4 Are there any good practices / lessons learned related to social inclusion that could be useful to share beyond the activity context? Activity documents IP USAID AOR Document review KII KQ 12. Does the activity include any environmental sustainability considerations? Document review KII FGD Data synthesis, Descriptive statistics, Crosstabulation, Triangulation KSQ 12.1 Does the activity assistance promote regional trade environmental sustainability? How? Activity documents IP USAID AOR Ministries of Agriculture Ministries of Commerce Ministries of Environment Document review KII 110 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 12.2 Does the activity assistance promote environmental sustainability as part of capacity building? How? Activity documents IP USAID CORs Ministries of Agriculture Ministries of Environment Document review KII KSQ 12.3 Are there any good practices related to environmental sustainability and sustainable agriculture that could be useful to share beyond the activity context? Activity documents IP USAID AOR Ministries of Agriculture Ministries of Environment Document review KII KSQ 12.4 How did the activity influence the dissemination of new technologies for sustainable agriculture in the region and the availability of these technologies to small farmers? Activity documents IP USAID AOR Ministries of Agriculture Research Institutes Small Farmers Document review KII FGD KQ 13. Has the activity had an impact on the number of young people and families leaving or returning to the area? Small farmers Small scale FGD 111 RTMA Evaluation Matrix Question Sub-question Indicator Data Source Data Collection Method Data Analysis EVALUATION CRITERIA: RELEVANCE KQ 1. Has the activity provided assistance that responds to your needs, expectations or policies? Data Synthesis Descriptive statistics Cross tabulation Triangulations KSQ 1.1 Are the value chains selected by the activity the most important for food security? Why? Document review Nathan Associates officers USAID COR Farmer associations Ministries of Agriculture Small Farmer Document review KII KSQ 1.2 Has the activity promoted regional trade as expected? How? Document review Farmer associations Ministries of Agriculture Small Farmer Document KII FGD KSQ 1.3 Has the activity promoted capacity building as expected? How? Document Nathan Associates officers Farmer associations Small Farmer USAID COR Document KII FGD EVALUATION CRITERIA: EFFECTIVENESS Question Sub-question Indicator Data Source Data Collection Data Analysis KQ2. In terms of food security/sustainable agriculture, what results has the activity achieved so far? Data Synthesis Descriptive statistics Cross tabulation Triangulations 112 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 2.1 Did the activity map and analyze value chains, to examine how they improve income of small farmers and poor households? Document Nathan Associates officers USAID COR Document review KII KSQ 2.2 Was the activity successful in identifying the value chains critical for rural households and small farmers´ food security? Document Nathan Associates officers USAID COR Document review KII KSQ 2.3 Did the selected value chains have an impact on the development of infrastructure and investment? KSQ 2.4 How many and what national value chains have been supported and consolidated by the activity? Document Nathan Associates officers USAID COR Document review KII KSQ 2.5 How did the grants program perform? What lessons can be learned from it? Did it identify any best practices? KSQ 2.6 How did the activity contribute to the increase of the production and income of small farmer groups? Nathan Associates officers Ministries of Agriculture Farmer associations Small Farmers Document review KII FGD KSQ 2.7 How did the activity contribute to improvement (quantity and quality) in the families´ patterns of consumption and nutrition? Farmer associations Small Farmers Document review KII FGD 113 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KQ3. In terms of regional trade, what results has the activity achieved so far? Data Synthesis Descriptive statistics Cross tabulation Triangulations KSQ 3.1 What national markets have the selected value chains targeted and developed? How? Nathan Associates officers Ministries of Economy, Farmer associations RTMA Grantees Trusted Trade Alliances, Development Alternatives Inc. Ministry of Agriculture Document review KII FGD KSQ 3.2 What international markets have the selected value chains targeted and developed? How? Nathan Associates officers Ministry of Economy, Farmer associations RTMA Grantees under Contract (a representative sample) Trusted Trade Alliances, Development Alternatives Ministry of Agriculture Document review KII FGD KSQ 3.3 How did the activity support a greater regional accumulation for CA countries to take advantage of trade under US quotas? Minister of Commerce Exports Promotion Institution Farmer associations AMCHAM Selected exporters RTMA Grantees Document review KII FGD 114 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 3.4 How did the activity support the application of CAFTA regulations to intra￾regional (Central American) trade? Minister of Commerce Exports Promotion Institution Farmer associations Central American Chamber of Commerce Selected exporters RTMA Grantees Document review KII FGD KSQ 3.5 How has the activity linked buyers to small farmers to improve trade by promoting improved environmental compliance and sustainability standards? Nathan Associates officers USAID COR Farmer associations Document review KII FGD KSQ 3.6 Has USAID documented the best practices and lessons learned to capitalize on the experiences of value chains? Nathan USAID COR KSQ 3.7 How many and what kinds of alliances for market access has the activity created and developed? Minister of Commerce Minister of Agriculture Exports Promotion Institution Farmer associations AGEXPORT FUSADES FUNDER Selected exporters RTMA Grantees 115 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 3.8 How has the activity contributed to market alliances and mechanisms that will likely last beyond USAID activity implementation? Minister of Commerce Minister Agriculture Exports Promotion Institution Farmer associations AMCHAM Exporters RTMA Grantees Document review KII FGD KSQ 3.9 How has the activity developed innovative models to establish regional buyer-seller networks that can be systematically replicated? Nathan Associates officers Minister of Economy Documents review KII KSQ 3.10 How many regional buyer-seller alliances have been established with the support of the activity to link small farmers and small businesses through a value-chain development approach, with at least five resulting in sustainable market driven contract relationships? (target: at least 20) Minister of Economy Minister of Agriculture Farmer associations Document review KII KSQ 3.11 Which “anchor companies” have been incorporated in the Alliance and value chains? Nathan Associates officers Ministers of Economy and Agriculture Farmer associations Small Farmers Document review KII 116 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 3.12 Does the activity implement cost-benefit analysis models to promote transparency along the value chain to ensure the incremental benefits and improved quality are transferred to small producers? Documents Minister of Agriculture Farmer associations) Small Farmers Document review KII FGD KSQ 3.13 How many special regional trade mechanisms were implemented with the support of the activity? For example, Advanced Rulings, Authorized economic operators, Port and Border Uniform PI appeals mechanisms, etc. (at least 4) Nathan Associates officers Customs officers Minister of Commerce Documents review KII Site visit to border customs KSQ 3.14 How has the activity facilitated export of products to the Central American Common Market and to the United States? Nathan Associates officers Exports Promotion Institution Farmer associations AMCHAM KSQ 3.15 Does a trade facilitation monitoring system exist that was supported by the activity? How effective is this tool to track the commitments of governments to trade facilitation? Nathan Associates officers Exports Promotion Institution Minister of Commerce Exporter Chambers Document review KII KSQ 3.16 Is this monitoring system administered by a regional private sector organization or national organizations? Nathan Associates officers USAID COR Private entity in charge of the activity Document review KII 117 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 3.17 How much has the time and cost needed to trade goods across borders in the region been reduced? Time reduced to 2 days Nathan Associates officers Customs, Minister of Commerce CATRANSCA FECATRANS Brokers Documents review KII Site visit to border customs KSQ 3.18 How many and what kind of regional or national Memorandums of Understanding with private sector organizations to promote the Central American Trade Facilitation Agenda has been signed? (Target: one regional or five national MoU). National Chambers of exporters AGEXPORT FUSADES FUNDER Documents review KII KSQ 3.19 To what extent have the activity activities contributed to the dissemination and use of science, technology, and innovation for trade facilitation? National Chambers of exporters AGEXPORT FUSADES FUNDER Documents review KII KQ4. In terms of capacity building and institutional strengthening, what results has the activity achieved so far? Data Synthesis Descriptive statistics Cross tabulation Triangulations KSQ 4.1 How has the activity supported the capacity building for customs officials? Nathan Associates officers Customs officers Documents review KII 118 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 4.2 Which academic center(s) have been identified to implement the Customs Specialist Certification Program? Nathan Associates officers Customs officers Academic centers Documents review KII KSQ 4.3 Have courses and their materials been officially recognized by the respective education authorities? Nathan Associates officers Customs officers Academic centers Documents review KII KSQ 4.4 How many customs specialists obtained the Certification of Trade Professional? Not less than 300 Customs Officials certified Nathan Associates officers Customs officers Academic centers Documents review KII Focus Group KSQ 4.5 How did the technical assistance provided by the activity improve the organizational capacity of SIECA to support the Central American economic integration process? Nathan Associates officers Academic centers SIECA COMIECO Documents review KII KSQ 4.6 How were the results of the Central American Trade Facilitation Leadership Development Program coordinated with SIECA? At least 6 individuals graduated Nathan Associates officers SIECA Documents review KII 119 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 4.7 How were the training programs coordinated with other donors, such as World Bank, World Customs Organization, and Inter￾American Development Bank (IADB)? Nathan Associates officers World Bank World Customs Organization IADB Documents review KII KSQ 4.8 How were training activities monitored? What information is available about the programs, guides, and materials developed? How are these materials available to other people and institutions? Nathan Associates officers SIECA Documents review KII KSQ 4.9 How many policies, regulations and administrative procedures have been consulted among stakeholders and presented to COMIECO? (target: at least 4 COMIECO SIECA Nathan Associates officers Documents review KII KQ 5. Has the activity promoted institutional collaboration among the other implementing partners? Data Synthesis Descriptive statistics Cross tabulation Triangulations KSQ 5.1 How has the activity collaborated with the USAID/USDA PAPA program? Activity COP USAID COR KII Document review 120 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 5.2 How has the Activity collaborated with UNOPS/RUTA’s Regional Sustainable Agriculture Program? Activity COP USAID COR KII Document review KSQ 5.3 How has the activity collaborated with CATL’s Central America Agribusiness Trade Logistics Program? Activity COP USAID COR KII Document review KSQ 5.4 How has the activity collaborated with USAID missions in Honduras and Guatemala? Activity COP USAID COR KII Document review KQ 6. What are the RITFS, as a strategy, major strengths? Activity documents IP SIECA Ministry of Economics Document review KII KQ 7. What are the RITFS, as a strategy, major weaknesses? Activity documents IP SIECA Ministry of Economics Document review KII EVALUATION CRITERIA: EFFICIENCY KQ8. Did the activity use its resources efficiently? Document Review Data Synthesis Descriptive statistics Cross tabulation Triangulations KSQ 8.1 Have the activity´s activities cost as expected? COP USAID COR Document review KII KSQ 8.2 Have activity’s activities performed on schedule? COP USAID COR Document review KII 121 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 8.3 Were activity activities carried out within budget? COP USAID COR Document review KII KSQ 8.4 Has the activity made good use of human resources? COP USAID COR Document review KII KSQ 8.5 Does the activity have regional/by country work plans? USDA officers USAID COR Document review KII KSQ 8.6 Does the activity have a Performance Monitoring Plan? USDA officers USAID COR Document review KII KSQ 8.7 Have activity indicators been monitored? USDA officers USAID COR Document review KII KSQ 8.8 Has the activity presented quarterly and annual execution reports? USDA officers USAID COR Document review KII KSQ 8.9 Did the activity commission a mid-term evaluation? USDA officers USAID COR Document review KII EVALUATION CRITERIA: SUSTAINABILITY KQ 9. How will activity- supported activities continue after USG support ends? Data Synthesis Descriptive statistics Cross tabulation Triangulations KSQ 9.1 How will market increase continue for value chains and alliance after the activity ends? Nathan SIECA Ministries Agriculture, Economy, Commerce Customs Farmer associations of selected products Documents review KII 122 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 9.2 How will trade facilitation continue after the activity ends? SIECA Ministries Agriculture, Economy, Commerce Customs Farmer associations of selected products Documents review KII KSQ 9.3 How will the monitoring system adopted by regional or national organizations continue beyond activity end? Nathan SIECA Private sector responsible for monitoring Ministry of Economy Documents review KII KSQ 9.4 How will the value chain-related buyer-seller alliances continue after the activity ends? IP Farmer associations of selected products (TBD) Partners of the Alliance Documents review KII CROSS CUTTING ISSUES EVALUATION KQ 10. Does the RTMA have a gender strategy? Document USAID officers Nathan Documents review KII Data Synthesis Descriptive statistics Cross tabulation Triangulations KSQ 10.1 Did the activity SOW include gender oriented considerations, information, indicators, and deliverables? Document Nathan Associates officers USAID COR Documents review KII KSQ 10.2 Were gender considerations included in the activity’s M&E Plan? Document Nathan Associates officers USAID COR Documents review KII KSQ 10.3 Does the activity include any gender-oriented deliverables? Document Nathan Associates officers USAID COR Documents review KII 123 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 10.4 What gender good practices or lessons learned were found in this activity? Document Nathan Associates officers USAID COR Documents review KII KQ 11. How has the activity promoted social inclusion? Data Synthesis Descriptive statistics Cross tabulation Triangulations KSQ 11.1 How has the activity encouraged the involvement of youth in its activities? Document Nathan Associates officers USAID COR Documents review KII KSQ 11.2 How has the activity encouraged the involvement of different ethnicities? Document Nathan USAID COR Documents review KII KSQ 11.3 Are activity indicators disaggregated by income level/age/race, if relevant? Document Nathan USAID COR Documents review KII KSQ 11.4 Are there any good practices / lessons learned related to social inclusion that could be useful to share beyond the activity context? KQ 12. Does the activity include any environmental sustainability considerations? Data Synthesis Descriptive statistics Cross tabulation Triangulations KSQ 12.1 Does the activity promote environmental sustainability? How? USAID COR Ministries of Agriculture Ministries of Commerce Documents review KII 124 Question Sub-question Indicator Data Source Data Collection Method Data Analysis KSQ 12.2 Are there any good practices related to environmental sustainability and sustainable agriculture that could be useful to share beyond the activity context? USAID COR Ministries of Agriculture Ministries of Commerce Documents review KII KQ 13. Has the activity had an impact on the number of young people and families leaving or returning to the area? Small farmers FGD 125 Central America Agribusiness Trade Logistics - IFC Evaluation Matrix Question Sub-question Indicator Data source Data collection Method Data analysis EVALUATION CRITERIA: RELEVANCE KQ 1 Has the activity provided assistance that responds to your needs, expectations or policies? -Document review -KII -Online surveys (NGOs, academicians, private companies) -Site visit Data synthesis Descriptive statistics Cross-tabulation Triangulation KSQ 1.1 Has USAID assistance enhanced food security as you expected? IFC Officers USAID AOR Document review KII KSQ 1.2 Has USAID assistance promoted regional trade as you expected? IFC Officers USAID AOR Ministries of Trade Document review KII KSQ 1.3 Has USAID assistance promoted capacity building as you expected? IFC Officers USAID AOR Document review KII 126 Question Sub-question Indicator Data source Data collection Method Data analysis EVALUATION CRITERIA: EFFECTIVENESS KQ2. In terms of food security/sustainable agriculture, what results has the Activity achieved so far? IFC Officers USAID AOR Document review KII KQ3. In terms of Regional Trade, what results has the Activity achieved so far? IFC Officers USAID AOR Ministries of Commerce Document review KII KSQ 3.1 Did the activity update the baseline information to update target values for both outcomes and impact? Baseline information updated within 3 months of activity execution. Baseline information again updated within one year of activity execution. IFC Officers USAID AOR Document review KII KSQ 3.2 Has the activity contributed to streamlining procedures for sanitary registration for processed food and beverages in Costa Rica, El Salvador, Guatemala and Honduras? IFC Officers Ministries of health Exporters Document review KII 127 Question Sub-question Indicator Data source Data collection Method Data analysis KSQ 3.3 Has the activity contributed to the harmonization of procedures for sanitary registration for processed food and beverages in Costa Rica, El Salvador, Guatemala and Honduras? IFC Officers Ministries of health Exporters Document review KII KSQ 3.4 Has the activity contributed to the automation of procedures for sanitary registration for processed food and beverages in Costa Rica, El Salvador, Guatemala and Honduras? IFC Officers Ministries of health Exporters Document review KII KSQ 3.5 Has the average number of days to register a processed food product for the first time been reduced? How much? The average number of days to register a processed food product for the first time reduced from 80 days to 55 days on average. IFC Officers Ministries of health Exporters Document review KII KSQ 3.6 Has the average number of days to register a processed food product for mutual recognition been reduced? How much? Average number of days to register a processed food product for mutual recognition reduced from 8 days to 1 day. IFC Officers Ministries of health Exporters Document review KII 128 Question Sub-question Indicator Data source Data collection Method Data analysis KSQ 3.7 Has the average cost to register a processed food product for the first time been reduced? How much? Average number of days to register a processed food product for mutual recognition reduced from $400 to $300. IFC Officers USAID AOR Ministries of health Exporters Document review KII KSQ 3.8 Has the average cost to register a processed food product for mutual recognition been reduced? How much? Average cost to register a processed food product for mutual recognition reduced from $145 to $120. IFC Officers USAID AOR Ministries of health Exporters Document review KII KSQ 3.9 Has the number of documents required for mutual recognition registration been reduced? How much? Number of documents required for mutual recognition of registration reduced from 4 to 2. IFC Officers USAID AOR Ministries of health Exporters Document review KII KSQ 3.10 Has the number of product registries mutually recognized increased? How much? Number of product registries mutually recognized increased from 4633 to 5096 (and to 6784 after three years of activity end). IFC Officers USAID AOR Ministries of health Exporters Document review KII KSQ 3.11 Did the activity meet its impact estimations in terms of cost savings? Compliance Cost Savings for first registration: $3,055,544) Private Sector Savings for mutual recognition: $7,589,050) IFC Officers USAID AOR Ministries of health Exporters Document review KII 129 Question Sub-question Indicator Data source Data collection Method Data analysis KSQ 3.12 Are there any lessons learned from the activities’ implementation? IFC Officers USAID AOR Ministries of health Exporters Document review KII KSQ 3.13. Are there any best practices adopted due to the activities? IFC Officers USAID AOR Ministries of health Exporters Document review KII KQ4. In terms of capacity building, what results has the Activity achieved so far? IFC Officers USAID AOR Ministries of health Exporters Document review KII -Data synthesis -Descriptive statistics -Cross￾tabulation -Triangulation KSQ 4.1 How and to what extent have the national processes for sanitary registration been improved? Number of policies formulated (regulatory and technological reforms) by national governments and regional entities related to Agribusiness Trade Logistics, including automation of procedures, and sanitary records of processed foods and beverages Trade Facilitation Commissioners (H, ES, G) IFC officers -Private companies Chambers of Commerce. Document review KII Data synthesis and tabulation, Triangulation of tabulated results 130 Question Sub-question Indicator Data source Data collection Method Data analysis KSQ 4.2 What is the state of the regional system designed and developed to facilitate mutual recognition of sanitary registries in CA? An ICT system for two national registries for processed food and beverages designed and developed (for Guatemala and Honduras). An ICT system for the automation of mutual recognition registration designed and developed (for three countries) IFC Officers USAID AOR Ministries of health Exporters Document review KII KSQ 4.3 Was a communication strategy for the activity developed and how effective has it been? Strategy of communication developed IFC Officers USAID AOR Document review KII KSQ 4.4 How many public-sector officials and private sector users have been trained on new procedures/system as well as their implementation? Number of public sector officials and private sector users, properly trained and that have received technical assistance IFC Officers USAID AOR Ministries of health Document review KII 131 Question Sub-question Indicator Data source Data collection Method Data analysis KQ5. Has the activity promoted institutional collaboration among RITFS implementing partners and with USAID Honduras and Guatemala? Document review KII -Data synthesis -Descriptive statistics -Cross￾tabulation -Triangulation KSQ 5.1 How has the IFC activity collaborated with UNOPS/RUTA’s Regional Sustainable Agriculture Program? IFC/RUTA officers Government Officers UNOPS/RUTA Officers Document review KII KSQ 5.2 How has the IFC activity collaborated with RTMA? IFC/RTMA officers Government Officers Document review KII KSQ 5.3 How has the IFC activity collaborated with PAPA? IFC/PAPA officers Government Officers Document review KII KSQ 5.4 How has the activity collaborated with USAID missions in Honduras and Guatemala? Activity COP USAID Honduras and Guatemala KII Document review 132 Question Sub-question Indicator Data source Data collection Method Data analysis KSQ 5.5 How did the activity contribute to the promotion / diffusion of best practices related to trade facilitation and integration, in order to consolidate regional value chains and export market access and to increase the effectiveness of regional food security policy and programs? Stakeholder opinions Reporting of techniques learned -Customs Administrations -Private Companies Chambers of Commerce -Customs Administrations -Academicians (National Agricultural School; Agronomy research center of the El Salvador national University; Agronomy research center of Zamorano Pan American Agricultural School; Agronomy research center of Rafael Landívar University; Agronomy research center of Agrarian National University￾Document review KII Synthesis and tabulation of data, triangulation KQ 6. What are the RITFS, as a strategy, major strengths? Activity documents IP SIECA Ministries of Economics Document review KII 133 Question Sub-question Indicator Data source Data collection Method Data analysis KQ 7. What are the RITFS, as a strategy, major weaknesses? Activity documents IP SIECA Ministries of Economics Document review KII EVALUATION CRITERIA: EFFICIENCY KQ8. Did the activity use its resources efficiently? - Document review KII -Data synthesis -Descriptive statistics -Cross￾tabulation -Triangulation KSQ 8.1 Have activity activities costs been as expected? Cost analysis by budget line item; comparison with possible alternatives IFC officers USAID AOR Document review KII Cost analysis KSQ 8.2 Have Activity activities been performed on schedule? IFC officers USAID AOR KSQ 8.3 Were activity activities carried out within budget? IFC officers USAID AOR Document review KII KSQ 8.4 Has the Activity made good use of human resources? IFC officers USAID AOR Document review KII KSQ 8.5 Does the Activity have regional/by country work plans? IFC officers USAID AOR Document review KII 134 Question Sub-question Indicator Data source Data collection Method Data analysis KSQ 8.6 Does the Activity have a Performance Monitoring Plan? USAID AOR Document review KII KSQ 8.7 Have activity indicators been monitored? IFC officers USAID AOR Document review KII KSQ 8.8 Has the Activity presented quarterly and annual execution reports? IFC officers USAID AOR Document review KII KSQ 8.9 Did the Activity commission a mid-term evaluation IFC officers USAID AOR Document review KII EVALUATION CRITERIA: SUSTAINABILITY KQ 9. How will Activity supported activities continue after CATL support concludes? Document review KII -Data synthesis -Descriptive statistics -Cross￾tabulation -Triangulation KSQ 9.1 Are the national governments committed to maintain and improve the quality of the system to facilitate mutual recognition of sanitary registries in Central America? Number of ministries using its planning and budgeting. -IFC -Private companies Chambers of Commerce Document review KII Tabulation, frequency of responses, data synthesis 135 Question Sub-question Indicator Data source Data collection Method Data analysis KSQ 9.2 How will technical assistance on automation of procedures and sanitary registers of processed food/beverages continue after the Activity ends? USDA Officers USAID CORs Ministries Agriculture Ministries Commerce Document review KII KSQ 9.3 How will technical assistance to facilitate mutual recognition of sanitary registries in Central America continue beyond the Activity? USDA Officers USAID CORs Ministries Agriculture Ministries Commerce Document review KII KSQ 9.4 How will technical assistance in capacity building continue after the Activity ends? USDA Officers USAID CORs Ministries Agriculture Ministries Commerce Document review KII KSQ 9.5 Are the organizational and management structures at national and regional levels suitable for the continuation of the activity? Structure implemented in local governments to serve beneficiaries -Ministry of Trade and Commerce (H, ES, G) - Private companies Chambers of CommerceUSAID/C AM Document review KII Data synthesis and tabulation, triangulation of tabulated results 136 Question Sub-question Indicator Data source Data collection Method Data analysis CROSS CUTTING ISSUES EVALUATION KQ 10. Does the Activity have a gender strategy? Document review KII -Data synthesis -Descriptive statistics -Cross￾tabulation -Triangulation KSQ 10.1 Did the Activity SOW include gender oriented considerations, information, indicators, and deliverables? USDA Officers USAID AOR KSQ 10.2 Were gender considerations included in the Activity M&E Plan? IFC Officers USAID AOR Document review KII KSQ 10.3 Does the Activity include any gender-oriented deliverables? IFC Officers USAID AOR Document review KII KSQ 10.4 What gender good practices or lessons learned have been derived from the Activity? IFC Officers USAID AOR Document review KII 137 Question Sub-question Indicator Data source Data collection Method Data analysis KQ 11. How has the activity promoted social inclusion? Document review KII -Data synthesis -Descriptive statistics -Cross￾tabulation -Triangulation KSQ 11.1 How has the activity encouraged the involvement of youth in its activities? IFC Officers USAID AOR Document review KII KSQ 11.2 How has the activity encouraged the involvement of different ethnicities? IFC Officers USAID AOR Document review KII KSQ 11.3 Are activity indicators disaggregated by income level/age/race, if relevant? IFC Officers USAID AOR Document review KII KSQ 11.4 Are there any good practices / lessons learned related to social inclusion that could be useful to share beyond the activity context? IFC Officers USAID AOR 138 Question Sub-question Indicator Data source Data collection Method Data analysis KQ 12. Does the Activity include any environmental sustainability considerations? Document review KII -Data synthesis -Descriptive statistics -Cross￾tabulation -Triangulation KSQ 12.1 Does the Activity assistance promote regional trade environmental sustainability? How? IFC officers USAID AOR Ministries of Agriculture Ministries of Commerce Ministries of Environment Document review KII KSQ 12.2 Does the Activity assistance promote environmental sustainability as part of capacity building? How? IFC officers USAID AOR Ministries of Agriculture Ministries of Commerce Ministries of Environment Document review KII KSQ 12.3 Are there any good practices related to environmental sustainability and sustainable agriculture that could be useful to share beyond the activity context? IFC officers USAID AOR Ministries of Agriculture Ministries of Commerce Ministries of Environment Document review KII 139 Question Sub-question Indicator Data source Data collection Method Data analysis KQ 13. Has the activity had an impact on the number of young people and families leaving or returning to the area? Small farmers Small scale agroprocessors Document review KII 140 ANNEX 5. EVALUATION TIMELINE RITFS Evaluation Schedule (December 2016 – May 2017) Sun Mon Tue Wed Thu Fri Sat 4 5 6 7 ET Leader arrives in San Salvador 8 ET members arrive in San Salvador 9 ET meeting with USAID COR Prepare Evaluation Plan 10 11 12 Prepare Inception Report 13 Prepare Inception Report 14 Prepare Inception Report 15 Prepare Inception Report 16 17 18 19 Prepare Inception Report 20 Prepare Inception Report 21 Prepare Inception Report 22 Prepare Inception Report 23 Christmas Holiday 24 Christmas Holiday 25 Christmas Holiday 26 Christmas Holiday 27 Prepare Database 28 Prepare Database 29 Prepare Database 30 Prepare Database 31 JANUARY 1 2 3 4 5 6 7 141 Sun Mon Tue Wed Thu Fri Sat January 8 ET arrives in San Salvador 9 ET meets RTMA COP. More documents requested ET meets RUTA COP. More documents requested ET planning field work, completing stakeholder mapping, preparing data collection tools, detailing schedule. 10 ET meets IFC COP. More documents requested ET planning field work, completing stakeholder mapping, preparing data collection tools, detailing schedule 11 ET meets USDA PAPA COP. More documents requested Team 1 meets RTMA to define evaluation schedule, logistics. ET planning field work, completing stakeholder mapping, preparing data collection tools, detailing schedule 12 ET planning field work, completing stakeholder mapping, preparing data collection tools, detailing schedule 13 Country clearance request for Honduras and Guatemala ET planning field work, completing stakeholder mapping, preparing data collection tools, detailing schedule 14 January 15 16 Inception report draft, ET planning field work, completing stakeholder mapping, preparing data collection tools, detailing schedule. Document review into database 17 Inception report draft, ET planning field work, completing stakeholder mapping, preparing data collection tools, detailing schedule. Document review into database 18 Inception report draft, ET planning field work, completing stakeholder mapping, preparing data collection tools, detailing schedule. Document review into database 19 ET submits inception report to COP Questionnaires translated into Spanish Document review into database 20 Inception Report submitted to USAID Document review into database 21 142 Sun Mon Tue Wed Thu Fri Sat 22 23 TL works inception report comments Team 1 field work in El Salvador (RUTA+PAPA) Team 2 field work in El Salvador (RTMA+IFC) Daily data entry 24 TL works inception report comments Team 1 field work in El Salvador (RUTA+PAPA) Team 2 field work in El Salvador (RTMA+IFC) Daily data entry 25 Kick off meeting Team 1 field work in El Salvador (RUTA+PAPA) Team 2 field work in El Salvador (RTMA+IFC) Daily data entry 26 Team 1 field work in El Salvador (RUTA+PAPA) Team 2 field work in El Salvador (RTMA+IFC) Team Leader El Salvador Daily data entry 27 Team 1 field work in El Salvador (RUTA+PAPA) Team 2 field work in El Salvador (RTMA+IFC) Daily data entry Team Leader El Salvador 28 RTMA FOCUS GROUPS Team 1 field work in El Salvador (RUTA+PAPA) Team 2 field work in El Salvador (RTMA+IFC) Team Leader El Salvador Daily data entry 29 Team Leader travels to DC 30 Team 1 field work in El Salvador (RUTA+PAPA) Team 2 field work in El Salvador (RTMA+IFC) Team Leader in DC Daily data entry 31 Team 1 field work in El Salvador (RUTA+PAPA) Team 2 field work in El Salvador (RTMA+IFC) Team Leader in DC Daily data entry February 1 Team 1 field work in El Salvador (RUTA+PAPA) Team 2 field work in El Salvador (RTMA+IFC) Team Leader in DC Daily data entry 2 Team 1 field work in El Salvador (RUTA+PAPA) Team 2 field work in El Salvador (RTMA+IFC) Team Leader El Salvador Daily data entry 3 Team 1 field work in El Salvador (RUTA+PAPA) Team 2 field work in El Salvador (RTMA+IFC) Team Leader El Salvador Daily data entry 4 143 Sun Mon Tue Wed Thu Fri Sat 5 ET travels to Honduras 6 Team 1 Field work Honduras (RUTA+PAPA) Team 2 Field work Honduras (RTMA+IFC) Team Leader Honduras Daily data entry 7 Team 1 Field work Honduras (RUTA+PAPA) Team 2 Field work Honduras (RTMA+IFC) Team Leader Honduras Daily data entry 8 Team 1 Field work Honduras (RUTA+PAPA) Team 2 Field work Honduras (RTMA+IFC) Team Leader Honduras Daily data entry 9 Team 1 Field work Honduras (RUTA+PAPA) Team 2 Field work Honduras (RTMA+IFC) Team Leader Honduras Daily data entry 10 Team 1 Field work Honduras (RUTA+PAPA) Team 2 Field work Honduras (RTMA+IFC) Team Leader Honduras Daily data entry 11 Team 1 Field work Honduras (RUTA+PAPA) Team 2 Field work Honduras (RTMA+IFC) Team Leader Honduras Daily data entry 12 13 Team 1 Field work Honduras (RUTA+PAPA) Team 2 Field work Honduras (RTMA+IFC) Team Leader Honduras Daily data entry 14 Team 1 Field work Honduras (RUTA+PAPA) Team 2 Field work Honduras (RTMA+IFC) Team Leader Honduras Daily data entry 15 Team 1 Field work Honduras (RUTA+PAPA) Team 2 Field work Honduras (RTMA+IFC) Team Leader Honduras Daily data entry 16 Team 1 Field work Honduras (RUTA+PAPA) Team 2 Field work Honduras (RTMA+IFC) Team Leader Honduras Daily data entry 17 Team 1 Field work Honduras (RUTA+PAPA) Team 2 Field work Honduras (RTMA+IFC) Team Leader Honduras Daily data entry 18 144 Sun Mon Tue Wed Thu Fri Sat 19 ET @ Guatemala 20 Team 1 Field work Guatemala (RUTA+PAPA) Team 2 Field work Guatemala (RTMA+IFC) Team Leader Guatemala Daily data entry 21 Team 1 Field work Guatemala (RUTA+PAPA) Team 2 Field work Guatemala (RTMA+IFC) Team Leader Guatemala Daily data entry 22 Team 1 Field work Guatemala (RUTA+PAPA) Team 2 Field work Guatemala (RTMA+IFC) Team Leader Guatemala Daily data entry 23 Team 1 Field work Guatemala (RUTA+PAPA) Team 2 Field work Guatemala (RTMA+IFC) Team Leader Guatemala Daily data entry 24 Team 1 Field work Guatemala (RUTA+PAPA) Team 2 Field work Guatemala (RTMA+IFC) Team Leader Guatemala Daily data entry 25 Team 1 Field work Guatemala (RUTA+PAPA) Team 2 Field work Guatemala (RTMA+IFC) Team Leader Guatemala Daily data entry 26 27 Team 1 Field work Guatemala (RUTA+PAPA) Team 2 Field work Guatemala (RTMA+IFC) Team Leader Guatemala Daily data entry 28 Team 1 Field work Guatemala (RUTA+PAPA) Team 2 Field work Guatemala (RTMA+IFC) Team Leader Guatemala Daily data entry MARCH 1 Team 1 Field work Guatemala (RUTA+PAPA) Team 2 Field work Guatemala (RTMA+IFC) Team Leader Guatemala Daily data entry 2 Team 1 Field work Guatemala (RUTA+PAPA) Team 2 Field work Guatemala (RTMA+IFC) Team Leader Guatemala Daily data entry 3 Team 1 Field work Guatemala (RUTA+PAPA) Team 2 Field work Guatemala (RTMA+IFC) Team Leader Guatemala Daily data entry 4 145 Sun Mon Tue Wed Thu Fri Sat 5 ET travels to El Salvador 6 Team 1 + Team 2 + Team Leader El Salvador Data Analysis 7 Team 1 + Team 2 + Team Leader El Salvador Data Analysis 8 Team 1 + Team 2 + Team Leader El Salvador Data Analysis 9 Team 1 + Team 2 + Team Leader El Salvador Data Analysis 10 ET Travels Home 11 12 13 ET El Salvador Team 1 + Team 2 + Team Leader Data Analysis and draft individual activity reports 14 ET El Salvador Team 1 + Team 2 + Team Leader Data Analysis and draft individual activity reports 15 ET El Salvador Team 1 + Team 2 + Team Leader Data Analysis and draft individual activity reports 16 ET El Salvador Team 1 + Team 2 + Team Leader Data Analysis and draft individual activity reports 17 18 19 20 21 Team Leader + 1 ET member drafting final reports 22 Team Leader + 1 ET member drafting final reports 23 Team Leader + 1 ET member drafting final reports 24 Team Leader + 1 ET member drafting final reports 25 26 27 Team Leader + 1 ET member drafting final reports 28 Team Leader + 1 ET member drafting final reports 29 Team Leader + 1 ET member drafting final reports 30 Team Leader + 1 ET member drafting final reports 31 ET submits draft report to COP April 1 2 3 COP returns comments on draft report 4 Team Leader integrates COP comments 5 Team Leader integrates COP comments 6 Team Leader integrates COP comments 7 COP submits draft report to USAID 8 9 10 11 12 13 14 15 146 Sun Mon Tue Wed Thu Fri Sat 16 17 18 19 20 21 USAID sends comments on draft report 22 23 24 Team Leader integrates USAID Comments 25 Team Leader integrates USAID comments 26 Team Leader integrates USAID comments 27 Team Leader integrates USAID comments 28 Team Leader integrates USAID comments 29 30 MAY 1 Team Leader integrates USAID Comments 2 Team Leader integrates USAID Comments 3 Team Leader integrates USAID Comments 4 Team Leader integrates USAID Comments 5 Team Leader sends final version to COP 6 7 8 COP submits Final Report to USAID 9 10 11 12 13 14 15 16 17 18 19 20 21 22 Final Report Presentation Team Leader + COP El Salvador 23 24 25 26 27 147 ANNEX 6. PERSONS INTERVIEWED Last name First name Position and Institution Data tool* Costa Rica Ahern Robert Leader, Agricultural Health and Food Safety, IICA KII Álvarez Daniela Coordinator Knowledge Management and Communication, RUTA KII Azofeifa Roberto Chief of Sustainable Production Department, MAG KII Cordero Ana Marisa Specialist, Agricultural Health and Food Safety, IICA KII Edeso María Dolores Coordinator Comp. 2 Sustainable Agriculture, RUTA KII Gavarette Evangelina General Coordinator, RUTA KII Gómez Ana Isabel Executive Director Executive Secretariat of Sectorial Agricultural Planning, SEPSA, MAG KII Herrera Alexander Coordinator Planning, Monitoring and Evaluation, RUTA KII Mazzuccheli Sergio Manager UNOPS in Costa Rica KII Montero Ricardo Specialist in Politics and Regional Integration, SECAC KII Quesada Ivannia Vice Minister MAG KII Seguro Oswaldo Consultant on Competitiveness, Trade and Agribusiness KII El Salvador Aceto Lorena USAID AOR, IFC KII Alfaro de Hidalgo Celia Unit Coordinator of Control and Hygiene of Foods, Environmental Health Department, Ministry of Health KII Alfaro de Morán Mayra Director Activity Central America Agribusiness Trade Logistics KII Amaya Marvín Head of Immigration, El Amatillo KII Argueta Ernesto Customs Post Manager, El Amatillo KII Argueta Otho Coordinator Technological Innovation Program at /FUSADES KII Artiga Raquel Member of the chocolate processing plant of ACPACI KII, SV Barriere Madecadel Owner, La Canasta Products, Santa Tecla KII Biguer Angela Customs advisor, American Logistics Group KII Blanco Francisco Quarantine Official, El Amatillo KII 148 Last name First name Position and Institution Data tool* Bonilla Tomás Member of the Salvadorian Coffee Council KII Cardoza Rosi Member of the chocolate processing plant of ACPACI KII, SV Cardoza Roxana Member of the ACOPASCA Cooperative FGD, SV Castañeda Vianney Member of the Cocoa Roundtable, Center of Research and Development of Health, CENSALUD, FGD Castillón Juana Member of the ACOPASCA Cooperative FGD, SV Clark Richard ex-coordinator Component 1 RTMA KII Coto Oscar Member of the Cocoa Roundtable, CENTA FGD Cruz Juan Quarantine Official, El Amatillo KII Cruz Calzaldia Gilberto Facilitator of the Cocoa Cooperative ACPACI KII, SV Cuéllar Daniel Member of the Coffee Roundtable, Consultant RUTA KII Cuéllar Rafael USAID AOR, PAPA KII Cuéllar Silvia Executive Director, COEXPORT KII de Pocasangre Ana Leonor Technical Manager, COEXPORT KII Del Cid Daniel Technical assistant, Environmental Health Department Ministry of Health KII Deras Cornelio Chief, Trade Single Window, Central Reserve Bank of El Salvador KII Dubille Kris Member of the Cocoa Roundtable, CENTA FGD Durán Heriberto Member of the Coffee Roundtable, Administrator CTPT. KII Enriquez María Evelia Member of the ACOPASCA Cooperative FGD, SV Escamilla Keny Chief, Agribusiness Division, General Directorate of Agricultural Economics MAG KII Escobar María del Carmen Member of the ACOPASCA Cooperative FGD, SV Escobar Vázquez Douglas Ernesto Director, Plant Health Directorate, MAG KII Figueroa Quiñónez Lauriano Regional Director for Food Safety, OIRSA KII Gallardo Mario Deputy Director of Central American Integration, Ministry of Economy KII González Alan Representative FAO El Salvador KII 149 Last name First name Position and Institution Data tool* González Juan Computer Systems Specialist Activity Central America Agribusiness Trade Logistics KII Guardado Gabriel Activity Coordinator (Central Customs - San Bartolo) KII Guzmán Carlos Chief, Market Information System, DGEA, MAG KII Guzmán de Mata Sandra Cooperation Office Director, MAG KII Hernández Francisco Monitoring and Evaluation Specialist, RTMA KII Hernández Wilberto Director, of Agricultural Economics (DGEA), MAG KII Herrera Gloria President of the Cocoa Cooperative ACPACI, Los Izalcos KII, SV Herrera Jorge Member of the Cocoa Roundtable, Farmer FGD Herrera Miguel Agricultural specialist USDA/FAS El Salvador KII Hirezi Miriam Member of the Coffee Roundtable, Executive Secretary of the CTPT KII Hueso Carlos Member of the Cocoa Roundtable, Lutheran World Relief FGD Huezo Elvy Marketing assistant, USDA/FAST El Salvador KII Interiano Jaime Activity Analyst, (Central Customs - San Bartolo KII Jiménez Cristian Head of Quality Laboratory, La Canasta Products, Santa Tecla KII Landaverde Amílcar Chief of Planning Division, Sectoral Planning and Policy Office, MAG KII Latino de Rodríguez Mary AOR, RUTA KII Lemos Gloria Member of the Cocoa Roundtable FGD Lima Julia Consultant Activity Central America Agribusiness Trade Logistics KII Marroquín Mario Member of the Coffee Roundtable, Technical Assistant of the Executive Secretary CTPT KII Martínez Ana Rosario Member of the chocolate processing plant of ACPACI KII, SV Martínez Néstor Chief of Market Reporting, DGEA, MAG KII Massín Rigoberto Board Member ACPACI KII, SV Mike Joshua Deputy Director Office of Economic Growth, USAID KII Mónico Salvador President, Association of Carriers and Customs Agents KII Montenegro Gabriela ex-COR for RTMA, ex-AOR for RUTA, and IFC activities, USAID/CAM KII 150 Last name First name Position and Institution Data tool* Morán Salvador Quarantine Official, El Amatillo KII Moreno Alvaro Agricultural Specialist, PROESA KII Murcia Grecia Member of the Coffee Roundtable, Communication CTPT KII Navarro Roberto Small cocoa farmer KII, SV Navarro Montes Douglas Chief of Food Safety Division, Plant Health Directorate, MAG KII Ochoa Cathy IT coordinator, Environmental Health Department Ministry of Health KII Ovando Carlos Customs Specialist, RTMA KII Paredes Carlos Export Director, PROESA KII Peña Paola Value Chain Specialist KII Pozo Carlos de Jesús Presidential Commissioner for Border Issues KII Pulque José Antonio Responsible for ACPACI harvest collection center KII, SV Recinos Gloria Member of the Cocoa Roundtable, Catholic Relief Services FGD Reserve Roody Director, Masters in Political Science, “José Simeón Cañas” Central American University KII Rivas Rodrigo Coordinator, Component 2 RTMA KII Rodríguez Luz Estrella Vice Minister of Economy KII Rodríguez Ana María Member of the Cocoa Roundtable, CENTA FGD Rosales Saúl Member of the Cocoa Roundtable, University of El Salvador FGD Salazar René Director of Integration and Trade CCIE KII Salazar Samuel Director Technological Innovation Program at FUSADES KII Tejado Elsi Marleni Member of ACOPASCA FGD, SV Urrutia Mario Officer Technological Innovation Program at FUSADES KII Valladares Ana Mariella Chief, Disease Prevention and Control Division, Animal Health Directorate, MAG KII Vargas Luis Subdirector of Agricultural Economics MAG KII Vásquez Willians Chief of the Sectorial Politics Division, MAG KII Guatemala Ajcac Batzin Juan Angel Honey Producer, Apícola Atitlán, San Pedro La Laguna FGD 151 Last name First name Position and Institution Data tool* Alarcon José Advisor for the Trade Facilitation Direction, Ministry of Economy KII Aldana Mónica Professor of the Faculty of Agronomy, San Carlos University KII Alvarado Yovanni Scope Inside Representative KII Ardón Erwin Director, Planning Division, MAGA KII Arriaza Juan José Operations Manager Superintendency of Tax Administration KII Botzoc Nazario Cocoa farmer FGD Caal Cau Domingo Cocoa farmer FGD Caal Clun Carlos Cocoa farmer FGD Caal Serquij Marcelino Farmer and manager of cocoa fermentation, Waxaquib Tziquin Association FGD, SV Catuu Morales Felipe Cocoa farmer Waxaquib Tziquin Association FGD Cho Asig Rigoberto Cocoa farmer, Waxaquib Tziquin Association FGD, SV Cho Chor Hermelindo President Waxaquib Tziquin Association FGD, SV Cho Tacaj José Cocoa farmer, Waxaquib Tziquin Association FGD, SV Chocoj Pec Pablo Vice President, Waxaquib Tziquin Association FGD, SV Chocol Asig Augusto Secretary, Waxaquib Tziquin Association FGD, SV Choy Chiall Valerio Honey producer, Atitlán Apiculture, Santiago Atitlán FGD Chub Gonzalo Cocoa farmer FGD Chuc Chuc Salvador Cocoa farmer FGD Cojulun Byron Administrator, FundaSistemas KII Colmenares de Ruiz Ana Silvia Director, Food Science Engineering, del Valle University KII Contreras Rosa Executive Director, ProPetén Foundation KII Cox Sean Agricultural Attaché, USDA, US Embassy KII De León Reneé Executive Secretary, PROMECAFE KII de Paiz Glenda Adviser in Rural Business Development, Economic Growth Office, USAID/Guatemala KII De Paz Marvin Undersecretary SESAN KII Díaz Roderico Specialist Region VII, ANACAFE KII 152 Last name First name Position and Institution Data tool* Domínguez July Operations and Environment Coordinator, AGEXPORT KII Drennan Todd Agricultural Counselor, USDA, U.S. Embassy KII Espinoza Eduardo Director of the Center for Economic Integration Studies, SIECA KII Fajardo Héctor President, CATRANSCA-FECATRANS KII Gallardo Ovalle Alejandro Assistant to the Executive Secretary, SIECA KII Galvez Carlos President, PCA, RTMA consultant KII Gamboa Otoniel Director of Operations, FundaSistemas KII García Williams SIECA, Director of Integration KII Girón Pablo Executive Secretary CONADEA KII González Martín Otoniel Cooperative Manager, San Pedro La Laguna FGD Guillén Ana Luisa Director Institutional Strengthening Department, SESAN KII Guzmán Lisette External Cooperation Department, SESAN KII Herrera Ismael AGEXPORT, Responsible for RTMA activities KII Herrera Juan Alberto Academic Secretary of the Faculty of Agronomy, San Carlos University KII Huertas Aura Professor and Researcher of the Faculty of Agronomy, San Carlos University KII Ixcoy Pérez Regino Honey producer, Apícola Atitlán, San Pedro La Laguna FGD Lacs Enrique Viceminister of Trade, Ministry of Economy MINECO KII Libby Margarita ex-coordinator Component 2 RTMA, IADB KII Marroquín Gilberto Technical assistant, FEDECOCAGUA KII Martínez Josefina Activity Manager, USAID Guatemala KII Martínez García David Executive Director, FundaSistemas KII McAndrews Thomas Economic Growth Office Director, USAID/Guatemala KII Mena Eric Agricultural technician, ProPetén Foundation KII Méndez Jorge Ex coordinator, Component 1 RTMA KII Mendoza Jennifer Executive Coordinator, Food and Beverages Commission, AGEXPORT KII Mendoza Quic Antonio Honey producer, Apícola Atitlán, San Juan La Laguna FGD Mesa Victoria Director of Integration, Ministry of Economy KII Molina Román General Manager, Grupo El Tesoro KII 153 Last name First name Position and Institution Data tool* Morales Ana Executive Director, FECAICA KII Morales Carlos ex-COP, RTMA KII Moreno Beatriz Coordinator Activity Section, ANACAFE KII Navarro Luis Executive Director, FEDECOCAGUA KII Obin Dulce Administrator, PROMECAFE KII Olivet Francisco Consultant, Component 1 RTMA KII Orellana Daniel COP PAPA, USDA/Guatemala KII Ortiz Henry Head of the Commerce and Marketing Department of MAGA KII Pacheco Rosa Elvira Vice Minister of Food Security and Nutrition of the Ministry of Agriculture, Animal Husbandry and Nutrition KII Palma Luis President, Customs Brokers Association KII Paz Werner Technical coordinator, ProPetén Foundation KII Paz y Paz Carmen Consultant RTMA, Business Rounds Component 1 KII Pezzarosi Herbert Advisor, Food Safety Division, MAG KII Pocasangre Ana Vilma Activity Manager, USAID Guatemala KII Pop Gutiérrez Álvaro Alexander Business Development Technician, FundaSistemas KII Quevedo Fernando Country Representative, IADB KII Quezada Raúl Agricultural technician and Activity coordinator, FundaSistemas KII Quib Esbín Cocoa Farmer, community facilitator, treasurer FGD Rivera Carlos Enrique Secretary General, FECAICA KII Rochmann Jessica Director of Administration, FundaSistemas KII Rosado Gerardo Executive Coordinator, Beans and Vegetables Committee, AGEXPORT KII Ruano Mauricio Competitiveness Advisor, SIECA KII Ruano Nelson Director, Food Safety Division, MAGA KII Saenz Annabella Advisor to the Viceminister. Ministry of Economy MINECO KII Sánchez Luis Eduardo Manager, Central American Trade Network, SIECA KII Sandoval Jaime Information Systems Consultant, SIECA KII Seb Alejandro Cocoa farmer, Waxaquib Tziquin Association FGD, SV Sierra Roca Edín Technician, FundaSistemas KII 154 Last name First name Position and Institution Data tool* Sosa Jaime Integrated Agricultural and Environmental Protection Program Coordinator, MAG KII Sterkel Waleska Executive Director, AMCHAM KII Tacaxoy Quiejú Gregorio Honey Producer, Atitlán Apiculture, Santiago Atitlán FGD Taracena Eduardo Coordinator of requirements issuing, Pest Risk Assessment Division, MAGA KII Tay Karla Agricultural specialist, USDA/FAS Guatemala KII Tejeda Gaudi SAT, Advisor to the Superintendent KII Tiúl Ico´ Levio Daniel Cocoa producer, Community facilitator FGD Tiúl Yaxcal Sebastián Cocoa farmer FGD Tot Paau Rúben Darío Manager, FundaSistemas in Cobán KII Vásquez Israel Honey producer, Atitlán Apiculture, Santiago Atitlán FGD Vásquez Barreno Jerónimo Honey producer, Atitlán Apiculture, Santiago Atitlán FGD Vásquez Xeché Juan Honey producer, Atitlán Apiculture, Santiago Atitlán FGD Vergara Carmen Gisela Secretary General, SIECA KII Villagrán Marta Ileana Member of Activity Section, ANACAFE KII Yalibat Vigdalia Assistant, FundaSistemas KII Yat Ángel Salvador Cocoa farmer and Community facilitator FGD Honduras Aguilar Flores Delmis Member REDMUCH FGD Aguirre Eyby Executive Director, AMCHAM KII Alemán Ana María Member REDMUCH FGD Andrews Héctor Regional Program Coordinator, USAID bilateral KII Barahona Orfilia Member REDMUCH FGD Borjas Ramón Technician Nutrition and Food Security Technical Unit. KII Bueno Mirian Chief of Food Safety, National Agricultural Health Service SENASA/SAG KII Bustamante Jenny Vice President REDMUCH FGD Castillo Rosa Adviser, Ministry of Environment FGD 155 Last name First name Position and Institution Data tool* Cuéllar Enid Leader, Agriculture Products Market Information System SIMPAH KII Destephen German Credit Services Manager, PROGRESA KII Domínguez Ciriaco Potato producer ECARAI & Board member FGD Domínguez María Salomé Potato producer ECARAI FGD Domínguez Sabino Potato producer ECARAI FGD Domínguez Santos Neri Potato producer ECARAI FGD Dunnaway Ana Director, UPEG FGD Durán Marlon Climate change and risk management unit coordinator. SAG FGD Funes Nelson Omar Technical Manager IHCAFE KII Gálvez Alberto Coffee farmer CAFEL SV García Desiree National coordinator Component 2 RTMA KII Gómez Ana Agriculture Specialist, USDA/FAST KII Gómez García Genaro Potato producer ECARAI FGD Gómez García José Inés Potato producer ECARAI FGD González Trinidad Vegetable producer FGD González Víctor Director of the Research Department, FHIA KII Guevara Gladys Member REDMUCH FGD Gutiérrez Carlos Coffee farmer CAFEL KII, SV Hernández María Rosibel Vegetable producer FGD Interiano Kenia Activity Management ECARAI FGD Jiménez Pedro Technical Assistant Food Security and Nutrition Observatory OBSAN-UNAH FGD Leiva Josué Coffee Technician “Las Malvinas” CAFEL KII Lemus Juan José Coffee technician CAFEL KII Lopez Karla Adviser, Secretariat of Industry and Commerce KII López Raúl President, Customs Brokers Association KII Márquez Reina Vegetable producer HORTISA (Lepaterique) FGD Martínez Carlos Vegetable producer FGD Martínez Constantino Potato producer ECARAI FGD Martínez Diana Plant Diagnostics and Monitoring, SENASA/MAG KII 156 Last name First name Position and Institution Data tool* Martínez Francisco Vegetable producer FGD Martínez Orlando Technical Assistant Doctoral Program UNAH FGD Martínez Adolfo General Director, FHIA KII Medina Rafael Executive Director, Chamber and Commerce and Industries of Tegucigalpa KII Medlicott Andrew Director, Market Access Activity, FINTRAC KII Mejía Rosauro Member REDMUCH FGD Meléndez Mauren Adviser, Development and Social Inclusion Secretary SEDIS FGD Mena Juan Pablo Packing Plant Manager, Dinant Corporation KII Méndez Eodora President, Board of Directors ECARAI (La Esperanza - Intibucá) FGD Moreno Marvin FAO Consultant FGD Oliva Amalia Liaison Assistant Doctoral Program UNAH FGD Ortega Claudia María Facilitator, Foro Nacional de Convergencia (FONAC) FGD Oseguera Margarita Coordinator Food Security and Nutrition Observatory OBSAN￾UNAH and Director Human Development Doctorate KII Oseguera Marvin Analys Honduran Council of the Private Enterprise COHEP KII Paz Dunia Secretary REDMUCH FGD Paz Ricardo Director, SENASA/SAG KII Peña Ricardo Director Planning, Evaluation and Management Unit (UPEG) and Honduras Representative of the Regional Technical Committee CAC (CTR-CAC) KII Pinto Magdalena Member REDMUCH FGD Reyes Jorge Activity Management Specialist Food Security & Energy, Economic Growth Office, USAID/Honduras KII Rioz Karla General Manager, Chamber and Commerce and Industries of Tegucigalpa KII Rodríguez Jesús Potato producer ECARAI FGD Rubio Geimy Member REDMUCH FGD Ruiz de García Lucía Owner, Agrigentum KII Sánchez Gertrudis Potato farmer ECARAI FGD Santos Héector Food Security Activity Management Specialist, Economic Growth Office, USAID/Honduras KII Servellón Freddy Vegetable producer, HORTISA Board President FGD 157 Last name First name Position and Institution Data tool* Servellón Gloria Vegetable producer FGD Servellón Johnny Vegetable producer FGD Servellón Manuel Vegetable producer, HORTISA Board member FGD Servellón Naun Arnulfo Vegetable producer FGD Servellón Pedro Darío Vegetable producer, HORTISA Board member FGD Servellón Sandra Lucía Vegetable producer FGD Silva Manuel Alejandro Vegetable producer FGD Silva Marina Vegetable producer, Board member HORTISA FGD Suazo Fanny Treasurer REDMUCH FGD Tejada Roberto Director of the Communication Department, FHIA KII Tovar Carlos Coordinator Technical Unit of La Vía Campesina FGD Uclés Lila President REDMUCH FGD Urbina Jerónima Trade and Integration Director, Ministry of Economy KII Vásquez Gregorio Potato farmer ECARAI & President of cooperative FGD Vaughan Gregory Agricultural Development Officer, Economic Growth Office, USAID/Honduras KII Vega Víctor Manuel Inspector, National Institute of Migration KII Villalobos Miriam Member REDMUCH FGD Zuñiga Adalberto Chief of Plant Health, SENASA/SAG KII United States of America Deane Arsala Trade/Competitiveness Specialist, World Bank Group KII Jennings Michelle former Director of the Economic Growth Office in El Salvador KII Johnston Erica International Program Manager USDA KII Perlman Lesley M&E Backstopping for CAM USAID BFS KII Romero Andrés International Agricultural Program Specialist, USDA/FAST KII Steffen Phillip Acting Chief, Asia-LAC Division Bureau of Food Security KII Vallejos Indalecio International Agricultural Program Specialist, USDA/FAST KII Note: * For data tools, KII are key informant interviews and FGD are focus group discussions. 158 ANNEX 7. EVALUATION TEAM ROLES AND CONFLICT OF INTEREST STATEMENTS The Evaluation Team was comprised of a Team Leader, Luis Bernal, MPP a Senior Evaluation Specialist; Fernando Hernández and Joke Vuurmans, experts in agriculture; Amy Angel and Rafael Cornejo, experts on trade, as well as Katia Zepeda, the evaluation analyst/coordinator. Luis Bernal, MPP, Evaluation Team Leader is a senior international development and evaluation consultant with more than 30 years of experience designing, implementing, and evaluating private and public initiatives for: economic growth; local development; private sector competitiveness; entrepreneurship; micro, small, and medium-sized business promotion; poverty alleviation, and social inclusiveness. His extensive Monitoring & Evaluation background is comprised of meta-evaluations, performance and ex-post evaluations of multi-site, multi-million and multi-country projects intended to measure a program’s efficiency, efficacy, impact, outreach, and sustainability. Mr. Bernal has led and collaborated on regional projects in Latin America and the Caribbean, as well as projects in: Barbados, Bolivia, Chile, Colombia, Costa Rica, Ecuador, Guatemala, Honduras, Panama, Chile, Spain, Liberia, and the United States, working for IDB, USAID, UN-PBSO and United Nations Development Programme. Mr. Bernal earned his Master’s degree in Public Policy from the University of Massachusetts and a certificate in Evaluation and Applied Research Methods from Claremont University. He also holds a Bachelor’s degree in Business Administration from the Universidad Externado de Colombia. Amy Angel is an Agricultural Economist and Trade expert. For twelve years Mrs. Angel was Senior Researcher and Institutional Development Strategist for FUSADES where she focused on Agriculture and Livestock research, food security, rural development, trade facilitation, trade negotiations. As a top-level adviser for the Government of El Salvador, she assisted the Minister Office in policy-making for Agriculture, Livestock and trade; she also represented the Government as agriculture trade negotiator for CAFTA, and other trade agreements related to sorghum, corn, pork and rice; moreover, she was responsible for design, coordination and implementation of the national agriculture and livestock public policy. Mrs. Angel has consulted in El Salvador, Belize and Trinidad Tobago working for USAID, IICA, World Food Programme, Chemonics, Abt Associates, among others. She earned a Bachelor degree in Agricultural Economics with emphasis in Agribusiness from Texas A&M University and is a Doctorate candidate in Agriculture Economics with emphasis in Agricultural Policy and Markets Analysis. Rafael Cornejo is an international trade expert with thirty-five years’ experience in international trade, free trade agreements, Latin America integration, markets access and market information systems. For more than eighteen years he has provided technical assistance to trade negotiation roundtables in several areas of Latin America. Mr. Cornejo is a consultant, lecturer and trainer specialized in rules of origin, as well as international trade information systems. He worked for 35 years for the IDB, and as an independent consultant he has worked for the Canadian International Development Agency, Interamerican Development Bank, United Nations Conference on Trade and Development, Latin America Association for Integration, the U.S. Department of Commerce, among others. Mr. Cornejo holds a Bachelor in Management from 159 Belgrano University, and a Bachelor in Political Economics from Buenos Aires University, both in Argentina. Fernando Hernández is an expert in Agricultural Economics with more than twenty years’ experience in designing, and implementing social and economic research projects, as well as M&E consulting assignments for which he specializes in qualitative research and analysis methods. His areas of consulting include quantitative methods for rural development, agribusiness, agricultural value chains, rural infrastructure, poverty alleviation, children and youth education on responsible sexuality, and rural sanitation in projects carried out by FAO, IDB, USAID USDA, World Food Programme, Technoserve, Swiss Contact, among others, mainly in rural Nicaragua. Mr. Hernández holds a Bachelor in Agricultural Economics, and a postgraduate degree in Agricultural Economics Policy, both from the National Autonomous University of Nicaragua. Joke Vuurmans is an Agricultural Economics and M&E expert with more than twenty-five years’ experience in design, implementation, monitoring and evaluation of projects for rural development with emphasis in sustainable development, food security, watershed management, ecosystems management, agribusiness, rural productivity, value chains, land use, gender mainstreaming and human rights. She has consulted for the European Union, USAID, the Netherlands Government, IICA, and the Belgium international cooperation. Mrs. Vuurmans has consulting experience in Belize, Bolivia, Central America, Chile, Colombia, Costa Rica, Cuba, Ecuador, El Salvador, Guatemala, Honduras, MERCOSUR, Mexico, Nicaragua, Panama, Papua New Guinea, Peru, Senegal, Sudan, Venezuela. Mrs. Vuurmans holds a Bachelor degree in Tropical Agriculture and a Master’s in Tropical Agriculture with specialization in Agriculture Extension, Food and Nutrition, both from Wagengeim University in The Netherlands. The Evaluation Team Leader was responsible for overseeing and coordinating all activities related to the Strategy’s performance evaluation (from inception, through field work, to final reporting), ensuring the production and completion of quality deliverables in a professional manner, and in conformance with the evaluation SOW. With support from the Evaluation Team, Bernal was responsible for the writing of major evaluation reports such as inception, weekly reports, presentations, and the evaluation final report. The four subject experts worked on teams of two: Team 1 was comprised of Amy Angel and Joke Vuurmans and they were jointly responsible for data collection, analysis, and reporting focused on PAPA and RUTA. The members of Team 2, Rafael Cornejo and Fernando Hernández, likewise, were responsible for data collection, analysis, and reporting, but focused on RTMA and IFC. Each group brought about evaluation reports of their respective projects which fed the final evaluation report delivered to USAID. All team members, however, worked as one team toward a successful completion of the evaluation as per its SOW. 160 161 162 163 164 165 ANNEX 8. DATA COLLECTION PROTOCOLS AND INSTRUMENTS Interview Protocol Key informant interviews are qualitative in-depth interviews with stakeholders from the different activities and/or related to the overall strategy. These include implementing partners, USAID regional and national mission personnel, government officials, small farmers and farm organization representatives, exporters, and trade associations. These stakeholders have been selected from contact lists provided by the activities, as well as from quarterly and annual reports provided by them, based on their level of participation in activity activities. Furthermore, this diversity of categories of informants allows the team to extract findings from the macro perspective at the overall strategy level, down to the micro level of both direct and indirect activity beneficiaries. The purpose of key informant interviews is to collect information from a wide range of people who have firsthand knowledge about the activities. These stakeholders, with their particular knowledge and understanding, can provide insight the key evaluation questions. All evaluation team members will conduct interviews, and in some occasions, when an informant potentially has information that is relevant to the subject area of different evaluation team members, more than on evaluation team member may be present. Interviews will be coordinated through implementing partners or directly by the support staff of the evaluation team. The interviews will be semi-structured, following a questionnaire which has been designed for each category of informant, but allowing the interviewer to modify, expand or withhold questions, depending upon the question’s relevance for the specific interviewee and upon information obtained during the conversation. Summaries of answers will be recorded by the interviewer and their text will later be entered into a database (using the Google Docs platform). These answers will then be combined in the database by key evaluation question, allowing for a more agile recompilation of the information from the series of interviews to be done and facilitating their analysis. Focus Group Discussion Protocol A Focus Group Discussion (FGD) is “a data collection technique in which a small group of people, usually between six and twelve individuals, is guided by a moderator to discuss specific topics in a structured way” (USAID, 2013). The moderator uses probing questions to solicit to the participants their opinions, ideas, and other information about specific areas or subjects of common interest related to the activity to be evaluated. The advantage of FGD is that they are quick, versatile and typically inexpensive, so they can be used with a broad range of stakeholder groups in a wide variety of settings. They allow evaluators to obtain a wealth of in-depth qualitative information. FGIs typically offer greater descriptive depth than what is provided through surveys and a broader base of perspectives than often is obtained through individual interviews. For these reasons, FGIs are commonly included as part of performance evaluations. 166 The key advantages of FGIs are that they are: 1. Interactive and Informative. FGDs enable participants to interact with each other, sharing and reacting to each other’s ideas. This process allows for new ideas to emerge that may not have been expressed by any one person. In other words, a FGD is more than just interviewing several people at the same time; it is more than the sum of its parts. 2. Time and Resource Efficient. One FGD can collect rich, substantial data from multiple stakeholders in one or two hours. Though an individual interview allows the evaluation team to delve deeper into the situation of one individual, and surveys can capture data from statistically representative samples, FGDs offer an efficient option for obtaining detailed qualitative data from up to a dozen people in a short period of time, without the need for any special equipment. There are also several limitations to FGDs that are important to consider. 1. Require advanced planning. FGDs call for getting together a group of people at the same place and time, which usually requires invitations to be distributed and confirmed in advance and, therefore, often implies greater coordination. 2. Vulnerable to moderator bias. The moderator plays a central role throughout the FGD process, especially when guiding the discussion and interpreting the results. Thoughtful design of the moderator’s guide, accurate recording, systematic coding and involvement of more than one individual in the data analysis can help minimize, but not eliminate, the risk of bias. 3. Limited confidentiality. While there are rare instances in which people may be more forthcoming in a group discussion than they would be on an individual basis, in most cases participants will be less likely to share very personal or sensitive information when they are speaking with a group. In many cases, FGDs are combined with other data collection techniques as part of a mixed-method design. Collecting information about the same issue but from different sources or different methods also allows for triangulation, the process whereby different sets of data are compared and contrasted to check for consistency, helping to increase confidence in the evaluation results. In other words, FGDs often serve as a key source of data in performance evaluations, and often do so in combination with other data collection techniques such as reviews of secondary data and individual interviews. In the particular case of the present Regional Integrated Trade and Food Security Strategy evaluation, we will take the following into consideration: How will the FGDs help to answer the evaluation questions? As already mentioned before, the advantage of FGD is that this data collection methodology enables participants to interact with each other, sharing and reacting to each other’s ideas, which allows for new ideas to emerge. Who will conduct the FGIs and why they are qualified to do so? A team of two people will conduct the FGD: a moderator and one assistant moderator. The person who has the primary responsibility for the evaluation of the particular activity will be the moderator. The assistant moderator will be one of the other members of evaluation team. Both the moderator and the assistant moderator are knowledgeable about the topic being discussed and familiar with the language and culture of the participants. The moderator facilitates the FGD by guiding the group through a discussion that stays on topic, is rich in information, and maintains a comfortable environment where participants feel free to 167 express their perspectives. The responsibility of the assistant moderator is to document the FGD, including taking thorough notes and making sure that all technology, such as audio equipment, is working properly. What are the rationale and method for deciding the number, timing, and location of the FGDs? The ideal number of FGD participants is between six and twelve. Because the purpose of an FGD is to generate a focused discussion on a specific topic, it is important not only to have enough people to hold an engaging discussion, but also to limit the size of the group so that all participants can express their views. FGIs will be scheduled at times and in places that are comfortable and convenient for the participants. If participants feel comfortable and at ease in the environment, they will be more likely to respond openly and honestly to the questions asked. How will the participants be selected and recruited? The selection of the participants depends on the questions to be answered, but as general rule, all of the participants must be as similar as possible with regard to the characteristics most relevant to the questions asked. If it is likely that males and females experience a activity or activity differently, perhaps because of traditional gender roles, we will separate FGDs—one with men and one with women. Focus group participants ideally should be representative of the larger relevant population–for instance, with individuals coming from various geographical districts, from various age groups, representing both genders, and so on. In the recruitment process the Implementing Partner of the activity plays a crucial role, since they have built an important relationship with the persons to be invited. How will the FGDs be recorded? In order to be analyzed and used the information gathered during the FGD, the assistant moderator will accurately and comprehensively record it. Data from the FGD will be recorded in different ways, and generally, it will involve note-taking and possibly audio/image recording. Recording the sound or images from an FGD will be a valuable supplement to written notes. Recording discussions also makes it possible to transcribe them and to code the content, which allows evaluators to conduct more detailed analyses of the data. How will the FGD data be analyzed and presented? The analysis of FGD data actually begins during the FGD and continues in a systematic way until the results are reported. Online Survey An online survey was conducted among USAID CORs, COP of the activities, and other people involved with the design and/or implementation of the Strategy and its activities. The survey was conducted from X to X, 2017. Of the x people invited to participate, x filled out the survey form. The questions for the survey and responses are shown below. Questions, results (mean, high, low) 168 Sample FGD protocol RTMA Project Guide for a FGD with women and men farmer members of ACOPASCA at El Paisnal, El Salvador January 23, 2017 I. Introduction In this evaluation, we will use FGD to collect qualitative information to assess performance indicators of the RTMA project. The FGD is held in the same place where the project beneficiaries live, and is complemented with individual interviews with a few Board members of the Cooperative. Likewise, there will be a visit to the production facilities funded by the project. The discussion will be about the perceptions, knowledge, and concern that the beneficiary organization members might have. The group activity will encourage participants to freely convey their comments and opinions on the subjects being evaluated within an open environment welcoming the free exchange of ideas. II. Methodology 1. FGD Objective To gather comments and opinions from women and men farmers benefited by RTMA, regarding the subjects of interest for project performance evaluation. 2. Participants’ profile - Between 10 and 12 persons - Equitable participation in gender terms - Members of the organization who have benefited from the project - Members who have been directly involved with the project - Participants who are interested, open, and communicative so that the group discussion runs fluidly. 3. Greetings The FGD facilitator greets the group and makes introductions facilitating a brief social exchange. The group is divided between women and men 4. FGD Questions - Please describe your participation in the RTMA project. What support did you receive from it? What production changes did you implement because of it? - We would like to know your opinions and comments regarding the Project that brought the packing plant to you, what do you know about such project? - Would you tell us what kind of assistance did you receive? What investment was made? Did you receive something else such as training or other services? - Please tell us how production was before the project and how it is now. 169 - How has your income changed since the RTMA project implementation? - Have the changes coming from the project represented changes in your personal life? Your family life? Has yours and your family life improved thanks to RTMA? Has life improved for other members of the Cooperative and their families? - Do you feel your income level has improved? How much? For instance, before you earned $$ and now $$? - If your income does have improved, what kind of purchases do you make with the extra money? - How has your family life improved thanks to efforts supported by RTMA? Is there any change in the family diet in terms of quantity, quality, balance, nutrition? - Has your family experienced other life improvements? Education? Clothing? Shoes? Health? Housing? Others? - Have your consumption patterns changed due to the new extra income? - Tell us what kind of expenses do you make now that you did not before? For instance, family getaways, or other? - If your diet has changes, tell us how it was before and how it is now. What foods do you buy now that did not before? - What changes have your family unity experienced since the production changes brought by RTMA started being implemented? - Has your family relations and quality of life improved? Deteriorated? Is it the same? 170 Sample FGD protocol Spanish Language version Proyecto Regional de USAID Para Comercio y Alianzas de Mercado – PRUCAM Guía Grupo Focal Productoras y Productores Miembros/Socios de la Cooperativa ACOPASCA en el Paisnal, en el Departamento de El Salvador Enero 23, 2017 I. Introducción En esta investigación se hará uso de la herramienta de grupo focal con el fin de recopilar información cualitativa que permita Evaluar indicadores de desempeño del Proyecto PRUCAM/RTMA/USAID/NATHAN. El grupo focal se efectúa en la localidad donde residen las y los productores beneficiarios, se complementa la actividad realizando entrevista en grupo a algunos representantes de la junta directiva de la cooperativa y se visitará las inversiones realizadas. En este sentido se organizará una discusión diseñada para obtener las percepciones, conocimientos e inquietudes de productores y productoras beneficiarios del proyecto. La actividad grupal facilitará que las y los participantes puedan expresar libremente sus opiniones y comentarios sobre los diferentes aspectos de interés relacionados a los objetivos y preguntas de evaluación, en un ambiente abierto para el libre intercambio de ideas. II. Metodología 1. Objetivo Grupo Focal: Conocer opiniones y comentarios de productoras y productores beneficiarios del Proyecto PRUCAM en relación a los temas y aspectos de interés para la evaluación de desempeño del Proyecto. 2. Participantes - Participantes entre 10 -12 - Participación equitativa en términos de género - Miembros de la organización beneficiados con los esfuerzos desarrollados por el proyecto - Manejan los temas relacionados a estos esfuerzos - Características personales apropiadas para facilitar la discusión grupal: participativos, pro activos y expresivos. 3. Presentación con productores El moderador del grupo focal realiza los saludos y presentaciones con los participantes, se facilita un breve momento de intercambio El grupo se sub divide en dos por criterio de género (hombres – mujeres) 4. Guía de Preguntas Grupo Focal - ¿Cuál fue su participación en el proyecto RTMA? ¿Qué apoyo recibió? ¿Qué cambios de producción implementó? 171 - Queremos conocer sus opiniones y comentarios del Proyecto que les ayudó con la empacadora, ustedes conocen sobre este proyecto? - Cuéntenos en qué consistió el apoyo que recibieron, cuál fue la inversión? Qué más representó el proyecto, capacitaciones? Otro tipo de apoyo que recibieron? - Cuéntenos como era la producción antes del proyecto y después del proyecto? - ¿Cómo ha cambiado su ingreso desde la implementación del mecanismo de apoyo de RTMA? - Estos cambios que vinieron con el proyecto también ha representado cambios en su vida personal? En la vida familiar? Cuáles son esos cambios? Ha mejorado la vida de usted y su familia? Y los demás miembros de la cooperativa y sus familias? - Usted siente que ha mejorado el nivel de ingresos de la familia, ahora puede ser cuánto más? Por ejemplo antes ganaba 100 y ahora gana ¿? - Si sus ingresos han aumentado, ¿qué tipos de compras realiza con ese ingreso adicional? - De identificarse incremento de ingresos familiares: cómo ha mejorado la vida de su familia con el esfuerzo realizado con el apoyo del proyecto? La comida (dieta familiar) cómo era antes y cómo es ahora, en cantidad? en contenido (balanceada y nutritiva)? - Otras posibles mejorías de la vida familiar: Escuela (educación)? Vestimenta y calzado? Salud? Vivienda? Otros? - ¿Han cambiado sus patrones de consumo debido a ese cambio en los ingresos? - Cuéntenos que gastos hace ahora que antes no hacía, por ejemplo, ahora suelen haber más paseos familiares? Sale a pasear con la señora y los hijos e hijas? O cuál? - ¿Usted compra diferentes alimentos ahora? Si es así, ¿Que son? - Si ha cambiado la dieta alimenticia, cuéntenos que alimentos compra ahora que antes no compraba? - ¿Qué cambios ha notado en su unidad familiar / viviente desde la implementación de los cambios en la producción de RTMA? - Qué opina en relación a la vida de la familia? Ha mejorado? Igual? Desmejorado? 172 ANNEX 9. DOCUMENTS REVIEWED 1. PAPA 1.1 USAID (August 2011). Evaluation of the regional and bilateral programs to support trade compliance under CAFTA-DR: Final Report. 1.2 USAID (April 27, 2011). Signed Agreement of the Activity Promoting Food Security and Trade Integration through Sanitary and Phytosanitary Standards (SPS) and Other Ag-Related Capacity Building. (includes: ANNEX A - Statement of Work – SOW, ANNEX B - Financial Plan and Budget. ANNEX C - Standard Provisions, and ANNEX D - Long-Term Technical Services). 1.3 USDA/USAID PAPA (July 2011). Promoting Food Security and Trade Integration Through SPS and Other Agriculture-Related Capacity Building. Work Plan July 2011- July 2012. 1.4 USAID (September 27, 2011). Addendum to the Agreement of the Activity Promoting Food Security and Trade Integration through Sanitary and Phytosanitary Standards (SPS) and Other Ag-Related Capacity Building. Modification Number: 01 1.5 USAID (May 3, 2016). Addendum to the Agreement of the Activity Promoting Food Security and Trade Integration through Sanitary and Phytosanitary Standards (SPS) and Other Ag-Related Capacity Building (includes: Modifications of ANNEX A and ANNEX B). Modification Number: 05. 1.6 USDA/USAID PAPA (October 2013). Promoting Food Security and Trade Integration through SPS and Other Agriculture-Related Capacity Building. Progress Report Quarter Four and Final FY 2013 October 1, 2012 – September 30, 2013 1.7 USDA/USAID PAPA (July 2014). Promoting Food Security and Trade Integration through SPS and Other Agriculture-Related Capacity Building. Progress Report Quarter Four and Final FY 2014 Oct 1, 2013 – Sept 30, 2014. 1.8 USDA/USAID PAPA (October 2015). Promoting Food Security and Trade Integration through SPS and Other Agriculture-Related Capacity Building. Progress Report Quarter Four and Final FY 2015 July 1, 2015 – Sept 30, 2015. 1.9 USDA/USAID PAPA (October 2016). Promoting Food Security and Trade Integration through SPS and Other Agriculture-Related Capacity Building. Progress Report Quarter Four and Final FY 2016 July 1, 2016 –Sept 30, 2016. 1.10 USDA/USAID PAPA (May 2011). Initiative for improved and harmonized agricultural statistics and sanitary-phytosanitary regulatory infrastructure in Central America. Progress Report Final PAPA Report 2005-2011. 1.11 Margaret McDaniel and Daniel Orellana (March 11, 2009). PowerPoint Presentation GAPS Analysis: CAFTA-DR Sanitary and Phytosanitary Trade Capacity Building. 1.12 Food and Drug Administration – FDA (February 2013). FDA´s International Food Safety Capacity￾Building Plan: Food Safety Modernization Act. 1.13 USDA/USAID PAPA (September 2012). CAFTA PAPA: Regional Logic Model. 1.14 USDA/USAID PAPA (_____). Case Study: Snow Peas and French Beans in Guatemala. 1.15 USDA/USAID PAPA (_____). Case Study: Inspection and Certification Program in Honduras. 1.16 USDA/USAID PAPA (_____). Case Study: Pesticides diagnostics at national laboratory (LNS). 1.17 _____ (November 4, 2015). Ministro de Salud Pública y Asistencia Social. Acuerdo Ministerial No.196-2015. 1.18 Diario Oficial de El Salvador Tomo No.404. Número 137 (July 24, 2014). Acuerdo No. 287 RTS 67.04.01.13 Buenas Prácticas Agrícolas de Frutas y Hortalizas. 1.19 La Gaceta de Honduras No. 33,109 (April 26, 2013). Acuerdo 007-2013 Reglamento CIRT. 173 1.20 Diario Oficial de El Salvador Tomo No.403. Número 73 (April 24, 2014). Acuerdo 176 RTS-LMR de Plaguicidas. 1.21 Acuerdo MAG No.51 El Órgano Ejecutiva en el Ramo de Agricultura y Ganadería. Equivalence in 2006. 1.22 Anexo de Resolución No. 243-2009. Reglamento Técnico Centroamericano RTCA 67.04.50:08 ALIMENTOS. Criterios microbiológicos para la inocuidad de alimentos. 1.23 La Gaceta de Honduras No. 33,529 (September 11, 2014). Reglamento de Frutas. 1.24 Diario Oficial de El Salvador Tomo No.404. Número 137 (July 24, 2014). Ley de RTS Inspección sanitaria de la carne porcina. 1.25 Diario de Centro América (November 12, 2015). Acuerdo Ministerial No. 491-2015 del Ministerio de Agricultura, Ganadería y Alimentación, del 03 de November del 2015, sobre el listado actualizado de plagas reglamentadas y disposiciones para determinar requisitos fitosanitarios de importación aplicables a envíos de origen vegetal, que se pretendan importar por primera vez al territorio nacional. 1.26 Ministry of Agriculture, Livestock and Food of Guatemala (June 15, 2006). MINISTERIAL AGREEMENT NUMBER 340-2006 (dispositions to recognize as equivalent the measures of innocuousness of foods of the United States of America for the systems of product inspection of bovine, pig and bird meat not processed). 1.27 _____ (_____). Protocolo Institucional. 1.28 _____ (_____). Training Activityions During Fiscal Year 2012. 1.29 _____ (_____). Reglamentos de SENASA modificados con apoyo de fondos USDA 2013-2014. 1.30 Anexo Resolución No. 282-2012 (COMIECO-LXII) Reglamento Técnico Centroamericano RTCA 65.05.52:11. Productos utilizados en alimentación animal y establecimientos. requisitos de registro sanitario y control. 1.31 Anexo 1 de la Resolución No. 257-2010 (COMIECO-LIX). Reglamento Técnico Centroamericano RTCA 65.05.51:08. MEDICAMENTOS VETERINARIOS Y PRODUCTOS AFINES. Requisitos de registro sanitario y control. 1.32 USDA/USAID PAPA (October, 2012). Promoting Food Security and Trade Integration through SPS and Other Agriculture-Related Capacity Building. Progress Report Quarter Four and Final FY 2012 October 1, 2011 – September 30, 2012. 1.33 USDA/USAID PAPA (October 2011). Promoting Food Security and Trade Integration through SPS and Other Agriculture-Related Capacity Building. Progress Report Quarter Four FY 2011 July 1 - September 30, 2011. 1.34 USDA/USAID PAPA (July 2011). Promoting Food Security and Trade Integration through SPS and Other Agriculture-Related Capacity Building. Work Plan July 2011 - July 2012. 1.35 USDA/USAID PAPA (October 2013). Promoting Food Security and Trade Integration through SPS and Other Agriculture-Related Capacity Building. Work Plan October 1, 2013 – September 30, 2014. 1.36 USDA/USAID PAPA (October 2014). Promoting Food Security and Trade Integration through SPS and Other Agriculture-Related Capacity Building. Work Plan October 1, 2014 – September 30, 2015. 2. RUTA 2.1 UNOPS/RUTA (February, 2012). Programa regional para la efectividad de las políticas de seguridad alimentaria y agricultura sostenible: Propuesta Documento de aplicación técnica y financiera para orientar la ejecución del programa – Resumen. 2.2 USAID (September 17, 2012). Signed Grant of the Regional Food Security Policy Effectiveness and Sustainable Agriculture Program (includes: Attachment 1 - The Schedule, Attachment 2 - Program Description, Attachment 3 - Standard Provisions, and Attachment 4: Logical Framework). 174 2.3 RUTA (November 2012). Regional Food Security Policy Effectiveness and Sustainable Agriculture Program - USAID/ECAM. Work Plan 2012-2013, Revised November 7, 2012. 2.4 UNOPS/RUTA (____). Regional Food Security Policy Effectiveness and Sustainable Agriculture Program: Presupuesto USAID Distribución por componente y actividad según categoría de inversión. 2.5 UNOPS/RUTA (_____). Regional Food Security Policy Effectiveness and Sustainable Agriculture Program: Quarterly Report September-December 2012. 2.6 UNOPS/RUTA (February 2013). Regional Food Security Policy Effectiveness and Sustainable Agriculture Program: Monitoring and Evaluation Plan (M&E PLAN). 2.7 UNOPS/RUTA (February 2013). Regional Food Security Policy Effectiveness and Sustainable Agriculture Program: Monitoring Indicator Table. 2.8 UNOPS/RUTA (_____). Regional Food Security Policy Effectiveness and Sustainable Agriculture Program: Quarterly Report January-March 2013. 2.9 UNOPS/RUTA (_____). Regional Food Security Policy Effectiveness and Sustainable Agriculture Program: Quarterly Report April-June 2013. 2.10 UNOPS/RUTA (_____). Sustainable Agriculture and Food Security Program. Annual Report September 2012 - September 2013. 2.11 UNOPS/RUTA (_____). Sustainable Agriculture and Food Security Program Fiscal Year 2014 - Work Plan October 2013 - September 2014 2.12 UNOPS/RUTA (_____). Sustainable Agriculture and Food Security Program. Annual Report October 2013 - September 2014. 2.13 UNOPS/RUTA (_____). Programa Agroalimentario Sostenible. Fiscal Year 2015 - Work Plan October 2014 - September 2015. 2.14 UNOPS/RUTA (_____). Sustainable Agriculture and Food Security Program. Annual Report October 2014 - September 2015. 2.15 UNOPS/RUTA (_____). Sustainable Agriculture and Food Security Program. Annual Report, Fiscal Year 2016, October 2015 – September 2016. 2.16 Secretaría de Agricultura y Ganadería – SAG (June, 2011). Plan de Inversión del País para el Sector Agroalimentario (PIPSA). Honduras, 2011-2014. 2.17 UNOPS/RUTA (October 2013). Programa Agroalimentario Sostenible: Contexto y Problemática del Café en la Región Trifinio | Conformación de la Mesa de Café de la Región Trifinio: Documento de Trabajo. 2.18 UNOPS/RUTA (October 2013). Programa Agroalimentario Sostenible: Propuesta de Organización y Funcionamiento de la Mesa de Café de la Región Trifinio / Conformación de la Mesa de Café de la Región Trifinio. 2.19 UNOPS/RUTA (October 2013). Programa Agroalimentario Sostenible: Plan Estratégico 2014 – 2018 de la Mesa de Café de la Región Trifinio / Conformación de la Mesa de Café de la Región Trifinio. 2.20 UNOPS/RUTA (November 2013). Programa Agroalimentario Sostenible. Uso actual y oferta de tecnologías sostenibles en las cadenas de valor del cacao en Honduras para mejorar la seguridad alimentaria - Informe Nacional Honduras. 2.21 UNOPS/RUTA (November 2013). Programa Agroalimentario Sostenible. Uso actual y oferta de tecnologías sostenibles en las cadenas de valor del cacao en Honduras para mejorar la seguridad alimentaria - Informe Nacional Nicaragua. 2.22 UNOPS/RUTA (November 2013). Programa Agroalimentario Sostenible. Uso actual y oferta de tecnologías sostenibles en las cadenas de valor del cacao en Honduras para mejorar la seguridad alimentaria - Informe Nacional El Salvador. 2.23 UNOPS/RUTA (November 2013). Programa Agroalimentario Sostenible. Uso actual y oferta de tecnologías sostenibles en las cadenas de valor del cacao en Honduras para mejorar la seguridad alimentaria - Informe Nacional Guatemala. 175 2.24 UNOPS/RUTA (November 2013). Programa Agroalimentario Sostenible: Comparación de Guías Metodológicas para la Implementación de Comunidades de Práctica. 2.25 UNOPS/RUTA (December 2013). Programa Agroalimentario Sostenible. Oferta mundial de tecnologías de producción de cacao prioritarias para elevar los rendimientos, mejorar la calidad de cacao y asegurar la sostenibilidad y seguridad alimentaria de las familias cacaoteras de Centroamérica. 2.26 UNOPS/RUTA (January 2014). Programa Agroalimentario Sostenible: Conformación de la Mesa de Café / Resumen. 2.27 UNOPS/RUTA (March 2014). Programa Agroalimentario Sostenible: Delicias de Cacao. 2.28 UNOPS/RUTA (June 2014). Programa Agroalimentario Sostenible: Políticas sectoriales e innovación en la agricultura - Experiencias y lecciones para fomentar productos estratégicos para la seguridad alimentaria en Centroamérica. 2.29 UNOPS/RUTA (June 2014). Programa Agroalimentario Sostenible. Análisis de los flujos comerciales agropecuarios en Centroamérica y República Dominicana: vinculación con la agricultura familiar y la seguridad alimentaria. 2.30 UNOPS/RUTA (June 2014). Programa Agroalimentario Sostenible. Políticas sectoriales e innovación en la agricultura: Experiencias y lecciones para fomentar productos estratégicos para la seguridad alimentaria en Centroamérica 2.31 UNOPS/RUTA (June 2014). Programa Agroalimentario Sostenible. Promoción de Tecnologías Sostenibles para la Cadena de valor del Cacao en Guatemala, Honduras, El Salvador y Nicaragua Fase I. 2.32 UNOPS/RUTA (September 2014). Programa Agroalimentario Sostenible. Promoción de Tecnologías Sostenibles para la Cadena de valor del Cacao en Guatemala, Honduras, El Salvador y Nicaragua Fase II. 2.33 UNOPS/RUTA (May 2015). Programa Agroalimentario Sostenible. 3 Propuestas de política para mejorar la productividad del maíz en el Corredor Seco de Honduras: Con la aplicación de la Matriz de Análisis de Políticas (MAP). 2.34 Angélica Patricia González, Oscar Alberto Martínez, René Francisco Núñez, José Miguel Cambara, y Willians Alfredo Vásquez (May 2016). Diplomado en Gestión de Políticas Públicas para la Mejora de la Seguridad Alimentaria. Fortalecimiento agroproductivo bajo riego para contribuir a la seguridad alimentaria en la zona occidental de El Salvador”. 2.35 Eva Romero, Xenia de Morán, Oscar Aguilar, Carlos Juárez, y Ricardo Flores (May 2016). Diplomado en Gestión de Políticas Públicas para la Mejora de la Seguridad Alimentaria. Propuesta de “Programa de Fortalecimiento a las capacidades estatales para la coordinación interinstitucional en el desarrollo de las Cadenas de Valor”, El Salvador. 2.36 Marco Loarca, Jaime Alvarado, Ricardo Bonilla, Rene Ortíz, José Reinerio Carranza, Nery Herrera (May 2016). Diplomado en Gestión de Políticas Públicas para la Mejora de la Seguridad Alimentaria. Tema: Evaluación del Programa de Alimentación y Salud Escolar PASE/SAN/PAF en el municipio de Atiquizaya, departamento de Ahuachapán, El Salvador. 2.37 Guillermo Pérez, Gloria Alicia Recinos, Silvia Evelyn Garcia de Rivera, y Gloria Guadalupe Alas de Rodríguez (May 2016). Diplomado en Gestión de Políticas Públicas para la Mejora de la Seguridad Alimentaria. La cadena de valor de cacao, una iniciativa de interés nacional para la seguridad alimentaria y nutricional. El Salvador. 2.38 Rebeca Sánchez, María Eugenia Núñez, Carmen Elena Estrada, Diana Carolina Castro, José Fredy Franco Rivera, y Patricia Olivares de Flores (May 2016). Diplomado en Gestión de Políticas Públicas para la Mejora de la Seguridad Alimentaria. Política de compras públicas a productores familiares con prácticas sostenibles, para abastecimiento del programa de alimentación y salud escolar. El Salvador. 2.39 Adalberto Blanco Fuentes (FECORACEN), Ángel Ernesto Beltrán (UNES), Mirna Yanet Morales (CNAF), María Candelaria Navas (Las Mélidas), y José Margarito Nolasco (FMLN) (_____). Diplomado en Gestión de Políticas Públicas para la Mejora de la Seguridad Alimentaria. Lineamientos generales: 176 Política Pública de Impulso a la producción Agroecológica de alimentos en la Agricultura Familiar Campesina, El Salvador. 2.40 María Ivette Lemus Barrera, Evelyn Corina García Ramos, Sergio René González Orozco, Marco Tulio Díaz Santos, Rossana Elizabeth Gómez Álvarez, Amparo Elvira Samayoa Morales, y Selvin Geovani Santizo López (_____). Diplomado en Gestión de Políticas Públicas para la Mejora de la Seguridad Alimentaria. Propuesta de la gestión, seguimiento y evaluación de la política nacional de seguridad alimentaria y nutricional en el marco del eje “Bienestar de la gente” del Plan Nacional de Desarrollo “Nuestra Guatemala K’atun 2032”. 2.41 Lucero Luna, Aracelly Seijas, Roberto Castillo, Víctor Orellana, Henry Ortiz, Erick Blanco, Edwin Duque, Mario Morales, y José Moreno (August 2016). Diplomado en Gestión de Políticas Públicas para la Mejora de la Seguridad Alimentaria. Fortalecimiento del Sistema Nacional de Extensión Rural -SNER￾para contribuir a la disponibilidad de alimentos en siete departamentos priorizados con desnutrición crónica, Guatemala. 2.42 _____ (2016). Diplomado en Gestión de Políticas Públicas para la Mejora de la Seguridad Alimentaria. Propuesta Programa de creación y fortalecimiento de capacidades para el abordaje de los ejes transversales de la Política Nacional de Seguridad Alimentaria y Nutricional, Guatemala. 2.43 Adonay Cajas Maldonado, Ana Rosa Castillo Canjura, Blanca Rodríguez, Silvia Xinico, Marvin de Paz Ramírez, Michelle Paz Santisteban, y Walter Chacón Marroquín (_____). Diplomado en Gestión de Políticas Públicas para la Mejora de la Seguridad Alimentaria. Plan para auditoría social de programas y proyectos que se ejecutan en el marco del SINASAN. Caso: Ventana de los mil días. Guatemala. 2.44 UNOPS/RUTA (November 2016). Programa Agroalimentario Sostenible: Diplomado en Políticas Públicas para la Mejora de la Seguridad Alimentaria - La experiencia de Guatemala, Honduras y El Salvador. 2.45 Universidad Nacional Autónoma de Honduras / Facultad de Ciencias Sociales / Doctorado en Ciencias Sociales con Orientación en Gestión del Desarrollo (September 2016). Diplomado en Gestión de Políticas Públicas para la Mejora de la Seguridad Alimentaria. Propuesta política pública para la seguridad alimentaria y nutricional en Honduras 2016-2030. 2.46 UNOPS/RUTA (_____). Programa Agroalimentario Sostenible. Cadenas de Valor del Cacao - Almacén de Buenas Prácticas. http://www.ruta.org/programa-agroalimentario-sostenible/cacao￾buenaspracticas/ 2.47 Comisión Trinacional del Plan Trifinio (October 1997). Tratado entre las Repúblicas de El Salvador, Guatemala y Honduras para la Ejecución del Plan Trifinio. 2.48 Comisión Trinacional del Plan Trifinio (November 2016). 30 Años de integración: Trabajando para el desarrollo sostenible fronterizo. 2.49 UNOPS/RUTA (October 2013). Programa Agroalimentario Sostenible. Plan Operativo 2014 – 2018 de la Mesa de Café de la Región Trifinio | Conformación de la Mesa de Café de la Región Trifinio. 3. RTMA 3.1 USAID (_____). Statement of Work (SOW) of the Final Performance Evaluation of the Regional Integrated Trade and Food Security Strategy 2011-2017. 3.2 USAID (July 2012). Request for Proposals No. SOL-596-12-000004 Regional Integrated Trade and Food Security Program (includes: SECTION B – supplies or services and price/costs, SECTION C – Description / Specifications/Statement of Work, SECTION D – Packaging and marking, SECTION E - Inspection and acceptance, SECTION F – Deliveries or performance, SECTION G – contract administration data, SECTION H – Special contract requirements, SECTION I – Contract clauses, and SECTION J – List of attachments). 177 3.3 Nathan Associates Inc. (August 2012). Technical Proposal of the Central America Regional Integrated Trade and Food Security Program. 3.4 Nathan Associates Inc. (_____). Annex C - Performance Management Plan. 3.5 Nathan Associates Inc. (_____). Attachment 3 – Results Framework. 3.6 Nathan Associates Inc. (_____). Attachment 4 – Logical Framework. 3.7 Nathan Associates Inc. (April 2013). Contract No. AID-596-C-13-00001 Regional Trade and Market Alliances Activity. 3.8 Nathan Associates Inc. (September 2013). Regional Trade and Market Alliances Activity. First Annual Work Plan. 3.9 Nathan Associates Inc. (November 2013). Regional Trade and Market Alliances Activity. Quarterly Report FY01/ QR-02 August-October 2013. 3.10 Nathan Associates Inc. (January 2014). Regional Trade and Market Alliances Activity. Quarterly Report FY01/ QR-03 November-December 2013. 3.11 Nathan Associates Inc. (March 2014). Regional Trade and Market Alliances Activity. Second Annual Work Plan. 3.12 Nathan Associates Inc. (March 2014). Regional Trade and Market Alliances Activity. Monitoring and Evaluation Plan. 3.13 Nathan (May 2014). Regional Trade and Market Alliances Activity: January-March 2014 Quarterly Report and Summary of Year 1 Activity. 3.14 Nathan Associates Inc. (February 2014). Regional Trade and Market Alliances Activity. Grants Management Manual Volume I. 3.15 Nathan Associates Inc. (February 2014). Regional Trade and Market Alliances Activity. Grants Management Manual Volume II. 3.16 Nathan Associates Inc. (May 2014). USAID Regional Trade and Market Alliances Activity. Data Quality Assessments (DQA). 3.17 Nathan Associates Inc. (June 2014). Regional Trade and Market Alliances Activity. Manual de Auditorías. 3.18 Nathan Associates Inc. (July 2014). Regional Trade and Market Alliances Activity. Quarterly Report FY02/ QR-01 April-June 2014. 3.19 Nathan Associates Inc. (October 2014). Regional Trade and Market Alliances Activity. Quarterly Report July – September 2014. 3.20 Francisco A. Hernández (December, 2014). Proyecto Regional de USAID para Comerio y Alianzas de Mercado. Monitoreo y Evaluacion. PowerPoint Presentation. 3.21 Nathan Associates Inc. (January 2015). Regional Trade and Market Alliances Activity. Quarterly Report October – December 2014. 3.22 Nathan Associates Inc. (April 2015). Regional Trade and Market Alliances Activity. Third Annual Work Plan. 3.23 Nathan Associates Inc. (May 2015). Regional Trade and Market Alliances Activity. Quarterly Report January – March 2015. 3.24 Nathan Associates Inc. (May 2015). Proyecto Regional de USAID para Comercio y Alianzas de Mercado. Manual de Indicadores. 3.25 USAID (June, 2015). Regional Trade and Market Alliances Activity: USAID / ACTIVITY FACT SHEET. 3.26 Nathan Associates Inc. (July 2015). Regional Trade and Market Alliances Activity. Quarterly Report April – June 2015. 3.27 Nathan Associates Inc. (September 2015). Regional Trade and Market Alliances Activity. Quarterly Report July – September 2015. 178 3.28 Nathan Associates Inc. (October 2015). Regional Trade and Market Alliances Activity. Reunión de Rendición de Cuentas – El Salvador. 3.29 Nathan Associates Inc. (October 2015). Regional Trade and Market Alliances Activity. Reunión de Rendición de Cuentas Componente 2 – El Salvador. 3.30 Nathan Associates Inc. (October 2015). Regional Trade and Market Alliances Activity. Reunión de Rendición de Cuentas – Guatemala. 3.31 Nathan Associates Inc. (October 2015). Regional Trade and Market Alliances Activity. Reunión de Rendición de Cuentas – Honduras. 3.32 Nathan Associates Inc. (October 2015). Regional Trade and Market Alliances Activity. Reunión de Rendición de Cuentas – Nicaragua. 3.33 Nathan Associates Inc. (January 2016). Regional Trade and Market Alliances Activity. Quarterly Report October – December 2015. 3.34 Nathan Associates Inc. (January 2016). Regional Trade and Market Alliances Activity. Reunión de Rendición de Cuentas Componente 2 - El Salvador. 3.35 Nathan Associates Inc. (February 2016). Regional Trade and Market Alliances Activity. Reunión de Rendición de Cuentas Componente 2 - El Salvador. 3.36 Nathan Associates Inc. (January 2016). Regional Trade and Market Alliances Activity. Reunión de Rendición de Cuentas - Guatemala. 3.37 Nathan Associates Inc. (January 2016). Regional Trade and Market Alliances Activity. Reunión de Rendición de Cuentas - Honduras. 3.38 Nathan Associates Inc. (April 2016). Regional Trade and Market Alliances Activity. Fourth Annual Work Plan. 3.39 Nathan Associates Inc. (July 2016). Regional Trade and Market Alliances Activity. Quarterly Report April – June 2016. 3.40 Nathan Associates Inc. (November 2016). Regional Trade and Market Alliances Activity. Monitoring and Evaluation Plan. 3.41 Nathan Associates Inc. (November 2016). Regional Trade and Market Alliances Activity. Quarterly Report Nathan Associates Inc. (January 2017). Regional Trade and Market Alliances Activity. January– March 2014 Quarterly Report and Summary of Year 1 activity.July – September 2016. 3.42 Nathan Associates Inc. (July 2016). Regional Trade and Market Alliances Activity. Quinta Reunión de Rendición de Cuentas – Guatemala. 3.43 Nathan Associates Inc. (June 2016). Regional Trade and Market Alliances Activity. Reunión de Rendición de Cuentas – El Salvador. 3.44 Nathan Associates Inc. (May 2016). Regional Trade and Market Alliances Activity. Reunión de Rendición de Cuentas – Honduras. 3.45 Nathan Associates Inc. (June 2016). Regional Trade and Market Alliances Activity. Reunión de Rendición de Cuentas – Nicaragua. 3.46 Nathan Associates Inc. (October 2016). USAID Regional Trade and Market Alliances Activity. Data Quality Assessments (DQA). 3.47 USAID / Office of Inspector General (October 28, 2016). AUDIT REPORT NO. 1-519-17-001-P of the Regional Trade and Market Alliances Activity – USDA 3.48 Nathan Associates Inc. (January 2017). Proyecto Regional de USAID para Comercio y Alianzas de Mercado. Organigrama. 3.49 Nathan Associates Inc. (February 2017). Proyecto Regional de USAID para Comercio y Alianzas de Mercado. Organigrama. 3.50 Nathan Associates Inc. (_____). Proyecto Regional de USAID para Comercio y Alianzas de Mercado. Presentación PowerPoint sobre el Proyecto. 179 3.51 Nathan Associates Inc. (_____). USAID Regional Trade and Market Alliances Activity. PowerPoint Presentation about the Activity. 4. IFC 4.1 USAID (_____). Program Description of the Central America Agribusiness Trade Logistics activity. 4.2 USAID (_____). Semester Progress Report January – June 2016. 4.3 USAID (_____). Semester Progress Report July – December 2015. 4.4 USAID (_____). Semester Progress Report January – June 2015. 4.5 USAID (_____). Semester Progress Report January – June 2014. 4.6 USAID (_____). Semester Progress Report July – December 2014. 4.7 USAID (March 27, 2015). IFC Grant Amendment. 4.8 USAID (September 30, 2015). IFC Grant Amendment. 4.9 USAID (October 11, 2013). IFC Grant Amendment 2013. 4.10 USAID/CATL (_____). Work Plan Year III January – December 2016. 4.11 USAID/CATL (_____). Work Plan Year II January – December 2015. 4.12 USAID/CATL (_____). Work Plan Year I January – December 2014. 4.13 USAID (_____). Central America Regional Agribusiness Trade Logistics Activity Proposal Final 2013. 4.14 USAID (_____). Central America Regional Agribusiness Trade Logistics Activity Indicators. 4.15 USAID (____). Central America Regional Agribusiness Trade Logistics Activity Proposal Final 2013. 4.16 USAID/CATL (March 28, 2014). Monitoring and Evaluation Plan. 4.17 WBG/IFC (January 2017). Budget and Financial Plan JAN17. 4.18 IFC(September 2016). Sistema de Integración Regional para Registros Sanitarios. Resumen de Proyecto. 4.19 IFC/CATL (June 28, 2016). Annex to Back To Office Report, Roatán, Honduras, Media coverage: Launch of the regional IT system for the recognition of sanitary registration in Central America. 4.20 SIECA (September 2016). Néstor Domínguez. Alcances y Gobernabilidad del Sistema Regional. Presentación Power Point acerca del proyecto. 4.21 WBG (November 15, 2016). Sistema de Integración Regional para el reconocimiento mutuo de registros sanitarios de alimentos y bebidas de Centroamérica. Informe Visita Guatemala. 4.22 WBG (September 28, 2016). Sistema de Integración Regional para el reconocimiento mutuo de registros sanitarios de alimentos y bebidas de Centroamérica. Informe Visita El Salvador. 4.23 WBG (September 13, 2016). Sistema de Integración Regional para el reconocimiento mutuo de registros sanitarios de alimentos y bebidas de Centroamérica. Informe Visita Honduras. 4.24 WBG/Ministerio de Salud El Salvador (September 28, 2016). Lista de Participantes Taller de Socialización Sistema de Gestión y Control de alimentos y bebidas de Centroamérica. 4.25 WBG/Ministerio de Salud El Salvador (September 28, 2016). Agenda Taller de Socialización Sistema de Gestión y Control de alimentos y bebidas de Centroamérica. 5. RITFS STRATEGY 5.1 USAID (September 2011). Regional Integrated Trade and Food Security Strategy 2011-2017. 6. GENDER 6.1 USAID (_____). SOW Final Performance Evaluation Regional Integrated Trade and Food Security Strategy 2011-2017: Estrategia de evaluación para el tema transversal de género. 6.2 USAID (March 2012). Gender Equality and Female Empowerment Policy. 180 6.3 USAID (July 2013). Integrating Gender Equality and Female Empowerment in USAID´s Program Cycle, ADS Chapter 205. 6.4 USAID (_____). Iniciativa de USAID de Monitoreo, Evaluacion y Aprendizaje: Transversalización del enfoque de género e inclusión en el ciclo de proyectos (PowerPoint presentation) 7. GENERAL 7.1 Chemonics International, Inc. (March 2006). Central America and Dominican Republic Quality Coffee Program (CADR QCP). Final Report. 7.2 FAO (______). La Contribución de los Programas PESA de Centroamérica a la Seguridad Alimentaria Nutricional: Logros, Lecciones Aprendidas y Perspectivas 2005 – 2015 7.3 USAID (January 2011). USAID Evaluation Policy – EVALUATION Learning from Experience. 7.4 _____ (August, 2011). Evaluation of the Regional and Bilateral Programs to Support Trade Compliance under CAFTA-DR: Final Report. 7.5 USAID (October 2012), Guatemala Evaluaciones Finales de Cuatro Proyectos de la Oficina de Crecimiento Económico: Informe Final 7.6 USAID (November 2013). USAID Technical Note Focus Group Interviews, Technical Note, Version 1.0. 7.7 USAID (September 2014). Plan of Alliance for Prosperity in the Northern Triangle: A Road Map. 7.8 USAID (_____). Feed the Future Learning Agenda. 7.9 USAID (October 2014). Feed the Future Indicator Handbook: Definition Sheets. 7.10 USAID (_____). Regional Development Cooperation Strategy 2015-2019. 7.11 USAID (_____). Increasing Regional Economic Integration Activity: ANNEX C - Logical Framework, and ANNEX L - Monitoring and Evaluation (M&E) Plan 7.12 _____ (October 2015). Estrategia Centroamericana de Facilitación del Comercio y Competitividad con Énfasis en Gestión Coordinada de Fronteras. 7.13 USAID (February 2016). Economic Growth Bilateral and Regional Program, Economic Growth Office, El Salvador. 7.14 USAID (10/12/2016) USAID Program Cycle Operational Policy, ADS Chapter 201. 7.15 _____ (_____). INTRADE/BID Estimaciones de las Tendencias Comerciales América Latina y el Caribe, Edición 2017. 7.16 USAID (_____). CAFTA-DR Programs Evaluation. ANNEX VI: Scope of Work 7.17 USAID (_____). Sample Evaluation Report Template 7.18 TechnoServe (March 2014). Better Harvest (COSECHA) Program USAID Global Development Alliance. Application (RFA) No. RFA-596-14-000001, March 14, 2014. 7.19 TechnoServe (______). Better Coffee Harvest (Cosechemos Más Café) Activity Monitoring & Evaluation (M&E) Plan. 7.20 Luisa Arredondo, Gerardo Flores, Carla López, and Kate Díaz (October, 2016). Evaluación de Medio Término del Proyecto Cosechemos Más Café. 7.21 Mendez England and Associates – ME&A (_____). Resume MEL Initiative. 7.22 USAID (October 2016). Mid-term review Better Coffee Harvest Activity. 7.23 TechnoServe (October 31, 2016). Better Coffee Harvest Activity. Quarterly Report July – September 2016. 181 ANNEX 10. FOCUS GROUP DISCUSSIONS ON THE EFFECT OF THE RTMA ACTIVITY Introduction The SOW of the ongoing performance evaluation of the Regional Trade and Food Security Strategy, requested that the Evaluation Team conduct several focus groups with producer groups supported by the Regional Trade and Market Alliances (RTMA) Activity. USAID/CAM wanted to understand to what extent support from RTMA to producer groups has contributed to improved farm-level production, income, consumption, and nutrition in order to respond to a RIG finding in the RTMA audit. The Focus Groups Three focus groups were conducted as described below: Date Producer group name Product Country Place Participants No. Women No. 01/28/2 017 ACOPASCA Asociación Cooperativa de San Carlos Plantains El Salvador El Paisnal, San Salvador 12 6 02/10/2 017 ECARAI Empresa Campesina Agroindustrial de la Reforma Agraria de Intibucá – Honduras Potatoes Honduras La Esperanza, Intibucá 12 5 02/18/2 017 ARSAGRO Asociación Regional de Servicios Agropecuarios de Occidente Beans Honduras Danlí, El Paraíso 14 6 Methodology The Focus Group Discussions were organized with support of RTMA implementing partners and the leaders of the producer’s organizations. The main characteristics of the events are: Producers’ organizations were chosen for having received the most comprehensive support from the RTMA program beyond being grant recipients. Also, logistical aspects such as the organizations willingness to collaborate, travel distance, meeting facilities, availability of the producers, including female producers, were also considered. Were carried out on Saturdays to facilitate producers’ attendance. Started with introductions, an explanation of the activity, a general discussion, and then separated groups of men and women. Were combined with a site visit so that the evaluation team could observe the facilities, machinery and equipment the activity provided. Asked for attendance of both directors and members of the organizations. Requested the participation of women. Had questions specifically intended for female producers. Were conducted in Spanish. 182 The Activities USAID/CAM wants to learn about the producers’ group participation in the RTMA activity; how it went, the support they received from RTMA, and what production changes were implemented, leading to what results. The RTMA program operated through implementing partners: FUSADES in El Salvador, FUNDER in Honduras, and AGEXPORT in Guatemala. In this respect, it must be noted that the program focused itself in the grants (agro-industrial investment component), but did not ever work directly with the producers. Farm-level activities, such as training and technical assistance, were carried out by the implementing partners per their own initiative and using resources other than those of RTMA. ACOPASCA Investment The investment/grant of the RTMA activity in ACOPASCA amounts to $249,157.92. For its part, the Association with support of the municipality of El Paisnal, invested $207,988.13 in complementary infrastructure. Components of the project were: 1) Equipment for plantain washing, classification and packaging, 2) Equipment for fruit dehydration, 3) Salaries for administrative staff and plant personnel at various times between September 2015, and April 2016, 4) Consulting fees: For the project coordinator, 5) Training: in manufacturing subjects ranging from management, operations, food hygiene, organizational leadership, costs calculation, fruit packaging, plant management, accounting software, as well as training experiences in Guatemala and Honduras, 6) Participation in Trade events in Guatemala and Honduras, 7) Training in a wide range of good agricultural practices. Implementation The agroindustry infrastructure was inaugurated in February 2016 but in its first year it has been used at much less than full capacity, as only the plantain washing pool is being utilized but not the classifying and packaging equipment; whereas the fruit dehydration equipment has worked only a few times to provide product to the local school. The major obstacles to a full plant exploitation have been: 1) lack of plantain export permit by the authorities, and 2) lack of sanitary registration for the dehydrated fruit. ARSAGRO Investment The ARSAGRO project consisted in purchasing and having installed equipment for cleaning and classifying beans. Total investment was $315,000 with a USAID contribution of $233,000 and $82,000 provided by the association. The project components included: 1) technological innovation, cleaning, polishing and packaging equipment for beans, 2) assistance on commercial Strategy, 3) administrative training for the association leadership team, 4) institutional strengthening, 5) training in a wide range of good agricultural practices. Implementation The bean processing equipment started production in December 2016 and has been used since then to process beans provided not only by ARSAGRO’s associates but other producers. The plant has a processing capacity of over 8,000 ton/year, which opens the option to provide the services to third parties. 183 The new processing line resulted in a cost reduction by eliminating 60 jobs of women who used to work on manual bean selection and classification. Likewise, the polishing machine yields a higher quality product which is better accepted by the market. ECARAI Investment The project invested $492,434, of which USAID funded $313,276.91 (63%); whereas ECARAI contributed $179,157.26 (37%). The major part was for purchasing machinery and equipment for potato brushing, classifying, washing, and packaging. Besides, the project included: 1) training to avoid post-harvest losses, 2) improvement in potato washing practices, 3) product differentiation in the market, 4) improvement in potato quality, through technical assistance at the farm level by FUNDER and the Escuela Agrícola Panamericana (Zamorano). Implementation The potato processing equipment started operating in December 2015 and has been in continuous operation since then. The plant is the final step of a comprehensive improvement on port-harvesting practices implemented by the producers and ECARAI, the local cooperatives, and the producers themselves. Those practices include better manipulation, washing, drying, adoption of better potato varieties, all of which has led to reducing costs, decreased product rejection, and job creation. Effect of the projects on personal income and consumption ACOPASCA Since neither the plantain processing plant nor the fruit dehydration plant have worked in full operating and commercial capacity yet, no new income coming from them has been received by the producers. However, the expectation of a greater plantain processing capacity has led ACOPASCA to increase their plantain￾cultivated area which in turn has meant a greater income coming from agricultural work. Likewise, from its overall net annual profits, ACOPASCA every April distributes bonuses to its members. In 2016 it was $2,000. ARSAGRO ARSAGRO’s leaders stated that before the project, they didn´t even know what the association’s profits were. Now, thanks to SIFER, the information system, they have information “that even peasants like us, can understand”. Last year (2015-2016) ARSAGRO had around $21,000 in profits and this year (2016 – 2017) are expecting profits over $43,000. Since the processing plant acquired by the project started in December 2016, the accounting cycle is still ongoing, and therefore the association members have not yet obtained earnings derived from the project so far. For ARSAGRO those profits are a reasonable expectation yet to be materialized. On the other hand, for those 60 women who were laid off, the project meant lost jobs and income. ECARAI For 2016 ECARAI had a sales goal of $1,652,000, which was exceeded. According to the producers, this has been a successful project that has contributed to the company being able to position itself much better in the market by offering a more differentiated product and with better quality and better prices. 184 Effect of the Projects on Living Standards ACOPASCA Although the producers have received no income directly from the infrastructure donated by the project (the largest activity of the cooperative is sugar cane growing), the annual bonuses received have been used by families on things such as home improvement, family health expenses, and support for their children’s education. The usual diet of these families includes meat once a week, chicken (3 lb. biweekly) as well as beans and squash that are grown in community, and corn for self-consumption. Overall no significant changes in consumption patterns have been experienced by the families. ARSAGRO As stated by the female producers, there are not tangible changes in income at individual level, or in the families’ consumption patterns attributable to the project. The family´s usual diet remains the same: beans, eggs, rice, and homegrown chicken. ECARAI With the additional income derived from the project, the producers say they have improved their homes, given their children better education, increased their crops, purchased larger properties for agriculture purposes, and improved their health and kinds of foods consumed. “We insist to our members to eat local products, including vegetables we produce, and avoid canned food.” General Conclusions The development of each one of the projects funded by RTMA is different. They range from those which have not even started operating yet, despite their equipment being in place for some time, to those whose production has started as recently as December 2016, and therefore do not count for the effect of this evaluation, since the evaluation period goes until September 2016. Concerns on project implementation arise from situations where costly infrastructure has been inoperative for months due to absence of electric power, flawed equipment purchasing decisions, lack of government permits and obstacles to market access, including government red tape, trade and food security policies, and price controls. Corruption in various sectors and levels is mentioned as an impeding factor as well. The RTMA activity was intended to work with farmers’ groups (associations, companies), not at farm-level or directly with producers, therefore the income benefits to farmers will be necessarily indirect; and, if present, have not yet reached farmers’ pockets. At the time of this evaluation, ARSAGRO was expecting profits from the project at the end of the 2016-2017 accounting cycle. The previous observation highlights a significant contrast in the RITFS design versus some of its expectations. For the most part, the Strategy was conceived to act, particularly on trade, at the institutional or organizational level, not farm-level. However, on the food security and nutritional side of it, the Strategy seeks to match Feed the Future expectations regarding improvement of income at the farm level. This is a mismatch between high level regional purposes and the activity-level means assigned to attain them. 185 Out of the three producers’ group, one has been able to successfully increase their profits so that a greater income improves producers’ living conditions. The other two groups have not obtained direct additional income from the RTMA projects, and so their families´ spending and consumption patterns remain unchanged.