i PERFORMANCE EVALUATION OF USAID OCEANS AND FISHERIES PARTNERSHIP FINAL REPORT August 6, 2019 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Social Impact, Inc and authored by Alan Ferguson, Senior Team Leader, and Michael Duthie, Senior M&E Specialist. ii (DELETE THIS BLANK PAGE AFTER CREATING PDF. IT’S HERE TO MAKE FACING PAGES AND LEFT/RIGHT PAGE NUMBERS SEQUENCE CORRECTLY IN WORD. BE CAREFUL TO NOT DELETE THIS SECTION BREAK EITHER, UNTIL AFTER YOU HAVE GENERATED A FINAL PDF. IT WILL THROW OFF THE LEFT/RIGHT PAGE LAYOUT.) iii PERFORMANCE EVALUATION OF USAID OCEANS AND FISHERIES PARTNERSHIP FINAL REPORT August 6, 2019 Evaluation Requisition No.: REQM-486-18-000277 Task Order No.: 72048618F00003/P00001 Contract No. AID-486-I-14-00001 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Social Impact, Inc and authored by Alan Ferguson, Senior Team Leader, and Michael Duthie, Senior M&E Specialist. DISCLAIMER The authors' views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ACKNOWLEDGEMENTS The Evaluation Team (ET) is grateful for the support provided by staff at USAID, USAID Oceans, Tetra Tech ARD, as well as staff and partners at the Southeast Asian Fisheries Development Center (SEAFDEC) and the Coral Reefs, Fisheries, and Food Security (CTI-CFF). The ET is also thankful for all logistical guidance provided by staff at USAID in Thailand, Indonesia, the Philippines, and Malaysia during qualitative data collection for this evaluation. iv TABLE OF CONTENTS FIGURES AND TABLES V ACRONYMS AND ABBREVIATIONS VI EXECUTIVE SUMMARY VIII Evaluation Findings and Conclusions viii Recommendations xi INTRODUCTION 1 METHODOLOGY 4 Evaluation Scope of Work 4 Evaluation Design and Methods 6 Limitations and Mitigation Strategies 7 FINDINGS 9 Evaluation Question 1 9 Evaluation Question 2 29 Evaluation Question 3 33 CONCLUSIONS AND RECOMMENDATIONS 37 Conclusions 37 Recommendations 40 ANNEX 1: SCOPE OF WORK 45 ANNEX 2: USAID OCEANS LEARNING SITES RESULTS CHAINS 57 ANNEX 3: EVALUATION SCHEDULE 58 ANNEX 4: KEY USAID OCEANS’S PARTNERS BY TYPE, DIFFERENT ENGAGEMENT MECHANISMS, AND THE PARTNER LEVEL 59 ANNEX 5: CONSENT SCRIPTS AND INTERVIEW GUIDES 61 ANNEX 6: LIST OF EVALUATION RESPONDENTS BY CATEGORY 70 ANNEX 7: LIST OF DOCUMENTS REVIEWED 72 ANNEX 8: ACHIEVEMENTS AND EXPECTED OUTCOME/ OUTPUTS, 2016-2018 REPORTS 76 ANNEX 9: EVALUATION TEAM COMPOSITION 83 ANNEX 10: EVALUATION DESIGN MATRIX 85 v FIGURES AND TABLES Figure 1: Number of Respondents by Sex, Type, and Location 7 Table 1: Cumulative Achievements (as of Quarter 1 FY2019) 3 Table 2: Catch Documentation and Traceability (CDT) Technologies 16 Table 3: EAFM Plans under USAID Oceans, March 2019 26 vi ACRONYMS AND ABBREVIATIONS ACDS ASEAN Catch Documentation Scheme AP2HI Asosiasi Perikanan Pole and Line dan Hand Line Indonesia ARD Associates in Rural Development ASEAN Association of Southeast Asian Nations ASSP ASEAN-SEAFDEC Strategic Partnership BFAR Bureau of Fisheries and Aquatic Resources, Philippines CDT Catch documentation and traceability CDTS Catch documentation and traceability system CTE Critical tracking event CTI-CFF Coral Triangle Initiative for Coral Reefs, Fisheries and Food Security DEX Data exchange eACDS Electronic ASEAN Catch Documentation Scheme EAFM Ecosystem Approach to Fisheries Management eCDT Electronic catch documentation and traceability EU European Union FAD Fish Aggregation Devices FGD Focus group discussion FIS Fisheries Information System FMOT Fisheries Marketing Organization of Thailand HW/GE Human Welfare/Gender Equity IUU Illegal, unreported, and unregulated [fishing] IPNLF Indonesian Pole and Line Federation ICTA Indonesian Coastal Tuna Alliance KDE Key data element KII Key informant interview M&E Monitoring & Evaluation MDPI Yayasan Masyarakat dan Perikanan Indonesia MMAF Ministry of Maritime Affairs and Fisheries Indonesia NGO Non-government organization NOAA National Oceanic and Atmospheric Administration PE Performance Evaluation PPP Public-private partnership PSMA Port State Measures Agreement RAFMS Rapid appraisals of fisheries management systems RDMA Regional Development Mission for Asia vii SEAFDEC Southeast Asian Fisheries Development Center SFMP Sustainable Fisheries Management Plan SIMP Seafood Import Monitoring Program SOCKSARGEN Socskargen Federation of Fishing & Allied Industries, Inc. STELINA Sistem Teusur dan Logistik Ikan Nasional Indonesia TOC Theory of Change TWG Technical Working Group USAID United States Agency for International Development USAID Oceans USAID Oceans and Fisheries Partnership VMS Vessel Monitoring System viii EXECUTIVE SUMMARY The United States Agency for International Development’s Oceans and Fisheries Partnership (USAID Oceans) is being implemented by Tetra Tech ARD in collaboration with the Southeast Asian Fisheries Development Center (SEAFDEC) and other key stakeholders to strengthen regional cooperation to combat illegal, unreported, and unregulated (IUU) fishing, promote sustainable fisheries, and conserve marine biodiversity in the Asia-Pacific region. The $20M Activity, planned for 2015-2020, supports the development of transparent and financially sustainable electronic Catch Documentation and Traceability (eCDT) systems, first in two learning sites (General Santos City, Philippines and Bitung, Indonesia), followed by limited scale-up into expansion sites across the region. The USAID/Regional Development Mission for Asia (USAID/RDMA) has contracted Social Impact (SI) to conduct a Performance Evaluation (PE) of the USAID Oceans Activity, focused on three questions: Evaluation Question 1: To what extent have the USAID Oceans’ approaches achieved and/or advanced the USAID Oceans’ objectives on a) regional coordination, b) partnerships, and c) capacity building? Evaluation Question 2: What factors (both internal and external) have enhanced, diminished, or have no effect on achievement of USAID Oceans expected results in reducing IUU fishing and strengthening fisheries management? Evaluation Question 3: What evidence(s) exists that the participating Association of Southeast Asian Nations (ASEAN) countries, especially in the learning sites and a selected expansion site (Kelantan, Malaysia) would potentially sustain the adoption and implementation of the eCDT systems to counter IUU fishing and strengthen fisheries management as a result of USAID Oceans' support? The evaluation is intended to inform USAID, USAID Oceans, SEAFDEC, and other key stakeholders on the outcomes achieved thus far by USAID Oceans with the purpose of assisting the final stage of the Activity completion and contributing to lessons for future USAID activities. The evaluation team conducted extensive document review and completed 115 key informant interviews over four weeks of fieldwork in Thailand, Indonesia, the Philippines, and Malaysia. EVALUATION FINDINGS AND CONCLUSIONS USAID Oceans began under the assumption that a region-wide approach to eCDT could be developed but due to significant country-level differences in needs, capabilities, and existing systems, the Activity shifted to focusing on supporting the development of customized country￾specific systems, building on the systems being developed at the time by the governments of Indonesia and the Philippines. The evaluation found that USAID Oceans has made a significant and timely contribution to the design, development, and piloting of eCDTs; the development of Ecosystem Approach to Fisheries Management (EAFM) plans; and in raising awareness on human welfare and gender equity (HW/GE) including integrating it into fisheries management planning. A broad approach has been promoted that combines catch documentation and traceability, better data management for fish information systems, and flexible fisheries planning that encompasses both technical and social issues. USAID Oceans has helped to establish and catalyze interest in an ambitious agenda for transformative changes to fisheries management in the region and engaged a large range of partners and supporters. EVALUATION QUESTION 1 REGIONAL COORDINATION AND CAPACITY DEVELOPMENT The USAID Oceans approach to regional coordination has been generally effective in raising the profile of issues and gaps in eCDT development, presenting technical information to and training ix Technical Working Group (TWG) representatives from 11 countries.1 It has provided a timely platform for regional discussions of seafood traceability and sustainable fisheries management. Capacity development of SEAFDEC and the Secretariat of the Coral Triangle Initiative for Coral Reefs, Fisheries and Food Security (CTI-CFF) in leading and coordinating CDT development and dissemination has been limited by a lack of clear expectations for the roles to be served by these regional bodies and a lack of organizational capacity to sustain regional coordination, although training and the handover of many knowledge products may facilitate future programs. At the national level, USAID has contributed to raising knowledge and awareness and assisting development of CDT systems and HW/GE-integrated EAFM planning, particularly in the relevant ministries in the two learning site countries, Philippines and Indonesia. These national eCDT systems are still undergoing development, and relevant ministries in both countries expressed concerns about capacity and resources for their full implementation. PARTNERSHIPS USAID Oceans has established a wide range of partnerships in both the public and private sector, including productive partnerships with government ministries, global seafood companies, processors, suppliers, sector associations, non-governmental organizations (NGOs), and academic institutions. This has added to the regional momentum for action on seafood traceability. A central theme of USAID Oceans’ partnership approach has been to encourage industry (suppliers, processors, buyers) to invest in eCDT systems that will improve the efficiency of their operations and regulatory compliance. In this regard, many of the prospective industry partners at the learning sites do not yet appear ready to invest due to a lack of evidence about technology advantages and return on investment. However, the business case evidence is currently being prepared and disseminated to enhance industry adoption on the basis of market competitiveness, operational and production efficiencies, and regulatory compliance. ECDT SYSTEMS USAID Oceans has supported the development of national eCDT systems and complementary private sector technologies in both Indonesia and the Philippines. In Indonesia, this has comprised supporting development of the Government of Indonesia’s national systems (e-logbook and Stellina) and three private sector technologies (TraceTales, Trafiz, and Pointrek VMS). In the Philippines, USAID Oceans supported development of the Bureau of Fisheries and Aquatic Resources national eCDT system and one private sector technology (Fame transponders). Although the national systems are still undergoing testing and refinement, USAID Oceans expects the national systems to be ready for initial roll out before the end of the Activity in 2020. The private sector technologies for recording fish catch and tracking key data through the value chain have included 17 ‘early mover’ fishing and processing companies testing the technologies in a few supply chains, but at the time of evaluation data collection, the first mover companies interviewed noted that systems were not yet integrated with the national systems. In Indonesia, USAID Oceans expects to complete a full eCDT system with 1-2 supply chains in Bitung though this requires further work on both the national system and the private sector technologies. In the Philippines, completion of the national system and the complementary technology will still leave gaps on both the catch and processing side. While the completion of a full eCDT system even in one value chain represents a significant achievement, it still leaves the more complicated task of institutionalization and roll out in different supply chains, fisheries, and geographies, which USAID Oceans notes is the responsibility of national governments. This is a long-term process and staff of the responsible government agencies expressed concerns about their current capacity to fully implement the systems following this testing phase and beyond the time frame of USAID Oceans. 1 Philippines, Indonesia, Thailand, Malaysia, Cambodia, Vietnam, Lao PDR, Myanmar, Timor-Leste, Papua New Guinea and Solomon Islands. x ECOSYSTEM APPROACH TO FISHERIES MANAGEMENT (EAFM) The EAFM capacity development has been greatly appreciated by nearly all stakeholder groups, providing skills development, resource materials, and substantive stakeholder buy-in for the development of six EAFM plans completed to date and involving Philippines, Indonesia, Malaysia and Thailand. The integration of human welfare and gender is a distinguishing feature, as is the connection to the improved catch data that will emerge from the development of eCDT systems. All of the plans were reports as approved or in the advanced stages of completion. The central question is whether there is sufficient capacity to continue the momentum and to effectively utilize the plans for fisheries management decision-making. Implementation effectiveness will therefore be dependent on the readiness and capacity of the participating authorities and local government units (LGUs), and their ability to mobilize commitments and resources, something that is beyond the scope of the Activity. HUMAN WELFARE AND GENDER EQUITY (HW/GE) The HW/GE component has also added to the international efforts to change the fisheries and fisheries management toward a more socially responsible and inclusive industry in the region. Many of the stakeholders interviewed commented on how they now have a new perspective about these issues. Some of the organizations have also appointed gender officers to carry the issue forward. While USAID Oceans has clearly contributed to increased knowledge, awareness and reporting on these issues, substantive, measurable action within the fisheries sector is still needed in the future. EVALUATION QUESTION 2 IUU fishing was universally recognized as a complex issue involving multiple sectors and stakeholders at all levels: global, regional, national, and local. Accordingly, the ability to make significant reductions in IUU fishing are necessarily constrained within a USAID Activity of limited duration focused on technical assistance and technology development and testing. The most important Activity factor in relation to reducing IUU fishing is the implementation of an eCDT system. A fully developed and implemented eCDT system has the potential to significantly reduce IUU fishing and improve fisheries management, particularly in a context of supportive external factors (i.e. national policies and market pressures). The USAID Oceans Activity remains highly relevant and important to the overall goals of USAID in the region related to IUU fishing and fisheries management. The eCDT systems supported by USAID Oceans are still in the pilot stage with a relatively small number of first movers and undergoing ongoing refinements. If they become established over the next few years, the eCDT systems must be supplemented with other fisheries management efforts to maximally reduce the level of IUU fishing. Market pressures, government policies, and capacity for monitoring and enforcement are particularly important factors. EVALUATION QUESTION 3 The likelihood for sustainability depends on the component of the system (whether national eCDT system or one of the private sector technologies), the link in the supply chain, remaining Activity efforts, and external factors. The improved catch data from eCDT systems is also expected to assist in maintaining fisheries management plans and decision support on harvest strategies. The national eCDT systems may have the highest prospects for sustainability, but they also require further support in their final refinement, and more significantly, in their wider implementation and budgetary support. The discussions with national fisheries authorities suggested that the capacity and resources required for wider implementation are currently not in place in either learning site country. The sustainability of USAID Oceans-supported technologies is closely linked to the financial costs and incentives associated with their adoption. The evidence￾based business cases for investment by the public and private sectors are therefore important to maintaining viability of the technologies that are being introduced. Finally, the sustainability of all system components depends highly on continued market pressure for CDT. xi RECOMMENDATIONS FOR ECDT SYSTEM INSTITUTIONALIZATION AND EXPANSION RECOMMENDATIONS FOR IMPROVING REGIONAL COORDINATION RECOMMENDATIONS FOR EAFM INSTITUTIONALIZATION RECOMMENDATIONS Based on these findings and conclusions, the evaluation makes the following recommendations: • To facilitate eCDT up-take, USAID Oceans should share the results of the pilot eCDT activities with stakeholders in both the learning and expansion sites. • To support eCDT system institutionalization, USAID should support learning site governments in policy advocacy, capacity development, and budget planning and allocation. • To support eCDT system institutionalization, USAID should finance mechanisms for up-take of new technologies that facilitate eCDT. • To encourage support in expansion countries, USAID Oceans should clearly communicate the expansion site plans for all expansion sites. • USAID Oceans should hold government-industry workshops on the value and business case for public and private sector investment in eCDT in Philippines, and Indonesia as a basis for further advocacy within the region. • USAID should assist SEAFDEC in further developing the strategies for small-scale fishing communities to adopt eCDT mobile applications and pilot interventions that enhance access to markets, financing, value-added processing, and income diversification for small-scale fishing households. • To facilitate regional expansion and integration of eCDT systems, USAID Oceans should ensure the proposed eCDT guidelines are part of a process aimed at encouraging regional consensus on CDT best practices and lessons learned. • To encourage regional leadership, USAID should work with SEAFDEC to clarify their role as a ‘regional champion’ for eCDT and support their capacity development to fulfill that role. • USAID should consider the extent to which bilateral missions can support the capacity and resource requirements for implementation of the national eCDT systems and EAFM plans • To support EAFM institutionalization, USAID Oceans should identify the opportunities and needs for active implementation of the EAFM plans that have been produced and provide follow-up support and advocacy. • To optimize data-based fisheries management, USAID should disseminate and build capacity on eCDT – Fisheries Information System (FIS) integration. 1 INTRODUCTION The United States Agency for International Development’s Oceans and Fisheries Partnership (USAID Oceans) is being implemented by Tetra Tech ARD in collaboration with the Southeast Asian Fisheries Development Center (SEAFDEC), the Coral Triangle Initiative for Coral Reefs, Fisheries and Food Security (CTI-CFF), as well as a wide range of global, regional, national, and local partners who bolster the capacity for, and implementation of, Catch Documentation and Traceability (CDT) systems. The $20M Activity, planned for 2015-2020, supports the development of transparent and financially sustainable CDT systems to help ensure that fish are legally caught and properly labeled. The objectives of USAID Oceans are to: 1) Implement a financially sustainable CDT system that links to Fisheries Information Systems (FIS) and is demonstrated within a regional Ecosystem Approach to Fisheries Management (EAFM) framework2; 2) Expand use of CDT to areas important for biodiversity in the Association of Southeast Asian Nations (ASEAN) and Coral Triangle regions; 3) Strengthen the capacity of regional organizations to conserve biodiversity using EAFM and CDT to combat illegal, unreported, and unregulated (IUU) fishing; and 4) Engage the private sector to ensure sustainability and uptake of CDT and EAFM, while advancing regional fisheries governance. USAID Oceans partnership works with the Government of the Philippines and Government of Indonesia to implement and test CDT systems in the Activity learning sites. The Activity selected two demonstration (or ‘learning’) sites in General Santos City, Philippines and Bitung, Indonesia to pilot the CDT system development, implementation, and testing. USAID Oceans works closely with a lead local partner at each learning site - SOCSKSARGEN Federation of Fishing and Allied Industries, Inc. (SFFAII) in the Philippines and Yayasan Masyarakat dan Perikanan Indonesia (MDPI) in Indonesia - and has a country coordinator based in each learning site, with the Activity’s main office is based in Bangkok, Thailand. At both learning sites, USAID Oceans conducted assessments to identify the main export markets and an in-depth analysis to support EAFM plan development and human welfare/gender equity (HW/GE) interventions. The Activity has identified the CDT requirements of different stakeholders and engaged the private sector to implement traceability and ensure a financially sustainable CDT system. It has attempted to identify early adopters among private sector partners to test the systems. The Activity is now expanding its technical assistance to selected expansion sites in Vietnam, Thailand, and Malaysia. To clarify, Catch Documentation and Traceability (CDT) is the practice of documenting key information about the harvest, processing, and transportation of a fisheries product to enable traceability of the seafood product back through each step of its journey—from its import or point of final sale back to its harvest and point of origin. There are many tools that support eCDT through fishery supply chains, including USAID Oceans’ supported solutions that include private sector technologies like Sisfo’s “Pointrek” device (hardware; with supporting software), FAME’s radio frequency transponders (hardware; with supporting software), and Altermyth’s “Trafiz” (software; handheld and desktop application); national level tools like BFAR’s “National eCDT System” or Indonesia’s eLogbook and STELINA system (both hardware and software); and as well as SEAFDEC’s eACDS (software). 2 EAFM is a holistic and integrated approach to fisheries management that encompasses the relationships between sustainable fish stocks, ecosystems, habitats, communities and governance frameworks. 2 The Activity strategy focuses on four key components: 1) Regional coordination and capacity development among ASEAN and CTI-CFF member countries for strengthened regional capacity to support and sustain CDTS, Public-Private Partnerships (PPPs), and EAFM; 2) PPPs employed in the design and implementation of the CDT systems and FIS and management frameworks; 3) Capacity building and institutionalization of CDTS and EAFM at demonstration ‘learning sites’ in the Philippines, Indonesia, and elsewhere; and, 4) Gender and human welfare integration into CDT systems, fisheries management processes, and seafood supply chains at the demonstration learning sites. The USAID Oceans Theory of Change (TOC) for addressing IUU and unsustainable fishing as presented in Activity documents is as follows: a) If the CDTS is robust, meets stakeholder’s needs, and provides economic incentives to the fishing industry from the increased demand for and value of traceable fishery products, then it will be adopted by members of the private sector in partnership with government agencies throughout the region; and b) If fisheries managers use catch documentation and other tools to inform fisheries management plans and regulatory regimes, then local and national fisheries governance will be strengthened; and c) If regional capacity and cooperation is built to support EAFM and CDT, then more institutions and countries in the region will endorse and sustain their use; d) Then illegal, unreported, and unregulated fishing will be reduced, and marine ecosystems will be maintained. ACTIVITY IMPLEMENTATION STATUS The Activity commenced in May 2015 and is scheduled for completion in May 2020. A Pause and Reflect Workshop was held in February 2018 and provided a series of lessons learned based on the experience to date, as well as revisions to the learning sites theories of change (Annex 2). The workshop also listed “important learning questions to handle outside the internal Mid-term Review (MTR).” A subsequent MTR report in July 2018 concluded that: ● The TOC’s assumptions3 were “largely validated but not yet demonstrated”. ● Caveats exist about the role that economic incentives play in encouraging adoption. ● If the eCDT system requirements are too onerous and the benefits not tangible, producers, who are the most critical part of the supply chain, may not adopt eCDT. ● Current conditions and reported declining production may result in producers selling to countries in the region without import requirements, with great negative impact on processors in the supply chain who are advocates of eCDT systems. ● Stakeholders recognize the importance of eCDT to strengthen fisheries management; however, their understanding of how eCDT data can be used for fisheries management remains vague. ● Data sharing challenges among countries may limit the interoperability needed for transboundary management of migratory species. ● Stakeholders recognize the value of regional capacity and cooperation, but each country has different needs and capacities to move forward with eCDT systems and EAFM. 3 There are 12 assumptions stated in the Activity Monitoring, Evaluation, and Learning Plan (AMELP). 3 The MTR recommended restructuring remaining implementation plans into two “operational” workstreams to focus on: Regional Capacity and Cooperation and Learning Site Demonstration. It highlighted a need to harmonize terminology, standards, and designs; determine suitable incentives; and develop legacy products for ongoing sustainability. It also proposed prioritizing the demonstration and documentation of eCDT systems at the two learning sites, ensuring regional and national participation and expanding partnerships for use of eCDT data in fisheries management. The status of Activity achievements against Life of Award (LOA) targets set in the Activity Monitoring, Evaluation, and Learning Plan (AMELP) are summarized in Table 1. The training achievements are on-track, as are the amount of CDTS-secured seafood and the number of industry firms participating in CDTS systems in the region. However, many of the other indicators related to CDTS and fisheries governance reform are currently under-achieved. For additional information on results chains/frameworks and outputs and outcomes to-date, see Annexes 2 and 8 respectively. Table 1: Cumulative Achievements (as of Quarter 1 FY2019) RESULTS INDICATORS Cumulative FY 2016-194 LOA TARGET Marine Ecosystems Maintained Number of hectares of biologically significant areas under improved natural resource management as a result of USG assistance (Economic Growth (EG).10.2-2) 40.2M 62.3M Reduced IUU and Unsustainable Fishing Result (R)1. Amount of illegal fish entering regulated markets is reduced Number of CDTS-secured seafood units (metric ton) purchased (Custom) 25.5 24 R2. Industry self regulates to keep illegal products out of commerce No. of stakeholders participating in and submitting data voluntarily to the CTDS (Custom) 49 38 Adequate use of Intermediate Result (IR).1 Catch Documentation and Traceability System (CDTS)/ASEAN Catch Documentation Scheme (ACDS) implemented in selected learning sites in the Asia-Pacific region by 2020 1.1 CDTS components and functionality demonstrated Number of Critical Tracking Event (CTE) transactions submitted into the CTDS by targeted stakeholders (Custom) 133 15,000 1.2 CDTS design meets agency and key stakeholder needs Percentage of targeted stakeholders who self-report “adoption” of the CDTS as basic business requirement (Custom) 0 60% Adequate National & Local Fisheries Governance IR 2. Improved fisheries management Asia Pacific selected countries by 2020 2.1 Decision/ integration tools for key agencies and stakeholders in place and functioning Number of innovations supported through USG assistance (disaggregated by type) (RDMA IRS3 PM1) 1 6 2.2 Fisheries management plans & processes improved Number of laws, policies, or regulations that address biodiversity conservation and/or other environmental themes officially proposed, adopted, or implemented as a result of USG assistance (EG.10.2-5) 2 6 4 Data from Monitoring and Evaluation (M&E) Report for Q1 2019 (October – December 2018). 4 2.3 DTS/EAFM/ PPP processes and procedures incorporate gender considerations Number of legal instruments drafted, proposed or adopted with USG assistance to promote gender equality or non-discrimination against women and girls at the regional, national, or sub-national level (Gender (GNDR)-1) 0 4 Regional Capacity and Cooperation IR 3. Strengthened regional capacity to support CDTS, PPP, and EAFM 3.1 Guidelines for implementing integrated CDTS, including ACDS, developed and adopted at regional level Number of regional or national governmental entities endorsing Integrated CDTS, including ACDS, and Guidelines (Custom) 0 5 3.2 Key stakeholders have capacities to use CDTS/FIS for fisheries traceability and management Number of people trained in sustainable natural resources management and/or biodiversity conservation as a result of USG assistance (EG.10.2-4) EAFM CDT 1,245 639 606 1,395 653 742 3.3 CDTS public-private partnerships enable CDTS demonstration and expansion Value in cash or in-kind (in USD) of public and private sector investments in sustainable fisheries and coastal and marine ecosystem conservation (Custom) 1.41M 4M 3.4 Regional, national, and local stakeholders engaged and committed to CDTS, PPP, and EAFM Number of new USG-supported public-private partnerships (PPPs) formed (Custom) 10 14 This report first presents the methodology used for the evaluation followed by a presentation of findings organized by EQ. Within the findings section, text boxes provide summary conclusions. The ET then presents detailed conclusions and targeted and actional recommendations for evaluation users. METHODOLOGY EVALUATION SCOPE OF WORK The purpose of the performance evaluation (PE) is to assess the effectiveness of USAID Oceans’ approaches to increasing the ability of regional fisheries organizations to conserve marine biodiversity and combat IUU fishing in the Asia-Pacific region. The evaluation is intended to assist USAID Oceans adaptive management and sustainability strategies as well as inform decision-making for future programming. USAID/Regional Development Mission for Asia (USAID/RDMA) intends to use the evaluation to address gaps in their knowledge of project results, factors, and sustainability potential. The Scope of Work (SOW) for the evaluation is presented in Annex 1. The evaluation is primarily aimed at answering questions that have not already been addressed by the MTR and the other mid-term review ` conducted by USAID Oceans or USAID/RDMA. This is not a comprehensive evaluation of the Activity design, results, and management, but rather a targeted evaluation of the questions presented in the SOW. The evaluation questions (EQs) and sub￾questions (see below) focus on the extent to which USAID Oceans’ objectives have been achieved and/or advanced, factors affecting results in reducing IUU fishing and strengthening fisheries 5 management, and the potential sustainability of results.5 A preliminary consultation was held in late 2018 with USAID/RDMA to clarify the meaning of the EQs. Evaluation Question (EQ) 1: To what extent have the USAID Oceans’ approaches achieved and/or advanced the USAID Oceans’ objectives on a) regional coordination, b) PPP and c) capacity building? a) Regional Coordination a. To what extent has USAID Oceans’ work with SEAFDEC and CTI-CFF enhanced their ability to: counter IUU fishing and strengthen fisheries management across ASEAN member countries; and effectively support ASEAN member countries’ adoption and use of eCDT systems to counter IUU Fishing and strengthen fisheries management? b) PPP a. To what extent does USAID Oceans effectively engage national governments and private sector partners within learning sites and expansion sites (Kelantan, Songkhla) in formalizing partnerships for a fully sustainable eCDT system? b. Which incentives for adopting eCDT systems are the most appealing to private sector partners to encourage adoption of eCDT systems? c) Capacity Building a. How promising have USAID Oceans’ capacity building efforts been in helping key targeted beneficiaries to achieve the following results? i. Strengthening SEAFDEC capacity as the regional champion for moving ASEAN member countries to adoption and use of an eCDT system; and ii. Building local and national government agencies to develop and implement an eCDT system? b. To what extent have human welfare, particularly gender equity and labor considerations, been addressed within the promotion and adoption of the eCDT systems being supported by USAID Oceans? EQ 2: What factors (both internal and external) have enhanced, diminished, or have no effect on achievement of USAID Oceans expected results in reducing IUU fishing and strengthen fisheries management? a) eCDT adoption and use among private sector “early mover” partners within seafood supply chains at both USAID Oceans learning sites? b) EAFM among national, provincial, and local fisheries management authorities at both USAID learning sites? c) Regional coordination among ASEAN and CTI-CFF member countries supported by SEAFDEC and the CTI-CFF Regional Secretariat? d) PPP among private sector “early mover” partners within seafood supply chains at both USAID learning sites? e) Capacity building and institutionalization of eCDT approaches and guidance within SEAFDEC and CTI￾CFF Regional Secretariat? f) Fisheries supply chain-related human welfare and gender equity awareness and actions being taken by stakeholders within the seafood supply chains at both USAID Oceans learning sites? EQ 3: What evidence(s) exists that ASEAN and CTI countries, especially in the learning sites and expansion sites would potentially sustain the adoption and implementation of the eCDT systems to counter IUU fishing and strengthen fisheries management as a result of USAID Oceans' support? a) What aspects of sustainability (legal, institutional, financial, cultural, etc.) can be observed in the Activity implementation and results? 5 The evaluation assesses the likelihood of sustainability of individual technologies and approaches; however, it cannot distinguish the effects of individual activities or approaches on the sustainability of eCDT more generally, as the Activity was conceived of, and implemented as, one Activity. Accordingly, it is not possible to parse out effects of individual activities. 6 b) What specific actions by which stakeholders (i.e., USAID/RDMA, USAID Oceans, SEAFDEC, CTI-CFF and beyond) will be required to ensure future sustainability of eCDT, EAFM after the USAID Oceans ceased? The EQs and sub-questions, drawn from the approved SOW and revised in coordination with USAID/RDMA and Activity stakeholders, provided the main guide to the scope and approach of the evaluation, the latter of which is described in the section below. EVALUATION DESIGN AND METHODS The Evaluation Design Matrix in Annex 10 was the primary guide for the evaluation approach. It outlines the main structure of the evaluation, organized by EQ and including details on the indicators, data sources/collection, and data analysis methods that were used to answer the EQs and sub-questions. Key Informant Interviews (KII), both individual and group, were held with USAID and USAID/RDMA staff, TetraTech ARD USAID Oceans staff, SEAFDEC and CTI-CFF Secretariat staff, government officials at several levels in multiple countries, seafood processors, middlemen and suppliers including local fishers at Manado/Bitung (Indonesia) and General Santos City (Philippines), and non-governmental organizations (NGOs) involved in the Activity. Data collection tools can be found in Annex 5. Fieldwork was conducted in Bangkok, Thailand; Jakarta and Bitung, Indonesia; Manila and General Santos City, Philippines; and Kuala Lumpur, Malaysia. The evaluation team (ET) selected these locations for in-person data collection based on the SOW, the EQs, locations of key Activity stakeholders, and previous activities underway or completed to date. The sites allowed the ET to collect data from stakeholders at the regional level, at both learning sites, and for two expansion sites in Thailand and Malaysia. Interviews were requested with stakeholders from other expansion sites, but responses were very limited. Sampling of key informants represented a general cross￾section of USAID Oceans participants and included US government staff (including USAID), USAID Oceans staff, USAID Oceans partners (defined as grantees or subcontractors), processors (including first mover companies), suppliers (including fishermen), national and local government (including representatives from local government and relevant ministries), regional organizations (including SEAFDEC and CTI-CFF), and others (including other organizations active in this sector and related USAID activities). Figure 1 summarizes the profile of respondents by country and type of respondent. Annex 9 includes details on ET members. 7 Figure 1: Number of Respondents by Sex, Type, and Location There were 115 respondents overall, mostly KII but some small group discussions with government departments and fishing boat crews. Figure 1 shows that 42.6 percent of respondents were female and that national and local government officials accounted for the highest percentage of respondents (37.2 percent). The most respondents were based in Indonesia (42.5 percent), followed by the Philippines (27.4 percent) and Thailand (25.7 percent). Thailand is higher than may be expected due to the inclusion of many regional respondents, including from SEAFDEC, USAID/RDMA, and USAID Oceans. Individuals contacted for interviews are presented in Annex 6. Documents reviewed are listed in Annex 7. The evaluation itinerary is presented in Annex 3. In nearly all cases, KIIs were conducted in person, with at least two ET members. In each interview, one team member served as the lead interviewer, with the other(s) providing additional or probing questions as relevant. In all interviews, the ET had at least one person present who was fluent in the language of the respondent to permit interviews in local languages or translation, as needed. The ET included four national evaluation coordinators in Thailand, Indonesia, Philippines, and Malaysia. Most interviews were conducted primarily in English. Each ET member took notes during interviews, and notes from each interview were compiled and summarized by either the Senior Team Leader or Senior M&E Specialist. Summary notes were consistently reviewed during fieldwork to identify emergent themes and topics (and occasionally, respondents) for follow up. Following fieldwork, the ET presented preliminary themes from fieldwork to USAID/RDMA and USAID Oceans in Bangkok. Finally, in preparation of this report, the Senior Team Leader and Senior M&E Specialist analyzed each interview, assessing key themes for each EQ. LIMITATIONS AND MITIGATION STRATEGIES The first potential limitation relates to the primarily qualitative methods utilized, which rely principally on the perceptions of key stakeholders and are influenced by individual experiences and varying interpretations of the questions asked by interviewers. Furthermore, the evaluation was not designed to distinguish definitively between contributions from USAID Oceans and other important projects (for example, funding from Sweden and the Japan Trust Fund to SEAFDEC) and contextual Measurement Unit: Number of Respondents by Sex, Type, and Location 8 factors (for example, the carding system of the European Union (EU)).6 The evaluation relies on key informant perceptions on the role of USAID Oceans in the outcomes assessed. The team sought to mitigate these limitations by: ● Collecting data from multiple respondents and respondent categories to triangulate and validate responses. By interviewing multiple respondents, including from different groups, the ET limited the bias that might exist with any individual respondent. This is particularly relevant when looking across respondent categories. For example, rather than only asking USAID Oceans staff about their involvement in the development of national eCDT systems, the ET also asked ministry officials at multiple levels as well as other knowledgeable stakeholders. ● Requesting concrete evidence or documents to verify qualitative responses. This helped the ET verify statements made by respondents, although it did not always clarify the exact level of contribution of USAID Oceans versus other sources. ● Use of explicit measurement indicators alongside the EQs and assessing the balance of overall evidence against the defined criteria and indicators in the evaluation matrix. A second limitation relates to the availability of respondents. While the ET was able to meet representatives from all key targeted stakeholders in both learning site countries, as well as regional organizations, some of the individuals on the list of stakeholders, especially in the expansion countries, did not respond to a request for interview, despite multiple attempts, or sent last minute substitutes. This does not limit the evaluation in its consideration of the learning site countries, where the vast majority of effort and resources were allocated, and the ET was able to collect data from key stakeholders in Thailand, Cambodia, and Malaysia to represent expansion sites. However, discussions with the Government of Malaysia were limited to staff in Kuala Lumpur. Representatives from Sabah, Malaysia and other expansion countries except Cambodia did not respond to requests for interviews. The ET had also planned to conduct an internet-based survey to solicit input from stakeholders in expansion site countries. However, given the lack responses to requests for interviews among those identified as most involved in the Activity in the additional expansion countries, the ET chose to cancel the internet survey because the likelihood of sufficient responses to permit any analysis of generalizable results was deemed to be extremely low. Nonetheless, as noted above, the ET was able to collect data from targeted stakeholders in both learning site countries as well as in two expansion site countries, which goes beyond the requirements of the evaluation SOW.7 The ET also encountered a few less significant limitations, noted in brief below. ● First, given that the eCDT systems and technologies were still in the development and testing phase, it was difficult for many respondents to consider questions related to sustainability, incentives, or costs for use. That said, many were able to speculate based on the expected benefits of the system or technology. ● Second, while most respondents were aware of and able to respond directly to questions related to Activity implementation and results, few were able to speak confidently about EQ 2 (assessing factors related to IUU) beyond their immediate areas of work. For example, fisheries officers in local government units (LGUs) spoke often about issues of monitoring and enforcement, and fishermen spoke about livelihood pressures, but few were aware of, 6 Although the PE was not designed with a focus on assessing attribution of outcomes, and the EQs are not focused on attribution, the EQs, particularly EQ 1, do ask about the effects of USAID Oceans on important outcomes, which implies some degree of assessment of the contribution of USAID Oceans to those outcomes. 7 The evaluation SOW references collecting data from the learning sites and one expansion site. 9 or able to speak confidently about, other potential factors. The team sought to mitigate this limitation by speaking with a wide range of stakeholders. ● Third, there were few available progress data on the adoption of the technologies by the targeted groups that are being assisted by the Activity; for example, there were no available follow-up survey data of the early users of Trafiz8 or the distributed eLogbook9 in Indonesia. Details of the latest outputs (e.g., cost-benefit study, business cases, e-Logbook implementation plans were not available during data collection). Also, without detailed post￾training data on the use of newly acquired skills, it was difficult to assess the impact of many short-term training events in terms of application and the subsequent use of the training results by trainees on-the-job. ● Finally, the evaluation was conducted according to the scope and evaluation questions discussed above which are focused on specific areas of interest to USAID and USAID Oceans, even though this did not correspond to a comprehensive evaluation of design, implementation, and results. The evaluation therefore does not explicitly assess the project management and implementation approach or the project design and theory of change. While this does not limit the validity of the findings and conclusions on the evaluation questions, it does mean that potentially important or interesting questions related to USAID Ocean’s logic modelling and the strategies for effective technical assistance were not within the evaluation scope and therefore not explicitly assessed. FINDINGS EVALUATION QUESTION 1 To what extent have the USAID Oceans’ approaches achieved and/or advanced the USAID Oceans’ objectives on a) regional coordination, b) PPP, and c) capacity building? REGIONAL COORDINATION The EQ focuses on the extent of enhanced ability of SEAFDEC and CTI-CFF to establish regional coordination and cooperation for countering IUU fishing and strengthening fisheries management across ASEAN member countries. The regional coordination achievements are also linked to capacity building results described below. SEAFDEC’s regional functions in coordinating CDT discussions have been driven by directives from the SEAFDEC Council seeking response to EU Regulation 1005/2008 and by the financial support of USAID, Japan, and Sweden for the development of the ACDS, and later eACDS, an eCDT system, and its initial piloting in Brunei Darussalam.10 Their involvement in USAID Oceans’s eCDT activities at the learning sites, and now at the expansion sites, has added to these functions. USAID Oceans’ stated objective regarding regional coordination is that SEAFDEC become “recognized as the regional champion for CDT and exert robust leadership in promoting further replication and adoption of the CDTS and eACDS systems across the region.”11 The Activity design strategy began with the assumption that a region-wide approach to eCDT could be developed, but 8 The eCDT system tool USAID Oceans supported. The mobile application was developed by Altermyth for USAID Oceans for small-scale fish suppliers and buyers at the point of landing in Bitung, Indonesia, and business management applications. It was launched in August 2018. 9 In Bitung, Indonesia, USAID Oceans launched in late 2018 the use of eLogbooks for data entry at sea near shore or at landing sites aimed at electronic recording of catch. 10 SEAFDEC, Onsite Training for Pilot Testing of the eACDS in Brunei Darussalam and the meeting of ACDS Committee, 19-22 June 2017. 11 USAID Oceans, AMELP, Revised Dec. 2018, p.12 10 the experience from the early stages of the Activity indicated a clear limit to establishing regional strategies.12 As such, USAID Oceans staff noted that the Activity shifted to focusing on supporting the development of customized country-specific systems - building on the systems being developed at the time by the governments of Indonesia and the Philippines. Despite this shift from a more regional, top-down to a country-specific, bottom-up approach to eCDT system development that might imply a different role for a regional body like SEAFDEC, the ET found no evidence of a concurrent, explicit shift or clarification in the stated outcomes for SEAFDEC and CTI-CFF capacity development or coordination roles. Moreover, the ET found no evidence of clear, objective, measurable outcomes or milestones related to SEAFDEC’s coordinating role or capacity levels against which to measure progress, either before or after the shift focusing on country-specific solutions. Nevertheless, the ET provides below an assessment of the current level of capacity for coordination and the extent of USAID Ocean’s contribution to that capacity. The approach adopted for enhancing regional coordination is primarily based on the direct technical training and support provided to SEAFDEC and CTI-CFF Secretariat and creation of a network of Technical Working Groups (TWGs) to coordinate and support member countries at the regional and national levels. USAID Oceans holds Annual TWG Meetings and Planning Workshops to identify regional and national priorities and to obtain input for work plans, and national and thematic workshops related to eCDT, EAFM, seafood certification standards, HW/GE issues, and other topics (see Annex 2). The USAID Oceans Communications Strategy supports this consultative effort with goals related to increasing audience awareness at international, regional, and national/site levels, motivating audience members to engage with USAID Oceans, and sustaining the learnings and achievements beyond the life of the Activity.13 Related to regional coordination, the ET identified the following four themes from interviews with key stakeholders. 1) All SEAFDEC staff interviewed believe they now have a broader command of the subject areas – eCDT, EAFM, and HW/GE than before the Activity started, but they also noted that their ability to maintain much of the eCDT activity depends upon funding availability. SEAFDEC staff reported that support from USAID/US Department of Interior (USDOI), Japan Trust Fund, and the Government of Sweden has jointly contributed to their expanded role in the region on IUU fishing and fisheries management. The evolving regional vision for eACDS development was recently summarized in a presentation by SEAFDEC as follows: …a pan-ASEAN eCDT to serve as a minimum common denominator across ASEAN that can ultimately facilitate the exchange of key information for these various countries’ requirements and respective eCDTS. With pilots in Vietnam and Brunei, a rich dialogue has been set up through this initiative between a number of key ASEAN countries to establish common Key Data Elements (KDEs) and Critical Tracking Events (CTEs) that can be consolidated and standardized for the purpose of more seamless and responsive intra and inter￾regional trade.14 12 The MTR Report (2018, p4.) stated “as each country has different needs and capacities to move forward with eCDT systems and EAFM, there is not likely a one size fits all for the region.” 13 USAID Oceans, Communications Strategy and Implementation Plan (20018-May 2020), Sept. 2018. 14USAID, The Oceans and Fisheries Partnership, Malaysia Capacity Gap Analysis and Partnership Appraisal, November 2018, p. 21. 15 E.g., SEAFDEC, Report of the Forty-first Meeting of the Program Committee, 5-7 November 2018, Sub-activity 1.1: 3rd Annual TWG Meeting and Planning Workshop, p. 266. 11 2) Various technical reports, fact sheets, and communication products on eCDT and HW/GE produced by USAID Oceans were noted by SEAFDEC staff as useful resources for ongoing dialogue within the region. However, stakeholders from all groups, including SEAFDEC, government officials, and private sector all requested more information on the projects and preliminary results from the learning sites. A dominant theme from the interviews with TWG representatives who attended regional workshops was requests for definitive information and guidance on workplans and budgets available from the Activity to report back to their managers. 3) Orientation to eCDT, EAFM, and HW/GE issues was appreciated by TWG representatives, but variations in country fisheries, CDT approaches, and capacity also affected the applicability of knowledge from regional meetings for follow-up action by member country representatives. Surveys of post-TWG meetings indicated positive scores where proceedings met expectations, provided valuable skills and materials, and encouraged open participation, indicating that the regional coordination workshops have been effective at this level, although as noted further in this report, communication issues still exist, and this does not address higher level outcomes related to actual changes in behavior or in regional coordination.15 4) SEAFDEC has developed and piloted eACDS themselves, but it has been very dependent on contractors and partnership with the Fisheries Marketing Organization of Thailand (FMOT), rather than internal capacity. SEAFDEC considers that their involvement in the development and testing of the USAID Ocean’s eCDT systems and technologies has been minimal. 16 SEAFDEC staff noted that they, at least currently, do not have the capacity to support the expansion of the USAID Ocean’s systems and technologies. They reported that while they have gained a greater understanding of the process for guiding a country through initial implementation of an eCDT system and can serve as a regional resource at that level, they do not currently have the capacity to lead in the technical development of a tailored system. Additionally, SEAFDEC staff noted that there was still significant confusion, both within SEAFDEC and among member countries, as to the distinction between eACDS (SEAFDEC) and the USAID Oceans supported eCDT technologies and systems developed in the Philippines and Indonesia, although as noted elsewhere in this report, these are not competing systems. CTI-CFF Secretariat describe their primary function as ‘workshop facilitation’ for members countries and reported minimal familiarity with eCDT systems (including those developed by USAID Oceans). CTI-CFF has also undergone significant changes, particularly related to staff turnover and reduced staffing levels, that have limited their capacity and resources to engage extensively in additional activities. 16 They have, however, either participated in, been invited to participate in, or have been the conveners of every USAID Oceans workshops and demonstration visit. 16 16 12 PUBLIC-PRIVATE PARTNERSHIP AND ENGAGEMENT The EQ focuses on the extent to which USAID Oceans effectively engages national governments and private sector partners for eCDT systems, and on the incentives that may be the most appealing to private sector partners to encourage adoption of eCDT systems. The expected PPP results are based on (a) regional, national, and local stakeholders engaged and committed to CDTS, PPP, and EAFM (as measured by the number of new US Government (USG)- supported PPPs formed), and (b) CDTS PPPs enabling CDTS demonstration and expansion (as measured by the value in cash or in-kind (in USD) of public and private sector investments) – see Table 1. The targets for Public Sector Engagement are: 14 new PPPs formed and $4M in investments generated. The endgame outcome statement is “key partners and industry stakeholders engaged in the development and use of the CDTS/FIS”, with a target of over 50 seafood actors verifying the legality, sustainability, and responsibility of 20 metric tons’ of seafood products via CDTS by 2020 through partnerships. The USAID Oceans Activity is committed to building the business case and financing mechanisms for eCDT. This is aligned with USAID’s Private Sector Engagement Policy aimed at leveraging private sector involvement and solutions to design and deliver development assistance.17 USAID Oceans’ partnerships encompass (i) Grants, (ii) Partnership Agreements, (ii) Partner sub-contractors, (iv) Memoranda Of Understandings (MOU) with partners, and (v) other informal partners. EXTENT TO WHICH USAID OCEANS EFFECTIVELY ENGAGES NATIONAL GOVERNMENTS AND PRIVATE SECTOR PARTNERS FOR CDT SYSTEMS Annex 4 documents an extensive range of global, regional, and local partnerships that has been developed through USAID Oceans including engagement with companies and industry associations, partnerships with host governments, alliances with NGOs and private foundations, and cooperation with multi-stakeholder initiatives/forums. To date, USAID Oceans reported that 10 partnerships have been developed, and the Activity is also engaging in more informal ways with 27 other companies and organizations. The vision for ‘Partnership Engagement’ contains a variety of aims, including a) leveraging donor funds, b) generating private investment, c) economic agreements between local government and fisheries associations, d) utilizing PPP as a mechanism to finance implementation of CDT/EAFM18, and 17 Private Sector is defined as “for-profit, commercial entities and their affiliated foundations; financial institutions, investors and intermediaries; business associations and cooperatives; micro, small, medium and large enterprises that operate in the formal and informal sectors; American, local, regional, and multi-national scale businesses; and for-profit approaches that generate sustainable income (e.g., a venture fund run by a non￾governmental organization (NGO) or a social enterprise).” USAID, Private-Sector Engagement Policy, Executive Summary 18USAID Oceans, Activity Report, Learning Exchange for Coral Triangle Member Countries, Manado-Bitung, Indonesia, 25-28 June 2018, p.1. Conclusion The timeliness and scope of USAID Oceans in sponsoring regional level dialogue and learning on eCDT, EAFM, and HW/GE in fisheries management has provided funding for coordination and networking activities that helped to increase SEAFDEC’s role and capacity to lead the dialogue with their member countries, but with the caveat that their institutional capacity to provide technical assistance to ASEAN countries on eCDT has distinct limitations. 13 e) undertaking a series of seven sub-component tasks.19 This makes for a complex set of challenges for the partnership component. The Partnership Review in 2018 provided a thorough summary of the CDTS design and development partnerships, market partnerships, and specific country partnerships. Many of the partners have provided valuable expertise, although not all of the partnerships have been successful or sustained, as noted in the Lessons Learned section of the report.20 From the perspective of Activity partners, USAID Oceans has worked through two local organizations in the Philippines (SFFAII) and Indonesia (MDPI) who have both led the identification and coordination with private sector first movers in the learning sites. The Activity has also worked directly with private sector technology developers, including Altermyth in Indonesia and FAME in the Philippines, on the development of eCDT technologies. The ET identified four key findings in relation to partnerships. 1) TWG members in expansion countries as well as SEAFDEC staff noted that building a regional network of TWGs and working partners builds expectations of substantive short￾term activity and available funding. The original regional technical advisory group did not prove effective, and the Activity therefore shifted to working directly with governments, industry associations, and NGOs.21 Some members of the TWGs in expansion countries are anxious about finalizing workplans and commitments, and how the learning site lessons can be applied to their country’s CDT issues and capacity constraints. Given these issues, some stakeholders noted to the ET that maintaining such a network without dedicated and specific workplans and funding may be difficult after Activity closure. 2) The ET noted significant benefits and drawbacks to relying on local organizations for learning site implementation. USAID Oceans staff noted that the approach of working through local groups (e.g., MDPI and SFFAII), who were reported by respondents to be well respected and had strong connections in the fisheries sector, made it easier for the Activity to identify and work with first mover companies. 22 Also, USAID Oceans staff noted that by working through established local organizations, there is a higher chance for sustainability since these organizations are likely to remain active in the sector following the completion of USAID Oceans. However, even though the working level technical partners have the local contacts, knowledge, and experience, they do not, particularly in Bitung and to a much lesser extent in General Santos, have significant local presence to oversee site activities to the extent necessary to monitor progress. USAID Oceans additionally provided one USAID Oceans site coordinator in each learning site (both of whom had developed extensive contacts and relationships in the learning sites and were highly respected by respondents) and one technical team member based in each of Jakarta and Manila, although these technical staff members travelled extensively across the region. This relatively light local presence was reported by government and private sector stakeholders at the local level as a limitation in terms of the Activity’s ability to effectively coordinate and communicate with suppliers and processers and respond to unforeseen issues as they arise. 19 USAID Oceans, Mid-Term Partnership Review, May 2018, p. 5.; note – only 6 sub-components listed in the Evaluation Terms of References (TORs). 20 USAID Oceans, Mid-Term Partnership Review, May 2018, p. 8-10. 21 See USAID Oceans, Mid-Term Partnership Review, May 2018, p. 10. 22 USAID Oceans staff reported that they are currently working with 16 processing or supplier companies in the Philippines and Indonesia. 14 Indeed, the ET heard from regional, national, and local stakeholders about a lack of information on some aspects of the Activity in the learning sites, particularly related to status and functionality of private sector eCDT technologies. One significant example is the relative lack of information on progress on eCDT technology development among ministry officials interviewed by the ET in both learning site countries and even among staff of one of the local implementers. 3) The Activity had started partnerships with umbrella organizations and associations such as Indonesia Coastal Tuna Alliance and the Association of Tuna Handliners (Philippines), but some members of such groups would also prefer a more substantive advocacy or educational role with their members, with appropriate funding as a delivery partner. These groups provide a strong platform for reaching small-scale fishers. 4) USAID Oceans has established many partnerships or relationships with both the public and private sectors, as evidenced by the collaboration with the Bureau of Fisheries and Aquatic Resources, Philippines (BFAR) and the Indonesian Ministry of Marine Affairs and Fisheries (MMAF) as well as the many private sector partners. The communication outputs have been comprehensive. However, there were significant gaps in awareness among government stakeholders of the work USAID Oceans was doing with the private sector. For example, one key ministry official in the Philippines noted that BFAR at the national level had received a report on the FAME transponders (the eCDT technology supported by USAID Oceans in the Philippines) but they were unaware of any details. A similar level of unfamiliarity with the status of private sector technology development status was expressed by officials in Indonesia and at SEAFDEC and CTI-CFF. Awareness was marginally higher at the local or regional level, but officials (e.g., fisheries staff at Bitung ports and BFAR regional officials) were still not familiar with details of implementation progress. Moreover, there was some uncertainty among government and SEAFDEC officials about the proprietary nature of some of the eCDT technologies and the extent to which they would be available or modifiable by governments. 23 Information on USAID Oceans’ progress is regularly shared with key partners through emails, newsletters, progress reports and other mechanisms, but this information was not being internalized by all key staff. INCENTIVES THAT MAY BE THE MOST APPEALING TO PRIVATE SECTOR PARTNERS FOR ECDT ADOPTION A central theme of PPP has been to encourage industry (suppliers, processors, buyers) to invest in eCDT systems that will improve the efficiency of their operations and regulatory compliance. The main findings related to incentives for private sector adoption are the following: 1) Many of the prospective industry investors at the learning sites, as recognized by USAID Oceans staff and the stakeholders themselves, do not yet appear to be ready to invest due to a lack of convincing evidence about return on investment in terms of regulatory compliance, access to key markets, premium prices for traceable fish, or production efficiencies.24 The uptake of supported technologies is low at present (as described more below), but this may change as the pressures for eCDT increase and the technologies 23 This contrasts with the USAID Oceans view: “The CDT system will be owned by the countries and will use an open source code so it can be easily accessible and can provide rapid technology transfer.” USAID Oceans Inception Workshop Report, 2015, p. 9. 24 The Partnerships Review report (p. 12) proposes development of a ‘commercial viability architecture for ICTSA’. 15 become more standardized, proven, and affordable. The level of achievement on technology adoption and spread within the Activity time frame is currently uncertain. The need for incentives depends upon the sector. Small-scale fishers are looking for financial incentives that reach to fishermen and are not caught by middlemen. Small-scale fishermen uniformly reported that currently incentives are very low for adoption because of both lack of regulatory requirements or enforcement for small-scale fishing and lack of any price premiums or demand or documentation that reaches this level. USAID Oceans staff note that to assist in generating incentives for small scale fishermen, they are working in the Philippines with LGUs to draft resolutions or ordinances that would support eCDT and EAFM plan implementation at the local level. USAID Oceans reports that in Indonesia, they are working on providing other types of incentives, such as assistance with licensing and provision of safety gear. 2) Larger commercial fishers are focused on complying with regulations in an efficient manner given that they already feel pressure for catch documentation. Accordingly, their incentives were primarily reported around efficient policy compliance and tools that improve efficiency (such as communication of catch data at sea and modules to monitor fuel and hold temperature). Processors are mostly interested in production efficiency incentives. The disincentives that may be imposed by potential high regulatory requirements for small fishers is a concern. 3) The willingness to pay was reported to be generally low, with some possible exceptions among commercial and some processors, in part due to lack of knowledge about benefits and costs, and the general reluctance to risk changes in traditional practices. For example, one processor and commercial boat owner in Indonesia had already purchased additional Pointrek devices (one of the technologies further described below), while another similar processor in Indonesia who was piloting the same devices was reluctant to commit to rolling out the device on other boats due to the limited evidence generated so far on its operation and benefits. The relatively small size of the early mover testers also limits information about possible incentives that may be required for larger scale adoption of the technologies. ECDT DEVELOPMENT The EQ focuses on the extent of results in strengthening SEAFDEC and national capacity for CDT development and institutionalization of eCDT systems in targeted countries. To do this, the ET describes capacity development at SEAFDEC in eCDT and then discusses eCDT development in the learning and expansion sites. Conclusion USAID Oceans has developed a wide array of partnerships across the public and private sector that have supported eCDT development in national systems and private sector technologies. However, gaps in communication and coordination remain between public and private sector partners, particularly around the private sector technologies. Incentives for uptake of eCDT systems are dependent on the sector. For processors, the most prominent incentives are efficiency and policy or market requirement compliance. For commercial fishermen, the incentives relate to policy or market requirement compliance and additional benefits of technology (such as catch communication at sea and fuel or hold temperature monitoring). For small scale fishermen, policy and market compliance is less relevant, but the potential for the additional technology benefits is appealing. For all stakeholders, cost remains a concern. 16 The expected result for this component is “CDTS functioning, in place and integrated with national FIS in at least two fisheries in priority biodiversity areas through key points of the value chain”. Table 2 lists the CDT technologies that have or are now being tested with some level of Activity support. Table 2: Catch Documentation and Traceability (CDT) Technologies Locations Partners Technologies Thailand, Ranong and Pattani ports Thai Union and MARS Inc. Petcare Fleet One (supplier) Inmarsat Vessel Monitoring System (VMS) technology with two-way communications, an e-logbook and a mobile phone application, Hi-Chat (aka ‘Fish Chat’), used for crew communications. 36-week testing in May-Dec. 2017. Evaluation of pilot completed.25 Philippines Philippines Bureau of Fisheries and Aquatic Resources BFAR eCDTS – Philippines Bureau of Fisheries and Aquatic Resources An internal, downstream traceability system to integrate eCDTS data capture with the existing supply chain databases. Philippines, General Santos Futuristic Aviation and Maritime Enterprises (FAME) and participating fishers USAID Oceans supported FAME vessel transponders to communicate with a cloud server for sending catch data, communications and location tracking and emergency distress button in near real time. Indonesia Private sector ‘first movers’ Pointrek/Inmarasat Vessel Monitoring System (VMS) Used by government and fleet owners to monitor locations, activities and conditions, including two-way communications and real-time catch reporting. Indonesia Indonesia Ministry of Marine Affairs and Fisheries (MMAF) E-Logbook/STELINA – National Fish Traceability and Logistic System Electronic catch recording device and internal downstream system that serves to bridge eCDT data capture technologies and integrate more than 12 existing fisheries management databases of MMAF. Indonesia, Bitung Altermyth, MDPI and participating fishers USAID Oceans supported Trafiz, mobile application developed by Altermyth for USAID Oceans for small-scale fish suppliers and buyers at the point of landing, and business management applications. Indonesia, Bitung MDPI Blue Ocean Grace International (BOGI) – ‘First Mover’ USAID Oceans supported Trace Tales, developed by USAID Oceans’ grantee Yayasan Masyarakat dan Perikanan Indonesia (MDPI). Enables data capture through the processing stages. Allows processors to electronically track their inventory as it moves through the processing factory—from receiving, to filleting, to packaging, freezing and shipping. Vietnam SEAFDEC/Fisheries Marketing Organization of Thailand (FMOT), Japan Trust Fund eACDS - SEAFDEC/FMOT supported initial start up to develop an eACDS supply chain system with web and mobile database modules. 25 Marine Change, Thai Union CDT and Crew Communications Pilot Assessment Report, March 2018. Note – only the evaluation of this pilot project was funded by the USAID Oceans. 17 SUPPORT FOR SEAFDEC ECDT CAPACITY SEAFDEC stated they began developing a CDT system in 2014 in response to EU Regulation 10005/2008. They have been testing and refining an eACDS system in Brunei Darussalam since 2017.26 The USAID Oceans MTR report noted the differences between eCDT and eACDS related to terminology and variations in system design scope, traceability methodology, and technologies.27 Interviews with USAID Oceans staff suggested that eACDS was appropriate for less complex countries, and while it requires tailoring for each country, the system represented a more top-down, regional approach. Although the USAID Oceans original approach envisioned a regional system similar to the eACDS, the Activity quickly found that some countries already had elements of a system and were working to further develop comprehensive electronic systems. Accordingly, USAID Oceans shifted to working with the relevant ministries in Indonesia and the Philippines to develop the country-specific systems and associated feed-in technologies, and much less on refinement of a regional system, such as eACDS. USAID Oceans’ work on development of the eCDT system designs included hardware and software, analyses of the eCDT capacity gaps in the participating countries, preliminary introduction of eCDT technologies and processes in seafood supply chains at the learning sites and providing support, as requested, for development and expansion of eACDS. Guidance on USAID Oceans’ approach to system design, development, and technical specifications has been provided in several publications, including the Data Requirements for Catch Documentation and Traceability in Southeast Asia: Critical Tracking Event and Key Data Element Framework and Glossary, or the (KDE Manual) and Fisheries Catch Documentation and Traceability in Southeast Asia: Technical Specifications, providing guidance—as well as the KDEs required for full end-to-end traceability. A key output has been the CDT Capacity Gap Analyses that provided extensive information of the situation and needs in each country. A major forthcoming milestone is the development of CDT guidelines with SEAFDEC, expected in the next few months. Evaluation respondents noted some key issues related to CDT capacity and institutionalization at the regional level, explained below. 1) SEAFDEC, and even more so CTI-CFF, had only limited involvement in USAID Oceans eCDT system development which focused more on working with national governments and the private sector in Indonesia and the Philippines. The expectations for SEAFDEC and CTI￾CFF Secretariat capacities to be developed were not precisely defined within the US Department of Interior (DOI) contracts and the Work Plan for Strengthening Organizational and Administrative Capacity. The main strategy has been based on developing and piloting of eCDT technologies and training users and others on the use of these technologies, and providing hardware and software support to government catch documentation systems 26 Currently the eACDS prototype system covers the processes from Port-Out control, offline reporting of catch at Sea, Port-in control, catch verification at landing sites and at processing plants for tuna fisheries, issuing movement document (MD) for fish buyer to serve local markets and/or to processing plants for exporting, requests for catch certificate by processors, and issuance of catch certificate by the Competent Authority for export. Further refinements (e.g., management information systems dashboard) are proposed in the next year. 27 It was noted: “USAID Oceans’ CDTS is intended to include not only traceability but also considerations of EAFM and human welfare, so the system is “broader” than the SEAFDEC ACDS product, which is concerned about traceability only. Also, the USAID Oceans’ system can be adapted to a country’s individual context to develop their own CDTS that can potentially include eACDS”; USAID Oceans and Fisheries Partnership 2nd Annual Technical Working Group Planning Meeting Report, 2017, p. 9 18 rather than institutional strengthening.28 No institutional capacity needs assessments have been completed for integrating CDT support functions into the services of these organizations, and few markers of institutional structure change are apparent, except for appointment of a gender focal person in each of SEAFDEC’s divisions, assisted by Swedish and USAID activities. 2) SEAFDEC noted that their eACDS work has been dependent on local contractors with multi-donor financial support. There are no CDT experts employed at SEAFDEC; the main issue was reported as a lack of capacity, including resources, to further eACDS development. 3) Some of the national partners in learning and expansion sites stated that they began their involvement in USAID Oceans with expectations of funding but later found that much of the work was limited to discrete technical assistance and workshops provided by the USAID contractor and a focus on private sector solutions. Likewise, SEAFDEC reported disappointment that USAID Oceans could not provide direct support for their staff to participate in events (according to USAID requirements) and that a separate agreement had to be reached to facilitate this.29 BITUNG, INDONESIA LEARNING SITE The fisheries at Bitung, Indonesia are dominated by tuna species. Evaluation interviews noted that there are about 5,100 vessels greater than 30 gigaton (GT) and 25,000 less than 10GT in Fisheries Management Area 716. It was stated that 70 percent of the tuna catch comes from small boats (under 10GT). Four CDT technologies (Table 2) are being piloted at Bitung learning site as described below. E-LOGBOOK/STELINA The Indonesian government has developed an automated catch documentation process (STELINA, (Sistem Telusur dan Logistik Ikan Nasional)) that incorporates and integrates catch data from 12 different government regulatory processes into a streamlined catch data management system up to the landing sites.30 It has also launched the use of eLogbooks for data entry at sea near shore or at landing sites aimed at electronic recording of catch on 13,000 vessels. USAID Oceans is helping the Indonesian government to procure eLogbooks for testing of catch reporting, mostly at the point of landing. The eLogbook system development was initiated in November 2018, approximately three months prior to evaluation fieldwork. Accordingly, there was little awareness of this system during field work. USAID Oceans reports that rollout of the equipment and training will occur on MMAF schedules, expected to begin in April 2019. 28 The approach to capacity development is centered on technical assistance rather than institutional change. USAID’s report, E3 Bureau Capacity Development Assessment: From Capacity Development to Sustainable Development, (June 2017) recognized the lack of a common approach, language, or metrics to evaluate the effectiveness of capacity development interventions and proposed four levels: individual, organization, sector, and the broader system. 29 A separate agreement with USDOI had to be established to support SEAFDEC staff to participate in Activity events. 30 Indonesia announced a commitment to implement electronic fishing logbook to domestic fishing vessels. This program started at the end of 2018 for 3,887 fishing vessels over 30 GT, and in 2019 will include 10,984 fishing vessels over 10 GT. https://en.antaranews.com/news/120023/indonesia-announces-commitment-to￾implement-e-fishing-logbook Retrieved March 20, 2019. 19 MMAF staff acknowledged that they have major capacity issues that limit their ability to convert the targeted 13,000 vessels to an online reporting system and to fully operationalize STELINA. The eLogbook and STELINA systems are managed by different directorates within MMAF, which increases the coordination challenges in their development and, especially, integration. Some fishing vessel captains interviewed claimed they received government training and are using the eLogbook without problems, although paper reporting is also occurring. There was no information available to the ET on the progress toward the targeted number of vessels using eLogbooks. The staff of a provincial port authority, responsible for oversight of 160 local fishing vessels (73 percent greater than 10GT) were aware of the eLogbook (not yet arrived at their port) but were completely unaware of STELINA. MMAF staff stated that they are awaiting a regulation to implement STELINA, and that they will need both government and industry support, internal capacity building and local facilitators for e-Logbook dissemination. Both systems are relatively new in development, and USAID Oceans reports that they expect awareness will increase in time. TRAFIZ MOBILE APPLICATION The mobile application, Trafiz, has been developed by USAID Oceans for use at the point of landing with small-scale fishers. The intent is to capture catch data from fishers by having buyers (or possibly even the fishermen themselves) use this app for data collection and business purposes. According to a knowledgeable Activity partner, there were several intermittent users of Trafiz at the time of evaluation fieldwork. The ET was able to interview one user and heard reports of two other users from other sources. One processor reported that of their 42 suppliers providing fish, only one (handline fisher from Sangihe) is a user of Trafiz, although USAID Oceans notes that many of these suppliers are based in another FMA that is not the target of USAID Oceans. A middleman in Manado, who also owns six fishing boats, has been using Trafiz on his mobile phone with some positive results, although he also continues to use the paper system for catch records. He recognized benefits from the application in terms of tracking costs but was uncertain if he will continue to use the Trafiz app because he still uses the paper system and has limited experience with Trafiz. The application designer reported that only one supplier is currently using the application. No other information was provided on current Trafiz users. Uptake is currently low, but USAID Oceans reports that they expect uptake will increase following further refinements. POINTREK/INMARASAT VESSEL COMMUNICATIONS Pointrek is being piloted by at least two large commercial fishing companies (PT SMS-BMB and Nutrindo) to provide continuous communication at sea and to monitor conditions on the vessels (fuel consumption, hold temperature, etc.). Although the devices were installed in July 2018, due to delays in receiving fishing licences, they only started actual use in early 2019. The devices cost approximately $3,000 upfront and $750 per year for the Inmarasat communication subscription, which is being subsidised for one year on the devices that were provided by USAID Oceans. BMB reported having the device installed in three boats with USAID Oceans support, but only one was currently operating, although without the two-way communication function enabled (awaiting a technician from Pointrek to install this function). This boat is also connected with eLogbook, but at sea reporting is currently not functional because the two-way communication system is not yet functional. Moreover, BMB noted that the boat captain still needs additional training on data entry and use. The other two boats have been at sea and have yet to receive training on the use of the system. Nevertheless, BMB has purchased two additional Pointrek systems independent of USAID Oceans. BMB also noted that the system is not currently set up to connect to processor systems. Nutrindo noted that they currently have five Pointrek devices installed, but their experience using them has been limited, as they have been waiting on licenses on boats, and at the time of interview had only had about one month of use. They noted that the tracking and communication system appeared useful. 20 While one of the companies interviewed had purchased additional devices, neither was confident about their long-term use, particularly on their entire fleets, due to the limited experience with the system in operation. There are over 5,100 fishing vessels greater than 30GT in Bitung and/or Manado that may be candidates for this technology if the cost barriers can be overcome, a major impediment. TRACE TALES, A SEAFOOD PROCESSING INFORMATION SYSTEM In Indonesia, USAID Oceans has also been introducing Trace Tales production technology, a system developed by their partner MDPI in Bali. Two seafood company supply chains in Bitung – BOGI and Nutrindo (underway) are adopting the technology with the intent of establishing demonstration of the full eCDT linkages across their supply chains. BOGI, which supplies processed fish to Anova31, has used Trace Tales to automate its entire documentation process with results for increased accuracy and efficiency in operations and data management, reduced product recalls and waste, increased capacity for data analysis and business decision making, and reduced operational costs.32 The labor efficiency of Trace Tales‘s ability to automatically trace back the origins and processing stages of a specific product for quality assurance and food safety purposes is considered a key feature of the technology. During the site visit, one of the first mover companies noted that use of Trace Tales for processing tuna loins has reduced labor requirements on the processing line by two employees and made their production easier to manage. They strongly support the proposed linking of catch input data from Trafiz into the Trace Tails system. However, it is important to note that the Trace Tails system must be tailored for each company and the costs of doing so can be significant, estimated at $10,000 by one company. With support from Anova, MDPI is also planning to install a ‘simplified mini’ Trace Tales version (that protects confidential data) to be linked to the complete chain with BOGI and others, including a small-scale processor, BMI, located in Surabaya, who reprocesses raw and semi-processed seafood from various sources including BOGI. GENERAL SANTOS, PHILIPPINES LEARNING SITE In the Philippines, USAID Oceans has been supporting the development of two components of an eCDT system: the BFAR eCDTs and Fame transponders. BFAR ECDTS The BFAR eCDTs is designed to be a downstream (from point of landing), electronic traceability, and documentation system and is built on a policy foundation dating to 2008. Since 2008, the Philippines has been working on a paper-based catch documentation scheme, including logsheets for certain types of vessels, in response to the EU’s adoption of Council Regulation 1005/2008.33 BFAR’s policy on catch documentation was further developed through BFAR FAO 238 signed in 2012 which required catch documentation for Catch Certificates, and in 2014, BFAR passed the BFAR Administrative Circular (BAC) 251 which lays out more detailed regulations on traceability requirements. 31 In 2018, Anova joined USAID Oceans’ network of partners to establish full-chain traceability for tuna products harvested in Southeast Asia that are imported into the United States. 32 https://www.seafdec-oceanspartnership.org/news/first-mover-partners-report-business-benefits-of￾traceability/. Retrieved March 20, 2019. 33 WCPFC 2017 Annual Report to the Commission: https://www.wcpfc.int/file/156742/download?token=D4Lp8xv_ Retrieved March 20, 2019. 21 USAID Oceans documentation noted in 2016, “the Philippines is in the process of developing and implementing a fully automated catch documentation system which is compatible with regional and EU requirements.” 34 This was corroborated in interviews with USAID Oceans and BFAR staff. Accordingly, both documentation and interviews suggest that the Philippines had been working on a catch documentation system, with the intention of making it electronic, prior to USAID Oceans direct support. That said, all respondents within BFAR at both the national and regional level noted that without USAID Oceans support, which has comprised of workshops, development ‘boot camps’, and pilot testing among first movers leading to system development feedback, it would have taken much longer (one respondent suggested more than twice as long) to achieve the current level of development. Currently, BFAR reports that they have a functional system for downstream traceability; however, it is still in the pilot testing and revision phase, which is necessary for any new system. The system has been developed based on the requirements of BAC 251, which respondents from BFAR, USAID Oceans, and first mover companies all noted was geared towards the requirements of cannery processors and was therefore difficult to apply in the fresh/frozen processing value chain. For example, one first mover processor noted, whereas cannery shipments typically rely on a single source or catch, one fresh/frozen shipment may include fish from multiple sources or catches, and the eCDTs was not set up to track processing of multiple catches in this way. All first mover companies interviewed noted that they had provided this feedback to USAID Oceans and BFAR and were waiting for the system to be modified to fit their processing approach. One first mover expressed frustration at having provided this feedback multiple times over a period of a few months with no apparent progress. All first mover companies interviewed, along with BFAR and USAID Oceans staff, recognized that to move forward with changes to the system, BFAR needs to first amend or revise BAC 251. USAID Oceans staff reported that they have already formally submitted the suggested revisions to BFAR, and BFAR national staff noted that the suggested revisions should be discussed, and they hope approved, during the next council meeting. While USAID Oceans and BFAR staff all expressed that they expected this would happen soon, the first mover companies were less optimistic, expressing less confidence in the speed of BFAR policy decisions or changes and unwilling to estimate time frames for completion. Nevertheless, once the policy is amended, all stakeholders expected that the system could be revised quickly for further piloting. If the system is revised, all first mover companies interviewed noted that they would use the system, as it would increase reporting efficiencies. However, they noted that they would still maintain the paper-based tracking systems they currently employ, and simply submit the data electronically (which would still increase efficiencies). None of the first mover companies interviewed were aware of any electronic processing tracking systems currently being implemented in the Philippines. FAME TRANSPONDERS FAME is a private sector start-up based in the Philippines that prior to USAID Oceans had been working on vessel monitoring systems (VMS). With USAID Oceans support, they have worked on developing a VMS targeted to small-scale fishers that integrates two-way communication. The system is designed with the following features: ● VMS that is accessible via a dashboard or app 34 USAID Oceans Output 1: VCA and CDT Requirements of the Tuna Industry: General Santos City. November 2016. 22 ● Emergency beacon that would highlight the location of a vessel in trouble to other users and the coast guard ● Two-way communication app that would allow communication with family members on shore ● Catch documentation and at-sea reporting through near field communication (NFC) cards The system is designed to communicate with gateways installed on land (and FAME reported that BFAR has agreed to allow installation of gateways on Fish Aggregation Devices (FADs)) at a distance of up to 50km. To extend the range, each transponder can serve as a repeater, which would theoretically significantly extend the working range of the communication system. The system is targeted to smaller scale vessels, though it is being piloted on both municipal (small-scale) and commercial vessels. However, larger commercial tuna fishing vessels in General Santos often fish in the High Seas Pocket 1 (HSP1) which is hundreds of miles from the coast of the Philippines, so this system would likely not be applicable to those boats, which would instead require a satellite-based system. At present, transponders have been installed on between 20-30 boats. 35 However, the implementation has encountered multiple challenges in the functionality of the system, associated with developing a new technology in a location with limited infrastructure. Some of these issues have included 1) issues with power supply and batteries in the gateways causing them to lose power and go offline, 2) occasional dropping of cellular signal to gateway, 3) users unplugging the device from their boat batteries (in fear that it would drain their battery or use fuel) which has led to transponders going offline and, more significantly, rust in the ports where the charging cable should be plugged into the transponder, and 4) lack of NFC cards (reported by one fisherman). In light of these issues, according to one USAID Oceans partner in General Santos, the system was not currently operational. That said, USAID Oceans staff reported that they were currently receiving data from the system. The ET requested to see the information dashboard during interviews with FAME staff in General Santos, and at the time of interview, no boats were currently sending a signal and the last update to the system had been six days prior. This may simply be because the fishers do not go out often (though it would seem likely that of 20-30 boats with transponders, at least one should be out at any given time), but it is clear that if the system was operational, it was, at the time of interview, not being heavily used.36 According to interviews with fishers in and around General Santos, the low usage stems from a lack of perceived benefits from the system. First, none of the interviewed fishermen were aware of how the emergency system would work in practice. They were also unaware that the system had not yet been connected with the coast guard. Two fishermen noted that if they had a tablet then they could potentially use the system to help another boat, but otherwise, they had no way of knowing where a boat in need of assistance was. Indeed, all respondents noted that in the case of an emergency, they would use their radios for assistance. Second, the two-way communication system was not yet operational, so in practice there is no way for users to communicate with family members or friends on shore. Third, none of the interviewed fishermen had smartphones, computers, or tablets that 35 When asking for the precise number, the ET received three different responses from USAID Oceans and FAME staff. In response to a draft version of this report, USAID Oceans noted that originally 24 transponders were installed in August 2018, but they needed to be replaced due to the issues cited in this report. USAID Oceans reports that 23 of the 24 transponders have been replaced. 36 USAID Oceans staff note that the evaluation field work was conducted, “during off season” for some fishers, when wave and weather conditions typically prohibit small-scale fishing operations.” None of the interviewed fishermen noted this during interviews, though it was not asked explicitly. However, some fishermen noted they were still waiting for financing to buy supplies to take their boats out. 23 could either read the NFC data or access the dashboard. Accordingly, none of them could access their data or really knew what was happening with it. Fourth, and perhaps most importantly, there is currently no incentive for its use. All the fishermen interviewed stated that they are not required, by BFAR, the port, or buyers, to produce catch documentation, nor do they receive any sort of price premium for fish with catch documentation. The fishermen interviewed by the ET did see potential benefits from the system, though mostly from additional modules that would allow them to track fuel and hold temperatures. They uniformly responded that the emergency system and communication with shore would be appreciated but were less important. When asked, all the fishermen interviewed stated that they would not be willing to pay for the system at present. Stakeholders outside of those directly involved in transponder development also consistently expressed skepticism that small-scale fishers would be willing to pay the price provisionally estimated by FAME (800-1,000 PHP or $16-20 per month for the base model with VMS and two-way communication; more for additional modules). Indeed, one local government official noted that small scale fishers often have difficulty paying the annual licensing fee which is roughly 700-1,000 PHP per year. USAID Oceans notes that they are working on ways to ensure that the system costs are not borne by small-scale fishermen. EXPANSION SITES The expansion sites phase I include locations in Vietnam, Malaysia, and Thailand. In Vietnam, the eACDS pilot project is underway led by SEAFDEC and supported by Fisheries Marketing Organization of Thailand (FMOT), USAID Oceans, and Japan Trust Fund. In Malaysia, the proposed expansion site is at Kelatan. In Thailand, the proposed site is at Songkhla. USAID Oceans’ final year workplan specified technical support and capacity building with Expansion I (Thailand and Malaysia) and II (ASEAN and CTI countries, working with local counterparts to support eCDT, planning, sustainable fisheries management, and human welfare interventions).37 In Malaysia, a gap assessment and rapid partnership appraisal was conducted by USAID Oceans in early 2018, through discussions with over 60 stakeholders from government, civil society, and industry actors, followed by a Validation Workshop in October 2018. It was reported that there are an average 20-24 forms and processes required across six agencies to complete a typical seafood export transaction for the EU or the US. Furthermore, these forms are currently housed in eight separate technology platforms. 38 The gap analysis provided useful information for CDT development as well as new orientation of the CDT issues with fishing industry stakeholders. It was noted in the report that there are manual but basic traceability processes in place such as stock tracking as required by their buyers for the processing sector, and for the completion of export and import permits. The interest in CDT was tempered by questions on cost implications, the availability of internet connectivity, and the need to demonstrate a clear business case and value added of eCDT for companies that goes beyond compliance to basic requirements. It was recommended in the Gap Analysis report that an appropriate pilot site capable of demonstrating the value-add of an eCDTS be identified, with the potential to run parallel pilots in Kelantan and another site to be determined, possibly Sabah or Sarawak. Specifically, USAID Oceans offered to share its experience, best practices, and knowledge from its ongoing implementation of eCDTS in the learning sites, facilitate Learning Exchanges, convene annual Technical Working 37 SEAFDEC, Report of the Forty-first Meeting of the Program Committee, 5-7 November 2018, p.261 38 USAID, The Oceans and Fisheries Partnership, Malaysia Capacity Gap Analysis and Partnership Appraisal, November 2018, p. 20 24 Group Meetings, and invite relevant stakeholders from Malaysia to participate in workshops and meetings during the Activity’s third and fourth year.39 Interviews with the fisheries departments in Malaysia and Thailand indicated concerns about the need for urgent follow-up information on USAID Oceans plans and budgets for the pilot work in their countries. USAID Ocean’s work plans for the Thailand and Malaysia sites have not been finalized and the stakeholders are unsure what commitments and activities are required. In Malaysia, the Department of Fisheries staff noted that Kelantan was one of the first sites discussed with USAID Oceans for a learning site and has yet to make any progress on actual plans. SEAFDEC staff stated that the Programming Committee of SEAFDEC is also proposing eACDS pilot projects at two sites in Malaysia – Selangor and Kelatan provinces. A CDT expansion roadmap was proposed in the Mid-Term Partnership Review report 40 , but according to USAID Oceans reports, the CDT Guidelines and Roadmap were still being developed. Also, Thailand government staff were unaware fisheries planning advice had been provided by USAID Oceans for Andaman Sea and the Gulf of Thailand. The Capacity Gap Analysis was completed for Vietnam from October 2017 to February 2018. The report noted the EU-issued yellow card concern driving interests in CDT, but challenges also exist in the form of the country’s large fishing fleet, sizeable small-scale fishery, and weak cooperation and partnership between stakeholders, such as between the ministries and actors along the seafood supply chain. 41 The report recommends applying the eACDS along with USAID Oceans’ recommendations for PPP opportunities to kick-off Vietnam’s eCDT efforts, with the possibility of setting up a pilot site at Nha Trang (Khan Hoa province). The report suggests that a PPP can be set up between the Department of Fisheries and willing industry players to establish a safe test environment for government and industry to trial different approaches towards acceptable eCDT, with industry providing regular input and offering their fleets and supply chains as case studies for the application of an eCDTS.42 39 USAID, The Oceans and Fisheries Partnership, Malaysia Capacity Gap Analysis and Partnership Appraisal, November 2018, p. 24 40 May 2018, p. 3. 41 USAID, The Oceans and Fisheries Partnership, Vietnam Capacity Gap Analysis and Partnership Appraisal, November 2018, p. 5. 42 USAID, The Oceans and Fisheries Partnership, Vietnam Capacity Gap Analysis and Partnership Appraisal, November 2018, p. 34. 25 ECOSYSTEM APPROACH TO FISHERIES MANAGEMENT (EAFM) The expected results from the EAFM component are specified as “management constituency strengthened and sustainable fisheries management plans implemented in two learning sites and relevant fisheries management areas.” These results also include “key partners and industry stakeholders engaged in the development and use of the CDTS/FIS”. The 2019 USAID Oceans workplan describes “Objective 3: Develop and strengthen human and institutional capacity of regional organizations to conserve marine biodiversity through an EAFM”. The EAFM component complements the other Activity components through use of a fisheries management framework that incorporates ecosystem variables, catch information and traceability, and human considerations within non-prescriptive plans that align with existing policies frameworks and implementation programs in each country. The development and demonstration of an EAFM Conclusion USAID Oceans’ focus has been on the development of tailored eCDT systems and technologies in the two learning site countries. They have established important regional dialogue on CDT development opportunities and challenges. While USAID Oceans has been effective in raising awareness and knowledge about eCDT systems at the regional and national level, they have not been as effective thus far in raising the capacity of SEAFDEC to guide countries independently in eCDT development. This is a function of the program design rather than delivery. In Indonesia, the development of an eCDT system is in the early stages and a full assessment of effectiveness will require more time for operational refinement of the technologies and effective connection between supply chain components. The learning site experience has provided essential testing information with a selected group of technologies and systems and a small set of willing partners. It provides preliminary information on the potential viability and adoption of the technologies at the Bitung site. Expanding the electronic capture of catch data and linking it to buyer and processor operations is a key focus for further development. Similarly, in the Philippines, the evidence shows that USAID Oceans helped catalyze the development of a downstream traceability system with BFAR. At this point, however, it still requires significant revision, to both policy and the system design, followed by additional pilot testing prior to formal rollout for wider use. USAID Oceans has also supported the development of an innovative transponder with the potential to provide catch documentation, VMS, and two-way communication for small-scale vessels, yet the system is still in development and has only received small-scale testing and use. Moreover, while the design envisions the data from the transponder feeding into BFAR eCDTS; that is currently not occurring. Indeed, BFAR (both national and regional), local government unit staff, and other key stakeholders in General Santos were largely unaware of the details of the FAME transponders and the progress of its implementation. Finally, while stakeholders uniformly agreed that the development and implementation of these systems would be a significant milestone, they would still leave a large gap in catch documentation for the larger boats not covered by the FAME transponders. The Philippines does have a log sheet that these vessels complete and submit upon landing (and this eventually could be integrated for electronic point of catch submission through satellite communication systems), but at present, the ET understands that this is completed manually and is not yet integrated into the BFAR eCDTs. USAID Oceans has offered to test Pointrek systems but BFAR has not yet approved the satellite ( ) f 26 approach by USAID Oceans has benefited from regional EAFM training and workshops of the SEAFDEC-Sweden Project and earlier training by other funders.43 The EAFM planning activities are outlined in Table 3 and Annex 8. According to Activity records, there were over 600 training exposures in EAFM (Table 1, 2.2). The outputs also include guidance on rapid appraisals of fisheries management systems (RAFMS) and the development of requested sustainable fisheries management plans (SFMPs) in the learning sites. Table 3: EAFM Plans under USAID Oceans, March 2019 Plans Description Current status 1. Sub-Regional Plan for Managing Transboundary Fisheries in the Sulu-Sulawesi Seascape Voluntary ‘framework’ plan with ecological, human welfare and governance goals, objectives, and management actions to be undertaken by willing governmental agencies, non￾governmental partners, and multi-lateral/regional organizations focused on fisheries management and biodiversity conservation in Indonesia, Malaysia, and the Philippines. This plan was endorsed on December 9-15, 2018 by Malaysia and Philippines governments, and the Seascape Working Group but not endorsed by the Indonesia government due to sovereignty concerns. On 12 March USAID Oceans submitted to CTI Regional Secretariat (RS) the finalized and updated version of the SOM14-Endorsed Sub-regional EAFM Plan, reflecting the full scope of all review comments and inputs received from NCCC Indonesia on 19 February 2019, via CTI RS 2. SFMP for Sarangani Bay and Adjacent Waters of the Sulawesi Sea in the Philippines The learning sites EAFM plan for BFAR 12. The plan incorporates CDT and human welfare elements for Sarangani Bay in the Philippines (BFAR #12 ‘General Santos learning site’) and provides a guide to Sarangani Bay and Sulawesi Protected Seascape. As of March 2019, this has been completed. On 29 March, USAID Oceans finalized and submitted the SFMP to USAID RDMA. 3. Fisheries Management Annex to the Sarangani Protected Area Plan in the Philippines Focuses on local government support for small-scale fisheries in specific areas where permitted in the protected area. During the final review session 12 February 2019, USAID Oceans received updated information from local governments and DENR. The plan is being reviewed and finalized, and will be submitted to USAID RDMA in Q3 FY19 4. SFMP incorporating CDT and human welfare elements for Indonesia Fisheries Management Area 716 A sub-national area plan 716 in North Sulawesi Indonesia (‘Bitung learning site’) that demonstrates the broad EAFM approach being promoted by USAID Oceans, and in line with the Sub-Regional Plan for Managing Transboundary Fisheries in the Sulu-Sulawesi Seascape. As of March 2019, this has been completed. On 27 March, USAID Oceans has finalized and submitted the SFMP to USAID RDMA. 43 E.g., “Regional Workshop on Piloting E-EAFM LEAD Materials” on 27-29 June 2016 in Rayong Province, Thailand, with funding support from the FAO/GEF/SEAFDEC/REBYC-II CTI Project; Training Course on Essential Ecosystem Approach to Fisheries Management (EAFM) in Myanmar 14 to 18 December 2015, and in Lao PDR from 2 to 7 November 2015. 27 5. Fisheries planning advice for Gulf of Thailand Through SEAFDEC consultative processes, the input will address threats, issues, objectives, and management options. On 14-15 March 2019, USAID Oceans engaged with FAO – APFIC during the Food and Agriculture Organization’s (FAO) Regional Workshop on “GoTFish: Promoting Blue Economy of the Gulf of Thailand through the Ecosystem Approach to Fisheries Management (EAFM)”. During this event, USAID Oceans submitted a summary of its August 2017 regional fisheries management planning workshop outputs on EAFM planning for the Gulf of Thailand through a presentation and printed report summary. FAO integrated these outputs into a Gulf of Thailand EAFM Framework as well as content within an FAO proposal submitted to the Global Environment Facility (GEF7) Trust Fund. 6. Fisheries planning advice for Andaman Sea Through SEAFDEC consultative processes, the plan will address threats, issues, objectives. and management options This is currently underway; as per Year Four Work Plan. During Q3 and Q4 of FY19, USAID Oceans will provide to FAO a printed report summary from the August 2017 regional fisheries management planning workshop outputs regarding EAFM planning for the Andaman Sea. USAID Oceans will explore with FAO on providing technical inputs into the Bay of Bengal Large Marine Ecosystem (BOBLME) Project Phase 2 EAFM participants reported increased understanding and skills in EAFM work (effects on fisheries management programs are not known). Respondents in both Indonesia and the Philippines almost universally expressed satisfaction with their level of participation in the development of EAFM plans supported by the Activity. The lone exceptions were some small-scale fishers in the Philippines who felt the perspective of small scale and/or handliners was not sufficiently incorporated in the development of the plans. Despite the lack of Indonesian government approval of the Sub-regional Sulu-Sulawesi Plan, it is still viewed as a useful agreement for lower level fisheries protection and management by the approving agencies. No information was found on the current efforts to implement the Sub-regional Sulu￾Sulawesi Plan one year after approval. The local authorities in the Philippines strongly supported the EAFM area plans under finalization, but local capacity to implement the plans is unclear. Local government officials, who would be responsible for implementation, noted that implementation would require budget allocation and resources, as the offices currently do not have sufficient capacity or resources. 28 HUMAN WELFARE AND GENDER EQUITY (HW/GE) CAPACITY BUILDING The expected results from this component are “human welfare, particularly gender and labor considerations, are incorporated into sustainable fisheries management of ASEAN and CTI member countries”. There have been significant efforts by donors and SEAFDEC to raise important issues on HW/GE objectives. USAID Oceans is viewed by nearly all stakeholders interviewed as a leader in raising awareness of HW/GE in the fishing sector.44 USAID staff and contractors also participated in many workshops, one of which led to a draft policy brief: “Applying Human Rights-based and Gender Equity Approaches to Small-scale Fisheries in Southeast Asia”.45 In Thailand, one respondent noted that at sea communication, as provided by some of the USAID Oceans supported technologies may also contribute to the reduction of human trafficking risks in the fishing industry in Thailand and other countries. The Activity has sponsored Regional Gender Workshops and completed country Labor and Gender Rapid Appraisals (Annex 8). A significant output has been the Gender Research in Fisheries and Aquaculture: A Training Handbook, which “was developed to build the capacity of practitioners, experts, and students working in fisheries and aquaculture in gender theory and practice through interactive presentations and case studies.” USAID Oceans reports that these workshops and documents draw heavily from the experiences and examples from the two learning sites. It was emphasized in the interviews that the HW/GE discussions and related outputs have increased the general understanding of how these issues relate to fisheries and the entry points for action. None of this existed before the introduction of these issues in the SEAFDEC-Sweden project and in USAID Oceans Activity. The inclusion of sex-disaggregated data in FIS is a significant change in standard procedures for fisheries data collection. While stakeholders consistently noted an increased awareness of these issues, there was less evidence of actions taken to improve gender equity beyond the establishment of gender focal persons in SEAFDEC and BFAR. USAID Oceans has recently signed agreements with partners in both Indonesia and the Philippines for projects related to HW/GE, but at the time of the evaluation, the workplans and objectives were still being developed. The interviews also noted that expansion site participants may have less understanding of why these issues are included in the Activity; some claim that gender is not an issue in their fisheries sector. SEAFDEC also requested further 44 E.g., Experts Workshop on Regional Approach for the Implementation of FAO Voluntary Guidelines for Securing Small-scale Fisheries: Human Rights Based Approach and Gender Equitability, Sept 26-28, 2017; Training Workshop on Gender Analysis in Fisheries Sector for SEAFDEC Gender Working Team, Mar 5-8,, 2018 IUCN-MFF-SEI-GWT, Regional Dialogue on Gender Dimension in Coastal and Fisheries Resource Management in South Asia and Southeast Asia, Nov 15-16, 2018. 45 SEAFDEC/Sweden, Policy Brief: Applying Human Rights-based and Gender Equality Approaches to Small￾scale Fisheries in Southeast Asia, 2018. Conclusion USAID Oceans has been effective in leading a participatory process for the development of SFMPs at multiple levels that engaged stakeholders from a variety of sectors. The scope of these plans to include human as well as ecosystem issues and to link CDT system data to fisheries management decisions in the context of these plans is an important innovation. The key challenge moving forward will be in generating the ongoing commitment, resources, and capacity within ministries and local government units for implementation of these innovative plans. Closely monitoring the initial stages of implementation is important. 29 development of the Training of Trainer (TOT) gender analysis and training modules and support for development of their capacity to implement these trainings. EVALUATION QUESTION 2 What factors (both internal and external) have enhanced, diminished, or have no effect on achievement of USAID Oceans expected results in reducing IUU fishing and strengthen fisheries management? This EQ focuses on the key internal Activity factors and external factors influencing IUU fishing and fisheries management. Nearly all respondents noted the complexity of IUU fishing and fisheries management, involving not just the fisheries sector, but also many other related areas including governance and policy-making, labor, ports, and trade. Respondents also noted that the issues varied by country, and one respondent highlighted that the issues differ at the regional, national, and local level. USAID Oceans staff and most other respondents also agreed that a project of USAID Oceans’ size, scope, and duration is unlikely to have significant impacts on IUU fishing, at least during the project duration, and that complementary external factors are critical for making sustainable change in IUU fishing. Given this complexity, the ET presents factors influencing IUU fishing and fisheries management from two, sometimes overlapping, perspectives below. First, the ET considers factors internal to the Activity, defined as those factors the Activity was directly intended to address. Second, the ET looks at factors external to the Activity. Although it is clear that the Activity logic envisions influencing some of these external factors further down its intended results chain, these are seen as more indirect influences that are much less within the manageable interest and outcomes of USAID Oceans. INTERNAL FACTORS AFFECTING REDUCTION IN IUU FISHING AND IMPROVEMENTS IN FISHERIES MANAGEMENT The most commonly mentioned internal factors were development and implementation of eCDTS and, to a lesser degree, the development and implementation of fisheries management plans (FMP) based on EAFM principles that utilize this data. ECDTS The implementation of an eCDT system was almost universally listed as a key tool for combatting IUU fishing and improving fisheries management. Fisheries officials in both the Philippines and Indonesia noted that policy development followed by monitoring and enforcement alone would not be enough, as resources are not sufficient for effective monitoring (discussed further below in external factors). In this context, they noted that eCDT could be an extremely valuable tool for reducing IUU in two main ways. First, it provides a mechanism for compliance with international market pressures for CDT (discussed below in external factors) which reduces IUU. For fishers and processors to enter the export value chains, they must demonstrate that their catch is legal, reported, and regulated, and the eCDT system, in theory, allows a mechanism to efficiently do that. Conclusion USAID Oceans (along with the SEAFDEC-Sweden project) has significantly enhanced awareness of the importance of gender and human welfare concerns in fisheries and produced tools for gender analysis and integration into fisheries management. They have supported the integration of HW/GE into SFMPs and in the KDEs of eCDT systems, but to date, these actions have had little, if any, appreciable impact on the lives of women and other marginalized groups. Translating the changes generated by the Activity into improvements for women and marginalized groups will take time and depends heavily on the effective implementation of the SFMPs. 30 Of course, there may still be fraud, but this is still expected to represent a smaller amount of IUU catch relative to the absence of a system for catch reporting or current paper-based systems which still only cover part of the fishing fleet. Secondly, fisheries officials reported that an eCDT system would allow for more targeted monitoring and verification, making monitoring and enforcement more efficient. All of this requires a functional system and strong policy framework. eCDT adoption and use among a set of ‘early mover’ partners provides a very modest direct reduction in IUU fishing, both because they represent a small portion of the fishery, and because those electing to participate as early movers are those most likely to comply with paper-based or other systems and regulations (for example, all processors interviewed noted that they are using paper-based systems currently for CDT). For eCDT to have significant effects on IUU fishing, it must reach beyond early movers, and extend widely, including among the small-scale fishers for whom regulation and monitoring has been minimal. PPP The approach taken by USAID Oceans has been to work on both government and private sector eCDT solutions, with the aim of integrating successful technologies from both sides. Since stakeholders recognized the potential benefits of an eCDT system on reducing IUU fishing, there is the potential for partnerships with private sector early mover partners to have a significant effect, but only insofar as the technologies are demonstrated to be viable and a business case is effectively made for their use. However, as implied above and discussed in more detail below under EQ 3, the sustainability of the private sector technologies was questioned by some stakeholders in both the public and private sectors due to the lack of clear demonstration of benefits and the costs associated with the hardware and subscriptions. USAID Oceans notes that the Activity design envisions focusing on incentives and business cases of eCDT technologies during its final year. REGIONAL COOPERATION Likewise, the capacity building and institutionalization of eCDT approaches and guidance within SEAFDEC and CTI-CFF Regional Secretariat also demonstrates potential for reducing IUU, but again, only insofar as they can effectively lead or support other countries in the development and institutionalization of eCDT systems. As described in EQ 1, both SEAFDEC and CTI-CFF Regional Secretariat, as well as USAID Oceans staff, recognized the current capacity constraints on development and institutionalization of national eCDT systems. National level fisheries officials in both learning site countries as well as SEAFDEC also noted the transboundary nature of tuna fisheries and the importance of regional cooperation in fisheries management, exemplified by the work on the sub-regional FMPs. Regional coordination among ASEAN and CTI-CFF member countries supported by SEAFDEC and the CTI-CFF Regional Secretariat provides an opportunity for countries to work together on approaches to reducing IUU, and USAID Oceans has supported many workshops and events bringing together TWG members from member countries. However, there are still constraints in regional cooperation, particularly related to concerns over sovereignty and ownership of fish in each country’s respective Exclusive Economic Zone (EEZ). SFMPS USAID Oceans support in developing SFMPs using EAFM principles with national, provincial, and local fisheries management authorities at both USAID Oceans learning sites clearly has the potential to improve fisheries management, though nearly all stakeholders, especially those in LGUs responsible for implementation, noted that in practice, this will depend highly on the allocation of resources and development of capacity at the local level for implementation. Further, fisheries officials at all levels in both countries recognized that data from an established eCDT system could greatly improve fisheries management in the context of an FMP. However, none 31 of the interviewed officials felt they currently had the resources, capacity, and framework to use such data for fisheries management. USAID Oceans contracted with partners in the Philippines (Mindanao State University) and Indonesia (MDPI) to conduct research on how eCDT data could be used in fisheries management, but these contracts were just starting at the time of the evaluation. HW/GE AWARENESS Finally, the ET found little evidence of how HW/GE awareness and actions being taken by stakeholders within the seafood supply chains would affect IUU fishing, with no respondents noting this as an important factor. EXTERNAL FACTORS AFFECTING REDUCTION IN IUU FISHING AND IMPROVEMENTS IN FISHERIES MANAGEMENT The extensive array of external variables associated with the Activity goals is reflected in the complex learning sites results chains in Annex 2. Respondents particularly focused on four main factors. MARKET PRESSURES By far the most consistently and strongly noted factor, internal or external, was market pressures. Among market pressures, the most prominently noted was the EU ‘carding’ associated with the EU Council Regulation 1005/2008, which requires countries wishing to export fish to the EU to take actions to reduce IUU fishing in line with EU requirements.46 Government officials in Indonesia, the Philippines, and Thailand all reported that this served as a strong incentive for their governments to take strong measures to reduce IUU fishing. This included serving as a catalyst for the development of documentation and traceability schemes. Likewise, all processing companies interviewed noted that the market pressures, particularly the EU requirements, provided a strong incentive for them to comply with CDT requirements. Processor respondents, more in the Philippines than Indonesia, also noted that the US Seafood Import Monitoring Program (SIMP) provided additional incentives for documentation and traceability, although this was less frequently noted relative to EU requirements. SIMP takes a different approach from the EU carding program, as it places the responsibility of compliance on the seafood importer in the US. The importer then passes these requirements on to their suppliers in other countries, but this does not create the same sort of pressure at the government or sector level. 46 Countries that are not meeting EU requirements are issued ‘yellow cards’, which serve as warnings that the country must take clear steps to address identified shortcomings. Countries that do not sufficiently address these shortcomings can be assigned a ‘red card’ which prohibits those countries (and companies operating there) from exporting seafood products to the EU. Conclusion The most important internal factor in relation to reducing IUU fishing is the implementation of an eCDT system. A fully developed and implemented eCDT system has the potential to significantly reduce IUU fishing and improve fisheries management, particularly in a context of supportive external factors (i.e. national policies and market pressures). To the extent USAID Oceans helps to generate eCDT systems that continue to grow after the conclusion of the Activity, the effects on IUU could be significant over time, particularly if additional external factors are addressed, as discussed below. 32 Lastly, it is important to note that small-scale fishers did not report market pressures as a significant contribution to IUU fishing. At present, there is little pressure, and hence compliance, with catch documentation among the small-scale fishers interviewed. They were not being required to submit catch documentation by their buyers or middlemen, even in cases where they suspected the fish would go on to export. Additionally, none of the fishermen interviewed reported any price premium or incentive offered by any buyer for catch documentation. NATIONAL POLICIES A second important external factor that was consistently noted in both Indonesia and the Philippines by actors from all sectors, including fishers, is national policies related to the fisheries sector. While regulations tied specifically to responding to EU requirements are addressed above, there have also been additional regulations, particularly in Indonesia, that were less directly linked to the EU requirements. In 2014, the Indonesian Ministry of Maritime Affairs and Fisheries implemented policies aimed at reducing IUU fishing, particularly by foreign vessels. This included a moratorium on fishing vessels of foreign origin, with strict enforcement and penalties, including sinking of vessels. Both Indonesian and Philippine fishers noted that prior to the moratorium, vessels from the Philippines and other countries, including small-scale vessels, routinely fished in Indonesian waters, but following the moratorium, they reported that this practice has essentially stopped completely. The moratorium was reported as having significantly different effects in Indonesia and the Philippines, with fishers in Indonesia reporting increased catch per unit effort (CPUE) and fishers in the Philippines reporting decreased CPUE. ENFORCEMENT CAPACITY A third frequently noted external factor was the monitoring and enforcement capacity of the national and local governments, which was noted by all local government and fisheries officials interviewed. LGUs in the Philippines noted that they are not able to monitor fish that are not landed at a port, and two of the three LGUs visited did not have port facilities. All LGUs in the Philippines reported that they do not have sufficient resources, including both human resources and boats, for effective monitoring and enforcement. Moreover, as noted above, small-scale fishermen reported not being required by buyers to submit catch documentation. Accordingly, a large portion of the fish caught in the small-scale sector appears to be unreported. This is a significant portion of overall catch for some tuna species. According to the Philippine Statistics Authority’s Fisheries Statistics of the Philippines, municipal fishermen accounted for 33 percent of the total Yellowfin catch and 48 percent of the Bigeye catch in 2016. LIVELIHOODS A final important external factor noted particularly by fishermen and local fisheries officials as relevant for small-scale fishermen was the lack of alternative livelihoods or incentives for documentation. Without alternative livelihood options, small-scale fishermen reported that they would continue fishing, as employment opportunities are limited in the learning sites, and, without incentives or enforcement for documentation, much of their catch would remain unreported because there is no reason to change current fishing practices. Moreover, the small-scale fishermen in the Philippines noted that they are highly dependent on financers in order to purchase provisions required for each trip and that this locks them into selling to these financers at the price and terms dictated by the financer. They expressed confidence that any premiums or incentives for documentation would be captured by the financer, middlemen, or processor and not increase the price offered to the fishermen. They uniformly reported, as did an industry association for handline fishermen, that some level of empowerment, through associations or financing options, would be required for small-scale fishermen to comply with documentation or adopt new technologies. While some actors in Indonesia were working towards Marine Stewardship Council (MSC) certification, 33 they noted, as did respondents in the Philippines, that such certification is time consuming and requires significant upfront costs. EVALUATION QUESTION 3 What evidence(s) exists that the participating ASEAN countries, especially in the learning sites and a selected expansion site (Kelantan, Malaysia) would potentially sustain the adoption and implementation of the eCDT systems to counter IUU fishing and strengthen fisheries management as a result of USAID Oceans' support? The EQ focuses on (a) the likelihood of sustainability of eCDT systems, (b) the factors influencing sustainability, and (c) the specific actions required for sustainability. There is considerable evidence for the sustainability of some elements of the eCDT, EAFM, and HW/GE work, primarily due to the global and regional forces that are driving traceability and sustainable fisheries management. This is reflected in the commitments of ASEAN Council for further development of eACDS, the ongoing recognition of the issue, helped by USAID Oceans, and continued enforcement of market pressures for documentation and traceability. Stakeholders reported that sustainability of specific outputs of USAID Oceans will depend upon a range of factors, including technical viability of proven CDT technologies and processes, stakeholder willingness and ability to implement EAFM plans, substantive measures to address HW/GE concerns, and the future institutional development of some form of regional support services by SEAFDEC and CTI-CFF. LIKELIHOOD OF SUSTAINABILITY OF ECDT SYSTEMS Although the external factors noted in EQ 2 demonstrate strong evidence for sustained development and implementation of eCDT systems, the evidence on sustainability of the specific eCDT systems and technologies supported by USAID Oceans is variable. NATIONAL SYSTEMS Both BFAR in the Philippines and MMAF in Indonesia have demonstrated and reported in interviews a strong commitment to completing and implementing their respective downstream eCDT systems, BFAR eCDTS and STELINA. Both ministries reported having the internal resources and capacity to refine their systems and are currently refining pilot versions that they expect will be completed within the year. However, both expressed concerns about their current level of resources, capacity, and readiness for large scale implementation of these systems. The situation is similar with the work on eLogbooks in both countries. While both countries are working on pilot versions of the eLogbook for upstream documentation, officials in both countries expressed concern about resources for implementation, including in supplying the hardware needed for reporting. The constraints to full implementation appear much larger for the eLogbooks than for the downstream eCDT systems. Since the national downstream systems in the Philippines and Indonesia were developed based on existing systems and the specific requirements of those countries, they are considered as very specific, customized solutions. Accordingly, for the same reasons that USAID Oceans decided to focus on country level, customized solutions, rather than regional systems, there is a very low likelihood that other ASEAN or CTI countries would take up the national eCDT systems from the Conclusion Given the current status of eCDT system development, external factors are currently more significant than internal factors in terms of their perceived impact on IUU fishing. Market pressures, government policies, and capacity for monitoring and enforcement are particularly important factors. 34 Philippines or Indonesia. However, SEAFDEC and CTI-CFF noted that their member countries have expressed strong interest in developing eCDT systems, including through refinement of eACDS in some locations. This was confirmed in interviews with fisheries officials in Thailand and Malaysia. COMPLEMENTARY TECHNOLOGIES The evidence of sustainability of the private sector technologies supported by USAID Oceans is less clear, in large part because these technologies are still under development, without complete business cases or clear demonstration of benefits or results. All the technologies supported (Trafiz, Trace Tails, Pointrek, and FAME) are either currently still under significant refinement or have only been trialled with one organization (Trace Tails). The evidence for sustainability of each technology is as follows: ● Trafiz: Despite the current need for additional refinements and the current low usage rates, there is some evidence of potential for scale-up and sustainability given that the application can be used free of cost, and it does present a current benefit, of tracking costs and displaying sales, to middlemen. ● Trace Tails: The likelihood of sustainability among the two supported first movers, one that is currently using and one for whom MDPI is currently under contract to refine and implement the product, is high given the reported efficiency benefits. Scale-up among other processors is uncertain due to the significant costs associated with refining the system for each processor and lack of documented data on benefits. ● Pointrek: The likelihood of sustainability is unclear. While boat owners and processors recognize the potential for significant benefits from the system, they were, at the time of field work, not fully operational, and the costs, both upfront and recurring, are significant. The owners interviewed were uncertain of the system’s continued use until they were able to fully test the systems and assess the benefits. ● FAME: The evidence for sustainability is low, at least among the targeted small-scale fishers, unless a source of financing is identified. Nearly all stakeholders outside of USAID Oceans and FAME expressed doubt that small-scale fishers would be willing and able to pay the recurring costs for the system, even if the envisioned benefits are realized. For all of these products, the implementation of a national eCDT system that these technologies can directly feed into would serve as a strong factor to drive use. USAID Oceans is working on an interoperability feature that would connect other systems, including Pointrek, into the MMAF elogbook system. FACTORS INFLUENCING SUSTAINABILITY The following sustainability factors were identified in the evaluation review of documents and interviews with stakeholders. 1) Leveraging US and EU regulation impact on CDT adoption: The EU regulatory impact on CDT concerns and action by seafood exporting countries in recent years was mentioned many times in interviews as a catalyst for change. Future expansion of CDT systems is enhanced by the global market demand for sustainable seafood and compliance with export regulations including US SIMP. 47 The USAID Oceans’ development of eCDT will be supported by SIMP and other international import requirements. 47 Seafood Import Monitoring Program, https://www.regulations.gov/document?D=NOAA-NMFS-2015-0122- 0112 Retrieved March 20, 2019. 35 2) Implementation of national policies requiring documentation: The development and implementation of national policies requiring electronic catch documentation and traceability was reported consistently by suppliers and processors as a strong incentive for uptake of eCDTS systems. This would provide a strong incentive for sustainability of both the national systems and the complementary technology supported by USAID Oceans. 3) Business case for private sector technology: The interviews noted that the business case is central for private sector sustainability and replication of Activity results. Production efficiencies offered to suppliers and processors by digital technologies, and the marketing and price premiums associated with certified sustainable seafood were noted as strong incentives to further drive the adoption of eCDT systems. 4) Synergies between USAID Oceans and other programs and projects: There is scope for some of the Activity outputs to be sustained or expanded by related USAID activities, regionally or bilaterally. These collaborative regional-bilateral linkages provide an opportunity for complementary actions at various spatial scales in support of reduced IUU fishing and enhanced fisheries management through the integrated EAFM approach and eCDT inter-operability that encompasses trans-boundary fisheries where such opportunities exist at a larger scale. However, currently these regional-bilateral and multilateral program linkages are not well defined. 5) Political readiness for CDT at regional and national levels: The interviews noted significant commitment for further actions on reduced IUU fishing that could support sustainability and progress on CDT design and development, EAFM expansion, and HW/GE integration into fisheries. The global interest and the side array of partners involved in USAID Oceans also enhance the potential for follow-up funding. But the interviews also highlighted capacity constraints at regional, national, and local levels. Future regional services at SEADEC and CTI-CFF and for specific national CDT systems institutional requirements remain unclear and a risk to sustainability when the financial support of USAID Oceans ends in 2020. SUSTAINABILITY STRATEGY The final year of the Activity will require concentrated effort to consolidate results and to maximize SEAFDEC/CTI-CFF and national ownership of the Activity outputs. Six key actions for sustainability were identified from the document review and interviews. These are integrated into the recommendations section of this evaluation report and only noted here. 1) Complete, compile, and disseminate eCDT supply chain demonstration results from the Philippines and Indonesia learning sites 2) Elaborate and disseminate recommended eCDT – FIS integration model 3) Scale up and institutionalize eCDT systems 4) Strengthen regional coordination and integration 5) Identify implementation opportunities and processes for the approved EAFM outputs 6) Finalize and disseminate HW/GE TOT modules. 36 Conclusion The likelihood for sustainability depends highly on the component of the system (whether national eCDT system or one of the private sector technologies), the sector, remaining Activity efforts, and external factors. The national eCDT systems have the highest prospects for sustainability, but they require further support in their final refinement, and more significantly, in their wider implementation and financing. The capacity and resources required for wider implementation are currently not yet in place in either learning site country. Nevertheless, USAID Oceans has generated significant momentum in the introduction and development of eCDT and demonstration of EAFM planning over the past four years and produced significant technical outputs that can guide further development of eCDT and EAFM. The sustainability of USAID Oceans-supported technologies is closely linked to the financial costs and incentives that underpin change in fisheries management. Adoption rates will depend on USAID Oceans and partners demonstration of the technology benefits, but it may also require financing or subsidization opportunities, particularly for small-scale fishermen. Finally, the sustainability of all these system components depends highly on continued market pressure for CDT. 37 CONCLUSIONS AND RECOMMENDATIONS CONCLUSIONS EQ 1 - PROGRESS ON KEY OUTCOMES The evaluation found that USAID Oceans has made a significant and timely contribution to the design, development, and piloting of eCDTs; the development of EAFM plans; and in raising awareness on HW/GE including integrating it into fisheries management planning. A broad approach has been promoted that combines CDT, better data management for FIS, and flexible fisheries planning that encompasses both technical and social issues. USAID Oceans has contributed to setting an ambitious agenda for transformative changes to fisheries management in the region and engaged a large range of partners and supporters. Working on so many fronts with such an array of partners and with high expectations of short-term results also presents challenges and choices about strategic focus during the final year of implementation. The Activity’s regional approach contributed to changes in multiple countries in the region, particularly in Indonesia and the Philippines. While it is unclear if a more focused bilateral approach would have made more progress to this point, particularly as there were multiple steps along the way that required time consuming government buy in or approvals, it does seem clear that any activity focused on institutionalization of eCDT systems would benefit from a more intensive, targeted country presence. The institutionalization of the systems developed and/or tested by USAID/Oceans will require much more intensive fieldwork, collaboration, and capacity development with local stakeholders, particularly government officials and fishers. REGIONAL COORDINATION The expected regional coordination functions of SEAFDEC and CTI-CFF are not well-defined (other than facilitating workshops). They both have limited resources and expertise for CDT leadership and the USAID Oceans capacity building to date has been focused on short-term training rather than the necessary institutional development to become regional leaders on the innovative concepts being promoted by the Activity. Neither SEAFDEC nor the donors have a clear concept of the regional role to be pursued. Work to date in SEAFDEC on CDT, EAFM, and HW/GE issues has been jointly supported by Japan, Sweden, and US, the latter two programs ending soon. Ongoing regional support by these partners is uncertain. ECDT USAID Oceans has developed the eCDT design parameters and engaged private sector partners to assist in testing solutions, raising the awareness and understanding (and expectations) of many stakeholders along the way, but operationalizing eCDT on-the-ground has so far been limited to a few ‘early movers’ testing the technologies in a few supply chains. In Indonesia to date, only BOGI has installed the Trace Tales system (although Nutrindo is anticipated). This is a modest beginning, with pilot learning from a few partners expected over the next few months to facilitate momentum with other seafood supply chains in the Bitung area. There is the potential for implementation of a linked upstream and downstream system within one or two supply chains that integrates USAID Oceans supported technologies upstream with the STELINA system downstream, but this may not be achievable during the available time. In the Philippines, the completion of an operating downstream system (BFAR eCDTS) seems within reach during the Activity period, but gaps will remain in the supply chain, with no comprehensive solution for point of catch documentation or processing (as Trace Tales does in Indonesia). While the completion of these system components, and potentially of a full supply chain in Indonesia, represent a significant achievement, a likely much more significant challenge remains in the scale-up 38 of a system to other suppliers, processors, geographies, and fisheries. There are significant capacity and resource constraints in scaling up once the systems are fully proven and established. The Activity design for eCDT placed an emphasis on end-to-end supply chain traceability through greater private sector involvement in contrast to SEAFDEC’s emphasis in eACDS for public sector enhancement of catch documentation and reporting. These are complementary approaches but, as noted in the MTR report, they created a perception of two different models for responding to the need for better overall CDT. This eCDTs–eACDS duality remains in the view of many of the stakeholders interviewed, even though the approach proposed in Vietnam and Malaysia is more blended and custom-designed based on specific needs in the countries. While it is clear the Activity has raised awareness on eCDT issues and opportunities, interviews with SEAFDEC and country representatives identified capacity constraints to fully establishing eCDT due to insufficient resources, and legal and organizational barriers and complexities of delivery. The challenges of coordinated regulatory and administrative reform across multiple government departments and the few resources available to deliver their planned CDT systems are priorities for many of the countries. These governance issues are limiting factors mostly outside the scope of USAID Oceans, and therefore, some frustration exists about Activity benefits from the perspective of capacity development of the national partners. Some of this gap may be addressed by results of the supply chain pilots to help national authorities move forward on recommended CDT technologies and processes. PARTNERSHIPS Respondents provided multiple views of the role of partner engagements, PPPs and private sector investment. The strategic part of the USAID Oceans Partnership Strategy was not clear to interviewees, and the approach seemed to focus on generating as many partnerships as possible. Most of the partnerships have been generally effective and supportive of Activity implementation, if not always results-oriented in terms of impact on the uptake of the interventions with suppliers, processors, and governments. While USAID Oceans has brought government and private sector partners together, significant partnerships between government and private sector have yet to emerge. In addition, the implementing partners do not always have the needed local presence to promote the technologies. EAFM The EAFM component aimed for “management strengthened and plans implemented”. The capacity development has been greatly appreciated, providing skills development, resource materials, and substantive stakeholder buy-in for those plans completed to date. EAFM is still underway and nearing completion. The central question is whether there is sufficient capacity to effectively utilize the plans for fisheries management decision-making, though this is not part of USAID Oceans’ explicit scope. HW/GE The HW/GE component has also added to the international efforts to change the fisheries and fisheries management toward a more socially responsible and inclusive industry. Many of the stakeholders interviewed commented on how they now had a new perspective about these issues and an understanding of how they interact with fisheries management. Some of the organizations have also appointed gender officers and focal point to carry the issue forward. To make the next step and to actually generate improvements in the lives of women and marginalized groups depends on translating the plans and capacity supported by USAID Oceans into implemented systems, though this falls beyond the current scope of USAID Oceans. 39 EQ 2 – FACTORS The main internal factors affecting the USAID Oceans goals of reducing IUU fishing and enhancing fisheries management relate primarily to the extent to which the Activity can create commitments by industry and governments to invest in CDT and sustainable fisheries management and for the EAFM plans to be implemented. Other factors include limitations associated with a traditional technical assistance Activity design, the complementarity of CDT to other strategies for reducing IUU fishing, and the potential for the USAID Oceans CDT technologies and EAFM processes in a few pilot projects having a larger impact on seafood supply chain and fisheries management practices. The main external factors influencing high level goal achievements relate to the extent of market pressures for traceability and sustainable fisheries; national policies and commitments to accelerate introduction of CDT systems; the capacity to provide for compliance, monitoring, and enforcement; the price of fish (and premium for certified, traceable fish) and profitability of fishing; and the availability of alternative food supply and livelihoods for local small-scale fishers. There are distinct areas of regional and sub-regional transboundary cooperation that require multi￾country responses. Strategies to address regional IUU have been identified by SEAFDEC and could be further pursued, including “regional cooperation on information sharing and development of regional record of fishing vessels; promoting policy dialogues with high level officials; and strengthening of the implementation of regional collaborative programs to follow up on international instruments and regional agreements”.48 EQ 3 – SUSTAINABILITY USAID Oceans has added incrementally to the development of CDT systems in the Asia-Pacific region and completed new, flexible EAFMs, thereby helping to improve tools for management of IUU fishing and fisheries at priority locations. Sustaining momentum for this model requires a longer￾term, country-driven process, and further institutional strengthening of SEAFDEC and CTI-CFF. For specific partners committed to particular Activity outputs, such as certain eCDT technologies (e.g., Trace Tales) and EAFM plans (e.g., local agreements), there appear to be the potential for sustainability and further replication, but for others (e.g., Pointrek, FAME transponders, systems for interoperability of databases) there remain many more uncertainties. In part because they are still being piloted, there is low commitment so far to the eCDT technologies promoted by the Activity and uncertainties to date about the implementation of EAFM plans. The few early demonstrators of Trace Tales show the best potential to maintain and expand results. SEAFDEC will continue to develop and test eACDS with support from Japan. The planned “handover” of important user-tested knowledge products to SEAFDEC, CTI Sec and country partners along with workshop orientation to them will greatly assist subsequent use and sustainability. However, the narrow approach to capacity development and the insufficient course of experiential, on-the-job learning by key partners over the past four years will also limit the extent to which these technical outputs will become fully institutionalized and have an impact beyond the life of the project. The sustainability attributes of USAID Oceans include market pressure through US and EU regulations, proposed dissemination of a series of business cases, operational partners with strong 48 SEAFDEC, Fish for the People, Volume 8, No. 1, 2010, P.18 40 outreach capabilities, a potentially important set of knowledge products, synergies with other USAID activities, and institutional commitments to further strengthening CDT development. RECOMMENDATIONS RECOMMENDATION 1 To facilitate eCDT up-take, USAID Oceans should share the results of the pilot eCDT activities with stakeholders in both the learning and expansion sites. Demonstration of eCDT-based seafood supply chains is expected over the next few months at the Philippines and Indonesia learning sites and at the SEAFDEC Brunei Darussalam pilot project. There will likely be some gaps in the completeness of the final pilot testing but there should be enough evidence from these projects to share the results to date from the components that have been completed and explain the practical issues and options, including regulatory and financing constraints, and challenges linking the stages within the supply chains. USAID Oceans should also conduct additional data collection with both current eCDT technology users/testers and non-users (prospective users) to better understand constraints to adoption and develop appropriate strategies for overcoming these constraints. 49 This information will assist public and private sector stakeholders in their decisions regarding eCDT system up-take. RECOMMENDATION 2 To support eCDT system institutionalization, USAID should support learning site governments in policy advocacy, capacity development, and budget planning and allocation. Following the establishment of national downstream systems and demonstration of a suite of viable complementary technologies, particularly upstream, catch documentation systems for both small￾scale/municipal and larger, commercial fishers, these systems must be scaled up and institutionalized, though much of this work falls beyond the scope or duration of USAID Oceans. While the documentation of business cases, assuming they are viable, will contribute to up-take of the private sector technologies, at least two significant additional areas of support are required for larger, more systematic institutionalization: a. Strengthen fisheries policy: Policy development and related legal, organizational, and fiscal measures will be needed to mainstream eCDT and other innovations into government operations and to push the industry to conform to international norms for sustainable fisheries. National and local governments and relevant ministries will need to take the lead on these projects, but there is scope, and need, for additional support. USAID, along with other donors and regional organizations like SEAFDEC, should provide guidance and advocacy on effective policy examples and development. This policy development support should not be limited to policies or regulations governing implementation of eCDTs but should extend to wider policies influencing IUU fishing and regulations governing implementation of EAFM plans and integration with eCDT systems. b. Funding Allocation and Capacity Development: While establishing supportive policies and regulations is an important first step, regulatory enforcement, particularly within the widely dispersed small scale artisanal fisheries is extremely difficult and other community￾based approaches are needed. In addition to other approaches for monitoring small scale fisheries, national and local governments will need to allocate additional funding for the resources required for implementation of the SFMPs, staff and equipment for monitoring and 49 USAID Oceans’ Year 4 Workplan includes planning for “surveys and in-depth interviews to assess the level of adoption (“uptake”) of eCDT systems, and their impacts (including costs and benefits).” To be most effective these should include non-users as well as those targeted for testing. 41 enforcement, and staff and equipment for collecting and analyzing data from eCDT systems to integrate into fisheries management planning. While USAID is restricted in its ability to provide direct funding support, USAID may still have scope for supporting planning for budget allocation and, importantly, capacity development for implementation. RECOMMENDATION 3 To support eCDT system institutionalization, USAID should finance mechanisms for up-take of new technologies that facilitate eCDT. If policies are developed to support an eCDT system and capacity for implementation and enforcement is developed among ministry and local government staff in the absence of positive incentives for adoption, particularly among small scale fishermen, there is a risk of further burdening and jeopardizing livelihoods for the poorest and most marginalized of fishermen. While business cases might show positive financial benefits of adoption, initial up-take may still be low or burdensome, so financing options should be developed that at minimum help to subsidize or amortize start-up costs. At a broader level, additional options for innovative financing approaches, such as innovative cost recovery strategies like blue bonds 50 that pass premiums or positive incentives through the supply chain should be explored. USAID Oceans is also exploring strategies to passing financial incentives to fishermen, including an approach to one-by-one tuna fishing (essentially catching tuna one at a time) and tying this to Marine Stewardship Council (MSC) processes, certification, and price premiums. There is promise in this approach, which is being pursued by ICTSA with support of USAID Oceans, but it does have challenges particularly in the up￾front costs and requirements for certification, which may require outside support. RECOMMENDATION 4 To encourage support in expansion countries, USAID Oceans should clearly communicate the expansion site plans for all expansion sites. The interviews indicated a high level of uncertainty (as of mid-March 2019) about the plans and budgets to pilot eCDT technologies in Malaysia and Thailand (no response from Vietnam contacts was received). This has led to frustration at some of the expansion sites. USAID Oceans and USAID/RDMA should carefully consider dropping or reducing the remaining expansion sites-II and providing opportunities for those site II TWG members to become involved with the expansion site￾I projects (Vietnam, Malaysia, and Thailand). Focusing resources in this way would increase the likelihood of generating progress in the limited time remaining and allow focusing of communication activities to reduce the likelihood for continued uncertainty or knowledge gaps in Malaysia and Thailand. RECOMMENDATION 5 To facilitate regional expansion and integration of eCDT systems, USAID Oceans should ensure the proposed eCDT guidelines are part of a process aimed at encouraging regional consensus on CDT best practices and lessons learned. A regional framework for customizing eCDT designs should be a key output linked to the eCDT guidelines. The guidelines should encompass lessons learned from all the piloting activities by USAID Oceans and SEAFDEC. The proposed guidelines should also be complemented by a consultative process that will assist in socializing the guidelines and addressing any mis-perceptions of varying 50 https://sdg.iisd.org/news/seychelles-launches-first-blue-bond/. 42 approaches and tools and serve to train relevant stakeholders on applications of the guidelines. The process should serve as a means of regional dialogue between countries on the basic technical rules and steps in building eCDT systems for both large and small-scale fisheries. Additional components of the national eCDT system for various fisheries sustainability, social, economic, and sovereignty issues can be adapted as needed in each country. Moving beyond USAID Oceans, at the regional level, USAID should continue to support SEAFDEC and ASEAN capacity, in a more focused way, in their advocacy and coordination functions, as well as in working to ensure core KDEs are collected in each national CDT system within the region, eventually working towards the ability to transfer limited data across systems. At the bilateral level, USAID should advocate within national ministries on concrete ways that regional cooperation and integration can be incorporated into policies and regulations. RECOMMENDATION 6 USAID Oceans should hold government-industry workshops on the value and business case for public and private sector investment in eCDT in the Philippines and Indonesia as a basis for further advocacy within the region. The willingness within government and the fishing industry to adopt new approaches and technologies, and to provide the financial support or financing mechanisms depends on the availability of information on viability, cost-effectiveness, and risks of making changes in established processes. USAID Oceans should consolidate information on efficiencies and rate of return on investment into a clear business case aimed at both government (cost savings from information technologies and coordinated inter-agency systems) and industry suppliers, processors, and buyer firms (return on investment, risk management benefits). The Ministries of Finance in partner countries need compelling economic evidence for enhanced budgets aimed at improved catch documentation and traceability systems and long-term administrative efficiencies offered by new technologies. Processors are also reluctant to invest without financial analysis of the return on investment. Fish suppliers are especially looking for incentives to adopt easy-to-use technologies; the potential type and scale of such incentives require further consideration. RECOMMENDATION 7 To support EAFM institutionalization, USAID Oceans should identify the opportunities and needs for active implementation of the EAFM plans that have been produced and provide follow-up support and advocacy. Although full implementation of the EAFM plans is beyond the scope of USAID Oceans and is the responsibility of the national and local governments, there are resource and capacity constraints for follow-up implementation by the designated authorities, but they can be reduced by having a set plan for initial implementation and backstopping support over the next year on how to maintain momentum for implementation. To maximize likelihood of institutionalization of these plans, USAID Oceans should support the development of implementation plans and support advocacy for complementary policies, regulations, and budgets. This may lead to additional support from USAID or other donors following USAID Oceans. RECOMMENDATION 8 To optimize data-based fisheries management, USAID should disseminate and build capacity on eCDT – FIS integration. USAID Oceans is working with partners to develop and document initial models for this. However, it is unclear (a) whether they will have time to widely disseminate and build capacity on these models, and (b) whether these models will be based on robustly, widely implemented eCDT systems 43 (as opposed to small scale pilots). In any case, following USAID Oceans, USAID should continue supporting development of these models, particularly as eCDT systems grow in use and evolve. Further, USAID should work with SEAFDEC to disseminate these models and build capacity of national government and local governments to institutionalize these models. RECOMMENDATION 9 To encourage regional leadership, USAID/RDMA should work with SEAFDEC to clarify their role as a ‘regional champion’ for eCDT and support their capacity development to fulfill that role. There is lack of clarity in USAID Oceans’ stated objectives on the specific role that SEAFDEC is envisioned to play in regional eCDT development post-USAID Oceans. Depending on that role, SEAFDEC may need further capacity development and resources, particularly with current funding support from USAID Oceans, Sweden, and Japan ending in the near future. Current institutional capacity for a long-term role in leading CDT systems development in the region is limited. No institutional capacity needs assessment has been completed, partly due to uncertainties about what is expected from regional organizations. The roll-out of eACDS in Brunei Darussalam and now in the USAID Oceans expansion sites, and the increased importance of documentation and traceability for the seafood industry will generate more demand from ASEAN countries for direct SEAFDEC assistance. Currently, SEAFDEC does not have the capacity and resources to support an increase in demand. Further, there may be scope for an increased role for SEAFDEC in contributing to sharing lessons learned and supporting advocacy for institutionalization of eCDT systems and EAFM plans. Similarly, CTI-CFF’s role in eCDT/eACDS should be enhanced and defined in conjunction with their corporate review and strategic planning process. USAID Oceans should assist in formulating institutional design proposals for both organizations, including financing plans for government, industry, and donor consideration. This institutional development initiative should be coordinated with the pending closure of the SEAFDEC-Sweden project and the one-year exit strategy for USAID Oceans to draw out CDT outputs, results, and lessons that need to be carried forward by SEAFDEC and CTI-CFF. As part of this, USAID/RDMA could consider supporting SEAFDEC in bolstering the Information and Training Section so that it can function as a more independent eACDS (or eCDTs, more generally) unit and CTI-CFF a limited set of well-defined eACDS support functions. RECOMMENDATION 10 USAID/RDMA should consider the extent to which bilateral missions can support the capacity and resource requirements for implementation of the national eCDT systems and EAFM plans. Many of the above recommendations require USAID actions at the local/national and regional/international levels, and these must be coordinated to optimally leverage lessons learned (from USAID Oceans and beyond). USAID/RDMA’s role should be focused on the projects that support regional coordination, regional integration, regional level policy advocacy, and knowledge sharing. This includes projects related to SEAFDEC capacity development and development and dissemination of learning materials, including on the post-USAID Oceans developments (policy advocacy and capacity development in support of eCDT system institutionalization, for example). National governments should then draw on support from SEAFDEC and these learning materials, possibly with support of bilateral missions, as available, to develop eCDT systems that fit their needs. Bilateral missions in the Philippines and Indonesia are likely now better positioned to support the recommendations related to institutionalization of eCDT systems and EAFM in their respective countries, with support from SEAFDEC and USAID/RDMA, as the policy advocacy, capacity development, and financing are likely to be country-specific and resource intensive. To be most effective, the activities at the regional and bilateral level need to be better harmonized and strategically linked, as they are currently segmented in different locations within the countries, particularly in Indonesia. 44 RECOMMENDATION 11 USAID/RDMA should assist SEAFDEC in further developing the strategies for small-scale fishing communities to adopt eCDT mobile applications and pilot interventions that enhance access to markets, financing, value￾added processing, and income diversification for small-scale fishing households. Small scale fishermen who depend on fishing for their livelihood will need support for their involvement in eCDT systems, including further development of custom mobile apps to enhance access to markets, improve early warning systems, and conform with government catch reporting requirements. Private sector engagement that has been initiated in the USAID Oceans Activity could be carried forward to focus more broadly on a range of strategies that reduce community vulnerabilities to declining stocks and market risks, enhance value-added, and increase income diversification away from fishing. It is unrealistic to expect small-scale fishers to be willing and able to bear additional costs to participate in eCDT systems unless they can see clear benefits. For any traceability price premiums to benefit small-scale fishermen, they must be empowered, for example in the form of associations or cooperatives, or offered financing opportunities to avoid elite capture by middlemen or financers. Additionally, alternative livelihood strategies could provide additional income streams, reducing their reliance on fishing, particularly during difficult times and severe weather events. Follow-up support aimed at small fisheries, drawing on USAID Oceans experience, and collaborating with SEAFDEC could expand the impact of eCDT technologies for economic and social transformation as well as improved fisheries management. This recommendation is intended as part of the regional effort to reduce IUU and fishing dependency in vulnerable communities, to enhance household incomes and gender equity, and to adapt to climate change through livelihoods diversification. 45 ANNEX 1: SCOPE OF WORK The performance evaluation (covering implementation from May 2015 through December 2018) aims to evaluate the effectiveness of USAID Oceans’ approaches to increasing the ability of regional fisheries organizations to conserve marine biodiversity and combat illegal, unreported and unregulated (IUU) fishing in the Asia-Pacific region. The evaluation will help inform both adaptive management strategies and implementation to effectively sustain the ongoing activity as well as inform decision making for future programming. The primary stakeholders to benefit from the findings of the performance evaluation include: a) USAID for the purpose helping design a future regional marine conservation activity; b) The USAID Oceans team, including the implementer, Tetra Tech, ARD, its main collaborator, the Southeast Asian Fisheries Development Center (SEAFDEC), and its other partners, for adaptive management for the remaining Life of Project; c) Other USAID partners currently supporting USAID Oceans to inform decision￾making on programming further U.S. Government investments and the sustainability of those investments, including: – USAID Bilateral Missions in the Philippines and Indonesia; – USAID Bureau for Economic Growth, Education, and the Environment (E3); – USAID Asia Bureau (USAID/ASIA) d) USAID’s external bilateral, regional, and international partners and key stakeholders addressing marine biodiversity issues across the Asia-Pacific region to share lessons learned, including participating host country government agencies and other international partners (bilateral donors, multilateral institutions, fishing industry and private sector, NGOs, academic institutions, and research institutions). In accordance with the USAID Evaluation Policy, the findings from the USAID Oceans performance evaluation will be shared as widely as possible, with a commitment to full and active disclosure. Furthermore, a summary including a description of methods, key findings and recommendations will be available to the public on-line in a fully searchable form. The dissemination of USAID Oceans evaluation report will be handled by USAID. BACKGROUND Award Information Activity Title USAID Oceans and Fisheries Partnership Life of Activity 5/14/2015 – 5/13/2020 Total Estimated Funds US$19.95 million Implementer Tetra Tech ARD, Inc. COR/Alternate COR Cristina Velez Srinivasan/Craig Kirkpatrick Development Context Southeast Asia’s oceans contain unique, highly productive and diverse marine ecosystems that provide food and livelihood to coastal communities in the region. Fisheries provide a significant percentage of the protein for the region’s poor, as well as an important export product. These fish stocks are declining due to weak governance, socio-economic conditions and ecosystem change. 46 Weak governance of fisheries in the region is characterized by (but not limited to) corruption, lack of stakeholder participation, poor enforcement, weak institutional capacity, overcapacity of fishing fleets, inadequate fisheries information and illegal fishing. Men and women employed in fisheries often are without livelihood alternatives, and a rapidly growing population demands the protein and employment that fishing provides. These conditions result in an open access situation in which the incentives are primarily to take all possible fish catch. This situation drives illegal, unreported and unregulated (IUU) as well as destructive fishing. It also drives a search for cheaper labor that leads to debt slavery and exploitation of vulnerable populations that often include women and girls. Overfishing and destructive fishing put direct pressure on marine biodiversity resulting in species loss and changes in species distribution and trophic structure. This situation reduces ecosystems’ resilience to stresses and exacerbates the negative impacts of ocean warming and acidification instigated by climate change. Coastal community resilience diminishes as the scale and degree of human competition and conflict over remaining fishery and marine resources increase over time. To manage fisheries effectively, Southeast Asian stakeholders need a well-designed set of management principles codified as law, the law needs to be enforced, and there needs to be a mechanism to inform and adapt governance and fisheries management as conditions change. Management principles are legally established in the region, but they are poorly and unevenly enforced. Enforcement in Southeast Asia is clearly outside of the mandate of USAID, but a substantial contribution could be made by creating a data system that informs the governance and the science of sustainable catch and fisheries management. a) Theory of Change and Logic Model The overall goal of USAID Oceans is to combat illegal, unreported, and unregulated (IUU) fishing and seafood fraud, promote sustainable fisheries, and conserve marine biodiversity. The objectives of USAID Oceans are to: 1) implement a financially sustainable catch documentation and traceability (CDT) system that links to Fisheries Information Systems (FIS) and is demonstrated within a regional Ecosystem Approach to Fisheries Management (EAFM) framework; 2) expand use of CDT to areas important for biodiversity in the ASEAN and Coral Triangle regions; 3) strengthen the capacity of regional organizations to conserve biodiversity using EAFM and CDT to combat IUU fishing; and 4) engage the private sector to ensure sustainability and uptake of CDT and EAFM, while advancing regional fisheries governance. USAID Oceans is implemented with congressionally mandated Biodiversity Conservation funds. All Biodiversity Earmark-funded projects must align with the USAID Biodiversity Policy and Biodiversity Code. The Code requires that funded projects must (1) have an explicit biodiversity objective; (2) be identified based on an analysis of drivers and threats to biodiversity and a corresponding Theory of Change (TOC); (3) must have the intent to positively impact biodiversity in biologically significant areas; and (4) must monitor indicators associated with each stated TOC for biodiversity conservation results. Since the Activity’s launch, USAID Oceans has been using a TOC to guide program development and management including monitoring, evaluation, learning and adaptive management. The TOC was developed following the Conservation Measures Partnership (CMP) Open Standards for Adaptive Management. Design began with identification of the project scope (geographic focus), ecosystem targets (key species and habitats, as well as human well-being targets linked to these 47 species and habitats), direct and indirect threats, and contributing factors and opportunities. Before TOC was developed, a Situation Model was created to explicitly show the relationships among the main contributing factors that drive one or more of the pressures that, in turn, impact the ecosystem targets of concern. This Situation Model was then used to develop a TOC. The USAID Oceans TOC articulates a set of assumptions about how the strategies and interventions will lead to a set of intermediate results that set the stage for achieving a desired long-term change. Situation Model The Situation Model (ATTACHMENT J.6) illustrates the predominant threats contributing to overexploitation and degradation of coral reefs and marine species in the Association of Southeast Asian Nations (ASEAN) and Coral Triangle Initiative on Coral Reefs, Fisheries, and Food Security (CTI-CFF) regions. Marine and land-based pollution, destructive development, and climate change are identified as major threats; with unsustainable and IUU fishing noted as the highest priority threats. These threats negatively impact both habitats and species, resulting in reduced reproductive capacity, habitat degradation, conversion and destruction, change in ocean temperature and pH level, and reduced diversity and abundance. With the purpose of protection and amelioration of biodiversity focal interests, the implementation of USAID Oceans will focus specifically on combating the highest priority threats—IUU and unsustainable fishing. Within USAID Oceans’ targeted geography, overharvesting, IUU capture, and destructive fishing methods are the key direct threats to marine biodiversity. Indirect threats, such as poor governance, characterized by a lack of enforcement or consultative management, and compromised management due to a lack of information or resources, exacerbate the problem. Population growth and increased global demand for fisheries products continues to fuel growth in the fishing industry; however, the declining return on investment in fishing and lack of enforcement result in negative impacts, driving demand for cheap labor and human rights abuses. Theory of Change USAID Oceans identified strategic approaches to address the direct and indirect threats depicted in the Situation Model. The key results expected from application of these strategic approaches are described in the TOC below and logic model using Result Chains in Annex 2. A. If the CDTS is robust, meets stakeholders’ needs, and provides economic incentives to the fishing industry from the increased demand for and value of traceable fishery products, then it will be adopted by members of the private sector in partnership with government agencies throughout the region; B. If fisheries managers use catch documentation and other tools to inform fisheries management plans and regulatory regimes, then local and national fisheries governance will be strengthened; and C. If regional capacity and cooperation is built to support EAFM and CDT, then more institutions and countries in the region will endorse and sustain their use; D. Then, illegal, unreported, and unregulated fishing will be reduced, and marine ecosystems will be maintained. b) Approach & Implementation USAID Oceans focuses on addressing weak conservation of marine biodiversity by building stakeholder capacity in, and improving information for, fisheries management. Approaches and implementation include: 48 Build a CDTS as Part of a FIS Information to advise management, ensure a quality product, and document the social and environmental impacts of fisheries is weak across the Asia-Pacific Region. At the core of the USAID Oceans is the development of the electronic CDT system (CDTS). The CDTS serves as the primary intervention that supports the partnership’s complementary work stream components, including EAFM, Public-Private Partnership (PPP), and human welfare interventions. USAID Oceans has developed a strategic approach that moves away from an inflexible, end-to￾end single software solution (i.e., classical system development life cycle approach with all design completed prior to work start-up) to an adaptive "Quick Win" strategy. The “Quick Win” strategy targets the development and testing of a minimally viable product and uses real-time learning to adapt and improve system development throughout the project lifecycle. During years one and two, the USAID Oceans team designed the appropriate approach and architecture to build an electronic catch documentation and traceability (eCDT) system that reflects the specific needs and capacities of national, provincial, and local government and private sector partners at two demonstration sites. The design of the eCDT architecture included guidance for the capture and access of key data elements collected within critical tracking events along the seafood supply chain in order to provide traceability of each seafood product (catch unit), as associated with a unique identifier. During years two and three, the USAID Oceans team identified and partnered with 26 private sector organizations in two learning sites (which are called “first movers”) who agreed to test and demonstrate eCDT solutions that provides seafood traceability throughout the stages of the seafood supply chain while also informing fisheries managers how to make decisions relating to the adaptive management and governance of relevant fisheries within the supply chain under an EAFM framework. During years three and four, the USAID Oceans team is focusing on measuring the uptake, utility, and sustainability of these electronic systems for the benefit of both seafood traceability and fisheries management, as well as documenting and sharing experiences and lessons learned relating to the costs, benefits, needs, and opportunities associated with region-wide adoption and implementation of eCDT systems and supporting technologies. Strengthen Capacity of Stakeholders to Operationalize the Regional EAFM The adoption of an EAFM that defines principles for growth, control, and maintenance of sustainable fisheries serves as an enabling framework for improving the health, productivity, and resilience of the whole ecosystem. The up-take and sustainability of the USAID Oceans supported Catch Documentation and Traceability System/ASEAN Catch Documentation System (CDTS/ACDS) and other actions will be dependent on mobilizing and building the capacity of champions at regional, national, and local levels. USAID Oceans has provided trainings to strengthen the capacity of local stakeholders to operationalize the regional EAFM framework developed with the US CTI-CFF Support Program and support initiatives to combat IUU and unsustainable fishing in the region. As part of this effort, USAID Oceans supports stakeholders to develop performance benchmarks to measure changes in EAFM capacity over time. USAID Oceans has also worked closely with SEAFDEC and CTI-CFF through the USAID Oceans’ 49 Technical Working Group representatives to ensure ongoing learning, collaboration, Activity planning and implementation with a cohort of consistent and stable counterparts. These counterparts also bridge support at the regional, national and local levels enhancing cross-learning and partnership formation. The Activity promotes capacity building and cooperation through a number of approaches including formal and informal training and network building efforts. USAID Oceans provides technical support and builds the capacity of SEAFDEC to be recognized as the regional champion of CDTS/ACDS, promoting further replication and adoption of the systems across the region. For CTI-CFF, USAID Oceans also supports the CTI Regional Plan of Action Goal for EAFM, including the development of the Sulu-Sulawesi Seas EAFM plan. USAID Oceans also works with USAID, the U.S. Department of Interior (DOI) and the National Oceanic and Atmospheric Administration (NOAA) to coordinate USAID resources committed to SEAFDEC and CTI- CFF. The Activity also leverages the expertise and experience of existing USAID bilateral projects, such as USAID/Indonesia’s Sustainable Ecosystems Advanced (SEA), to harmonize EAFM efforts in the learning sites. Use Grants Strategically to Achieve Technical Objectives and Build Local Capacity USAID Oceans planned to use its grants under contract (GUC) to achieve the innovation and biodiversity-focused impacts of the project. All grant agreements are explicitly linked to USAID Oceans’ Result Areas, and in most cases, directly fulfill USAID Oceans’ indicators. Engage the Private Sector for Sustainability The private sector is a key focus of USAID Oceans. Led by the Senior PPP Advisor, USAID Oceans has designed and implemented the CDTS and management frameworks such that they are viable tools for the private sector and aligned with their long-term interests. Priorities for engagement with the private sector are in building the business case and financing mechanism for the CDTS. The USAID Oceans partnership and industry engagement strategy has six components: (1) develop a partnership and industry engagement strategy that integrates with USAID Oceans’ objectives and key partners’ priorities; (2) identify and map potential industry and NGO partners at the regional, national and site-level and in key value chains and end- markets; (3) develop quick win partnerships to support the design and demonstration of the CDTS in at least two fisheries and three to four value chains in Years 1 and 2; (4) develop a select number of high-impact partnerships to support the replication and expansion of the CDTS at the site, national and regional levels; (5) strengthen industry associations and NGOs capacity to support CDTS adoption and expansion, and build stronger industry collaboration and constituencies to expand and sustain the CDTS and champion sustainable fisheries management; and (6) build the capacity of SEAFDEC and national governments in partnership development and industry engagement, linking with their institutions’ plans and programs. USAID Oceans measures private sector engagement through two indicators: value or cash (in USD) of public and private sector investments in sustainable fisheries and coastal and marine ecosystem conservation; and Number of new USG-supported PPPs formed. Prioritize Vulnerable Populations through Targeted Gender and Social Inclusion Interventions Women are heavily engaged in fisheries, primarily in the areas of processing and sale of catch. They are often at a disadvantage compared to men in terms of access to credit, educational level, 50 opportunities for higher-paid jobs, and market information. As a key principle and cross-cutting requirement, USAID Oceans has designed and implemented interventions starting from gender analysis and labor studies to identify gaps to capacity building components to inform women through the CDTS and FIS systems, increase their capacity to engage in decision making at key points along fisheries value chains, and engage and empower them to use the FIS. USAID Oceans’ M&E Plan captures human welfare and gender equality considerations throughout the program cycle—from design, to implementation, to measurement of impact as presented in the Results Chain—in support of USAID’s Gender Policy and Female Empowerment and RDMA’s Smart Development Screen on Gender Equality. c) Geographic Scope and Targeted Partners/Stakeholders Defining Geographic and Biodiversity Priorities USAID Oceans has engaged ASEAN and Coral Triangle Initiative on Coral Reefs, Fisheries and Food Security (CTI-CFF) member countries and selected learning and expansion sites to maximize biodiversity conservation, regional impact, and sustainability (Refer to Table 1) 51 In compliance with the USAID Biodiversity Code, priority biodiversity areas in the region were targeted including learning sites in Indonesia and the Philippines which bound the Sulu-Sulawesi Seas. A results-chain for each learning site was developed as part of the Pause and Reflect workshop in February 2018. The results-chain (as presented in ATTACHMENTS J.7 - J.10) presents how each strategic approach being implemented to achieve results in each learning site. USAID Oceans also identified expansion sites beyond the Sulu-Sulawesi Seas using national biodiversity priorities and applying the Biodiversity Code requirements as key criteria for selection focusing on vulnerable habitats and species. The other selection criteria include: transboundary elements are involved such as trade; national government site and fisheries priorities are being met; industry preparedness to engage is present; overall political will is evident; sustainability is likely to be achieved; and the operating environment is sufficiently viable for long term engagement. Table 4: USAID Oceans Learning and Expansion Sites Learning Sites Implementation Year 2016 - 2020 Expansion Sites – 1 Implementation Year 2017 - 2020 Expansion Sites - 2 Implementation Year 2019 - 2020 Bitung, Indonesia (National Fisheries Management Area # 716) General Santos City, Philippines (Sarangani Bay Protected Seascape; w/surrounding Exclusive Economic Zone waters) Songkhla, Thailand Kelantan, Malaysia Nha Trang, Vietnam Cambo dia Myanm ar Lao PDR Brunei Darussalam Singapore Papua New Guinea Solomon Islands Timor-Leste Targeted Partners/Stakeholders USAID Oceans has been working with three categories (levels) of partner and stakeholder groups which are: 1) regional stakeholders, 2) national stakeholders, and 3) learning site stakeholders. These stakeholders and partners comprise of regional institutions, national and 52 local governments, private sectors and industry, NGOs, and universities. Each stakeholder/partner plays different roles as you can see from the following graphic. The list of stakeholders that the contractor shall consult with (at the minimum) is provided in ATTACHMENT J.11 Oceans List of Stakeholders. d) Intended Results USAID Oceans created indicators and targets based on the results along the TOC and they are presented in the M&E plan. Additionally, USAID Oceans also developed the 2020 Oceans Endgame (ATTACHMENT J.12 Oceans 2020 End Game); a tool to provide strategic direction and focus for planning and implementation that synthesizes the USAID Oceans Results Framework, the life of Activity targets, and strategic approaches. USAID Oceans 2020 Endgame Results ● 10 million hectares of marine ecosystems maintained through the development of four sustainable fisheries management plans incorporating EAFM, CDTS, gender, and human welfare considerations, with corresponding management constituency established and strengthened. ● Reduced illegal, unreported, and unregulated fisheries through the development of Catch Documentation and Traceability System (CDTS) and regional implementation roadmap that is: – technically robust, – financially sustainable, and – spurs intra- and inter-regional trade; and is 53 ● jointly developed with relevant private and government stakeholders and effectively used to: – reduce IUU fishing and seafood fraud, – improve fisheries management, – promote equity and fair labor practices, and thus collectively leads to improved biodiversity conservation and equitable development in the ASEAN and CTI-CFF region. ● Over 16,000 Critical Tracking Event (CTE) transactions processed by the CDTS in the seafood industry supply chain resulting in at least 20 metric tons of seafood units purchased. ● Over 50 seafood actors verifying the legality, sustainability and responsibility of 20 metric tons of seafood products via the CDT system by 2020. ● 2 innovations developed to enable effective fisheries transparency and sustainability bringing significant large-scale improvements to the Asia-Pacific region. ● ASEAN regional adoption of the CDTS/ACDS will be achieved. ● SEAFDEC will be recognized as the regional champion of the CDTS/ACDS with the capacity to exert robust leadership in promoting further replication and adoption of the systems across the region. ● At least 4 legal instruments including laws, policies, strategies or regulations to promote gender equality or non-discrimination against women and girls. ● $4 million in industry and government contributions to sustain the CDTS through at least 8 sustainable partnerships, and SEAFDEC industry and other non￾government organizations engagement on shared solutions to combat IUU fishing through the CDTS. EVALUATION QUESTIONS USAID/RDMA together with USAID/E3/Forestry and Biodiversity Office (FAB) and the USAID Oceans’ implementing partner organized a Pause and Reflect Workshop back in February 2018 to review learning questions along the results-chains and identify some key areas/questions for learning and performance evaluation purposes. Using the refined learning questions that resulted from the Pause and Reflect Workshop, an internal mid-term review was conducted by Tetra Tech consultants in March 2018 to review the key assumptions and identify priorities to support achievement of USAID Ocean’s endgame results and adaptive management. The findings and recommendations of the internal mid-term review (ATTACHMENT J.13 Oceans Internal Mid-Term Review) are being used as a basis for the EQs. USAID/RDMA has worked further with the implementing partner and USAID/E3/FAB to finalize the performance evaluation purpose and questions focusing on closing the knowledge gaps on 1) the effectiveness of key approaches to achieving expected results within the result-chains based TOC, especially in the area of capacity building, public- private partnership and regional coordination, 2) factors that enhanced or diminished the achievements of the expected results in the TOC and 3) how to enhance effectiveness of USAID marine biodiversity conservation 2 as well as help guide the design of future programming to maximize development outcomes in maritime security and biodiversity conservation. The EQs below are listed in the order of priority. All key and sub-questions must be answered. 54 1. To what extent have the USAID Oceans’ approaches achieved and/or advanced the USAID Oceans’ objectives on regional coordination, PPP and capacity building? Specifically, the following sub-questions must be answered. Regional coordination: a. To what extent has USAID Oceans’ work with SEAFDEC and CTI-CFF enhanced their ability to: i. counter IUU fishing and strengthen fisheries management across ASEAN member countries; and ii. effectively support ASEAN member countries’ adoption and use of eCDT systems to counter IUU Fishing and strengthen fisheries management? Public-private partnership and engagement: a. To what extent does USAID Oceans effectively engage national governments and private sector partners within learning sites and a selected expansion site (Kelantan, Malaysia) in formalizing partnerships for a fully sustainable eCDT system? b. Which incentives for adopting eCDT systems are the most appealing to private sector partners to encourage adoption of eCDT systems? Capacity building and institutionalization: a. How promising have USAID Oceans’ capacity building efforts been in helping key targeted beneficiaries to achieve the following results? i. Strengthening SEAFDEC capacity as the regional champion for moving ASEAN member countries to adoption and use of an eCDT system; and ii. Building local and national government agencies to develop and implement an eCDT system. b. To what extent have human welfare, particularly gender equity and labor considerations, been addressed within the promotion and adoption of the eCDT systems being supported by USAID Oceans? 2. What factors (both internal and external) have enhanced, diminished, or have no effect on achievement of USAID Oceans expected results in reducing IUU fishing and strengthen fisheries management? The responses must include analyses of the following: a. eCDT adoption and use among private sector “early mover” partners within seafood supply chains at both USAID Oceans learning sites; b. EAFM among national, provincial, and local fisheries management authorities at both USAID learning sites; c. Regional coordination among ASEAN and CTI-CFF member countries supported by SEAFDEC and the CTI-CFF Regional Secretariat; d. PPP among private sector “early mover” partners within seafood supply chains at both USAID learning sites; e. Capacity building and institutionalization of eCDT approaches and guidance within SEAFDEC and CTI-CFF Regional Secretariat; and f. Fisheries supply chain-related human welfare and gender equity awareness and actions being taken by stakeholders within the seafood supply chains at both USAID Oceans learning sites. 55 3. What evidence(s) exists that the participating ASEAN countries, especially in the learning sites and a selected expansion site (Kelantan, Malaysia) would potentially sustain the adoption and implementation of the eCDT systems to counter IUU fishing and strengthen fisheries management as a result of USAID Oceans' support? a. What specific actions by which stakeholders (i.e., USAID/RDMA, USAID Oceans, SEAFDEC, CTI-CFF and beyond) will be required to ensure future sustainability after the USAID Oceans ceased? C3.3.3 EVALUATION DESIGN AND SUGGESTED METHODOLOGIES a) Evaluation Design, Data Collection and Analysis Methods This study is a performance evaluation. The evaluation must utilize mixed methods. The methodology should be collaborative and participatory, including implementing partners, key and vulnerable populations benefitting from the Activity interventions/services, and key stakeholders (including USAID staff) as much as possible in evaluation planning and implementation. USAID suggests Key Informant Interview (KII), Focus Group Discussion (FGD) and surveys among selected key stakeholders, e.g., government fisheries authorities, relevant regional organizations, private sector actors from fish production, fish processing, and technology companies (and industry associations), research institutes or universities, aid agencies and other donors, and non￾profit conservation organizations, or others who have the knowledge to offer about the changes (or lack thereof) referred to in the research question as primary data collection methods for this evaluation. It is anticipated that the field data collection will be conducted in English in most places, but there may also be a need for local language (Bahasa Indonesia) and translation expertise to ensure data accuracy. USAID also requests the contractor to conduct a thorough desk study and use its learning and findings to propose the areas of inquiry and methods that are best suited to answering the EQs and achieving the stated purposes. USAID will provide a list of stakeholders as one of the key documents. USAID expects that the contractor will use additional methods to achieve the purpose described above. USAID recommends doing one rigorous process tracing to understand USAID’s contributions to at least one major success that led to the achievement of the Activity. In addition to the above suggestions and requests, the contractor must provide a detailed explanation of the proposed methodology for carrying out the work as well as providing summary in the matrix below in the design phase. However, the contractor shall not use this matrix in place of a detailed description of the design and methodology. Research Questions Suggested Data Sources Suggested Data Collection Methods (including sampling methodology, where applicable) Suggested Data Analysis Methods [List key and sub￾questions in separate rows.] [List any possible primary and/or secondary data sources relevant to each research question.] [List relevant method(s) to each research question. If require any methods/ tools, please specify.] 56 The contractor is expected to spend most of their fieldwork in Bangkok (Thailand), Jakarta and Manado/Bitung (Indonesia), Manila and General Santos (Philippines), Kelantan (Malaysia) and a day in Kuala Lumpur to meet with Malaysia’s Department of Fisheries. The contractor may reach stakeholders in Songkla, Thailand and Nha Trang Vietnam using phone and/or internet platform as means of communication. The team must propose the sample size for each type of respondents/informants in the design. b) Limitations or constraints of the design and methodologies Evaluation limitations include a broad scope of inquiry, sampling bias, and response bias. To minimize the effect of these limitations, the ET is required to triangulate the findings from multiple data methods and sources. There might be some constraints to achieve high response rate from target respondents for each individual method; so, the contractor must propose the sample size for each type of respondents/informants in the design. In Indonesia, the contractor may need to conduct KII or FGD using translators; there is possible loss of meaning and data richness. 57 ANNEX 2: USAID OCEANS LEARNING SITES RESULTS CHAINS BITUNG, INDONESIA – UPDATED RESULTS CHAIN GENERAL SANTOS, PHILIPPINES – UPDATED RESULTS CHAIN 58 ANNEX 3: EVALUATION SCHEDULE Period Activity Outputs Feb 1-8, 2019 Fieldwork data collection preparation, USAID Oceans Evaluation Workshop Preparations, Data collection arrangements Feb 11, 2019 Inception presentation (Tetra Tech) and In-Brief (USAID/RDMA), including discussion of evaluation instruments, methodology, and fieldwork plan Presentation Feb 12-15, 2019 Thailand Fieldwork: Regional Interviews, SEAFDEC, CTI, ASEAN Representative Interviews in BKK; Interview Notes Feb 18-23, 2019 Indonesia Fieldwork Interview Notes Feb 26 - Feb 27, 2019 Malaysia Fieldwork Interview Notes Feb 25 - Mar 1, 2019 Philippines Fieldwork Interview Notes Mar 4, 2019 Out-brief with Implementing Partners and stakeholders Presentation of preliminary findings Mar 5, 2019 Out-brief with USAID Oceans Management and USAID/RDMA Out-brief Presentation Mar 6-29, 2019 Review Notes and Prepare Preliminary Draft Draft Evaluation Report 59 ANNEX 4: KEY USAID OCEANS’S PARTNERS BY TYPE, DIFFERENT ENGAGEMENT MECHANISMS, AND THE PARTNER LEVEL Type Partner name Mechanism Regional￾level Country￾level Seafood companies - Large Thai Union MOU X Anova Informal partnership through MDPI grant Indonesia Mars Petcare Informal through USAID Thailand CTIP Thailand Seafood companies – Small to medium PT Nutrindo PT BOGI CDT implementation through MDPI (grantee) Indonesia PT SMS-BMB Informal partnership Indonesia Marchael Sea Ventures Rell & Renn Fishing Corp. Jebo Fishing Tuna Explorers Inc. Dex Sea Trading Rd Fishing Corporation General Tuna Canning Corporation Rell & Renn Seafood Sphere Inc. Philcinmic Industrial Corp. Mommy Gina Tuna Resources Celebes Canning Corp. Sta. Cruz Seafood Inc Citra Mina Canning Corporation CDT implementation through SFFAII (grantee) Philippines Technology – large company Inmarsat and PT Sisfo (Inmarsat’s local partner in Indonesia) Partnership Concept Note X Technology – start-ups FAME Partnership Concept Note Philippines Altermyth Subcontract Indonesia ThisFish Informal partnership X NGOs Technical Advisory Group (TAG) ● Partners: Future of Fish, GFTC, Marine Stewardship Council, Fishwise, Monterey Bay Aquarium Seafood Watch, WWF, International Sustainable Seafood Foundation, Catapult, MDPI and Thai Union Agreed Terms of Reference X Future of Fish Subcontract X Global Food Traceability Center (GFTC) Subcontract International Pole and Line Foundation (IPNLF) Partnership Concept Note X Seafood Watch Partnership Concept Note X MDPI Grantee Indonesia Marine Stewardship Council (MSC) Informal partnership but expecting a partnership concept X 60 note in Q2Y4 Indonesia Coastal Tuna Sustainability Alliance (ICTSA): MDPI, IPNLF and AP2HI Subcontract Indonesia Industry associations SFFAII Grantee Philippines AP2HI Currently informal partnership but expecting a partnership concept note in Q2Y4 Indonesia GDST Informal partnership but expecting a partnership concept note in Q2Y4 X Regional Platforms SAFET Informal partnership X Responsible Business Forum Informal partnership X Foundations Walton Family Foundation Indirect engagement through NGO partners X David and Lucille Packard Foundation Indirect engagement through NGO partners X 61 ANNEX 5: CONSENT SCRIPTS AND INTERVIEW GUIDES The following are semi-structured interview guides and consent scripts utilized for the USAID Oceans PE. Consent Script: Key Informant Interviews Hello, my name is _______ and I am with Social Impact, a US-based research organization working under contract with the United States Agency for International Development (USAID). We are conducting an evaluation of USAID’s Oceans and Fisheries Partnership implemented by Tetra Tech ARD in collaboration with the Southeast Asian Fisheries Development Center (SEAFDEC) and other key stakeholders to strengthen regional cooperation to combat illegal, unreported, and unregulated (IUU) fishing, promote sustainable fisheries, and conserve marine biodiversity in the Asia-Pacific region. Thank you for taking the time to meet with us. The interview today is expected to take 45 – 60 minutes, though you can stop participating or leave the room at any point without consequences. If during this interview, we ask any questions that you do not wish to answer, you don’t have to respond. We will ask you questions about [tailor for each KII respondent category]. Your involvement in this discussion is completely voluntary and you are under no obligation to participate. You can stop participating or leave the room at any point without consequences. If during this group discussion, we ask any questions that you do not wish to answer, you don’t have to respond. We will be taking notes, but all of your responses will be kept confidential and any information you provide that might identify you will be kept confidential to the fullest extent under local law and U.S. Government policy. For reports we write about the research, your answers will be combined with those of other people and presented in a summary format. There is no payment or direct benefit to you for participating in this interview. We do not anticipate any major risks to you for participating other than losing time you could spend on other things. If you have any concerns, you may contact the Social Impact Institutional Review Board at irb@socialimpact.com or +1 703 465 1884 with questions about the study or results. Additionally, you may contact [insert name and contact information for local contact in each country]. I will leave a copy of this form with you. Do you have any questions? Do you understand that your participation is voluntary? Yes ______ No ________ Do you understand that you can stop participating at any time? Yes _________ No __________ CONSENT STATEMENT: I understand and agree to participate in this study. Respondent provided consent (written if possible): Yes_____ No_____ 62 REGIONAL INTERVIEW GUIDE: SEAFDEC/CTI-CFF SECRETARIAT Q1: 1. What financial and technical support have you received from USAID Oceans? 2. What are observable changes in your organization’s capacity to address IUU fishing and fisheries management issues that you believe are a result of your work with USAID Oceans (please provide examples of evidence)? (organization structure, services, resources, staff skills)? E.g. changes in capacity to undertake CDT gap analysis? How do they compare to before the Project? 3. What activities have you pursued to counter IUU fishing and to strengthen fisheries management with support from USAID Oceans? What are observable results? How have these been different from what you had implemented before? 4. Has there been any follow-up assessment of results from the training and technical assistance? a) If so, what were the results? Can you share any documentation? 5. Can you provide any examples of specific results in ASEAN countries that can be attributed to these activities? 6. Are there other organizations in the region actively involved in these issues? What linkages exist, if any, with the USAID Oceans program? 7. Are you aware of any USAID Oceans projects that focused on human welfare, e.g., labor and gender concerns? a) Do you believe that USAID Oceans has changed the way your organization, or the sector more generally (if so, please provide examples)? i. Considers human welfare, e.g., labor and gender aspects in programming or policy decisions? ii. Generates benefits for women or vulnerable populations? iii. Considers labor/human welfare aspects in programming or policy decisions? 8. Integration of gender and labor considerations into the promotion and adoption of the eCDT system? Has your organization had more involvement in private sector partnerships/engagement as a result of USAID Oceans? If so, how? a) If so, how have these contributed, or not, to achievement of your desired results? Q2: 1. What factors do you feel have been most important in affecting, positively or negatively, IUU fishing and fisheries management at a regional level? Please provide any evidence available. 2. What factors do you feel have been most important in affecting, positively or negatively, IUU fishing and fisheries management at the national level in the country you are most familiar with? Please provide any evidence available. 3. Are there any key gaps at the regional level that still need to be addressed by the Project or others? Please elaborate. 4. Is there anything specifically needed to improve the performance of the USAID Oceans Project in its final year? This could include things specifically related to support to your organization or more broadly in terms of achieving the larger objectives. 5. How important do you think each of the following factors are in reducing IUU fishing and strengthening fisheries management? Please provide a rating of 1-5 (with 1 being very little effect and 5 being very significant effect) and explanation of your rating: (for handout to be used by respondents, see below) a) A demonstration of eCDT adoption and use among private sector “early mover” partners within seafood supply chains at both USAID Oceans learning sites; b) EAFM among national, provincial, and local fisheries management authorities at both USAID learning sites; 63 c) Regional coordination among ASEAN and CTI-CFF member countries supported by SEAFDEC and the CTI-CFF Regional Secretariat; d) PPP among private sector “early mover” partners within seafood supply chains at both USAID learning sites; e) Capacity building and institutionalization of eCDT approaches and guidance within SEAFDEC and CTI-CFF Regional Secretariat; and f) Fisheries supply chain-related human welfare and gender equity awareness and actions being taken by stakeholders within the seafood supply chains at both USAID Oceans learning sites. 6. What other factors do you think have important influences on IUU fishing and fishery management? Q3: 1. How likely is it that these activities and results can be sustained by your organization after the Project? What are the factors that may affect potential for sustainability? 2. Who will sustain the progress on eCDT after the USAID Oceans Project ends in May 2020? 3. What could be done by the Project to enhance sustainability of results? REGIONAL INTERVIEW GUIDE: ASEAN COUNTRIES’ FISHERIES AUTHORITIES Q1: 1. What involvement have you had with the USAID Oceans Project? 2. How has the Project helped to combat illegal, unreported and unregulated fishing in your country? Can you provide some details/evidence? How does this compare to before the Project? 3. How effective has the support been from SEAFDEC or CTI-CFF in efforts to combat IUU fishing and to introducing eCDT? Could anything have been done differently? 4. Have any public and private sector partnerships (PPPs) been used to assist promotion of eCDT in your country? In what way? How could they be improved, if at all? 5. Are you aware of any social welfare for labor measures that have been included in any of the seafood supply systems as a result of USAID Oceans? If so, please elaborate and provide any documentation available. 6. Are you aware of any gender equity measures that have been included in any of the seafood supply systems as a result of USAID Oceans? If so, please elaborate and provide any documentation available. Q2: 1. What factors do you feel have been most important in affecting, positively or negatively, IUU fishing and fisheries management at a regional level? Please provide any evidence available. 2. What factors do you feel have been most important in affecting, positively or negatively, IUU fishing and fisheries management at the national level in the country you are most familiar with? Please provide any evidence available. 3. Are there any key gaps at the regional level that still need to be addressed by the Project or others? Please elaborate. 4. Is there anything specifically needed to improve the performance of the USAID Oceans Project in its final year? This could include things specifically related to support to your organization or more broadly in terms of achieving the larger objectives. 5. How important do you think each of the following factors are in reducing IUU fishing and strengthening fisheries management? Please provide a rating of 1-5 (with 1 being very little effect and 5 being very significant effect) and explanation of your rating: (for handout to be used by respondents, see below) a. A demonstration of eCDT adoption and use among private sector “early mover” partners within seafood supply chains at both USAID Oceans learning sites; 64 b. EAFM among national, provincial, and local fisheries management authorities at both USAID learning sites; c. Regional coordination among ASEAN and CTI-CFF member countries supported by SEAFDEC and the CTI-CFF Regional Secretariat; d. PPP among private sector “early mover” partners within seafood supply chains at both USAID learning sites; e. Capacity building and institutionalization of eCDT approaches and guidance within SEAFDEC and CTI-CFF Regional Secretariat; and f. Fisheries supply chain-related human welfare and gender equity awareness and actions being taken by stakeholders within the seafood supply chains at both USAID Oceans learning sites. 6. What other factors do you think have important influences on IUU fishing and fishery management? Q3: 1. How likely is it that these activities and results can be sustained by your organization after the completion of USAID Oceans? What are the factors that may affect potential for sustainability? 2. Who will sustain the progress on eCDT after the USAID Oceans Project ends in May 2020? 3. What could be done by the Project to enhance sustainability of results now? LEARNING SITES INTERVIEW GUIDE: NATIONAL AND LOCAL PARTNERS Q1: 1. What involvement have you had with the USAID Oceans Project? 2. Have any specific changes in institutions, or resources or staff skills occurred as a result of the Project (and what evidence do you have of these changes)? How does this compare to before the Project? 3. How useful and effective has the support been from USAID Oceans and the Project team? 4. How effective do you feel efforts have been to establish eCDT at the learning site? What is your impression of progress so far? What should happen to improve progress? 5. Are you aware of any USAID Oceans activities that focused on human welfare, especially labor and gender concerns? a. Do you believe that USAID Oceans has changed the way your organization, or the sector more generally (if so, please provide examples)? i. Considers human welfare, e.g., labor and gender aspects in programming or policy decisions? ii. generates benefits for women or vulnerable populations? 6. Has your organization had more involvement in private sector partnerships as a result of USAID Oceans? If so, how? a. If so, how have these contributed, or not, to achievement of your desired results? Q2: 1. What factors do you feel have been most important in affecting, positively or negatively, IUU fishing and fisheries management at a regional level? Please provide any evidence available. 2. What factors do you feel have been most important in affecting, positively or negatively, IUU fishing and fisheries management at the national level in the country you are most familiar with? Please provide any evidence available. 3. Are there any key gaps at the regional level that still need to be addressed by the Project or others? Please elaborate. 65 4. Is there anything specifically needed to improve the performance of the USAID Oceans Project in its final year? This could include things specifically related to support to your organization or more broadly in terms of achieving the larger objectives. 5. How important do you think each of the following factors are in reducing IUU fishing and strengthening fisheries management? Please provide a rating of 1-5 (with 1 being very little effect and 5 being very significant effect) and explanation of your rating: (for handout to be used by respondents, see below) a. A demonstration of eCDT adoption and use among private sector “early mover” partners within seafood supply chains at both USAID Oceans learning sites; b. EAFM among national, provincial, and local fisheries management authorities at both USAID learning sites; c. Regional coordination among ASEAN and CTI-CFF member countries supported by SEAFDEC and the CTI-CFF Regional Secretariat; d. PPP among private sector “early mover” partners within seafood supply chains at both USAID learning sites; e. Capacity building and institutionalization of eCDT approaches and guidance within SEAFDEC and CTI-CFF Regional Secretariat; and f. Fisheries supply chain-related human welfare and gender equity awareness and actions being taken by stakeholders within the seafood supply chains at both USAID Oceans learning sites. 6. What other factors do you think have important influences on IUU fishing and fishery management? Q3: 1. How likely is it that these activities and results can be sustained by your organization after the completion of USAID Oceans? What are the factors that may affect potential for sustainability? 2. Who will sustain the progress on eCDT after the USAID Oceans Project ends in 2020? 3. What could be done by the Project to ensure/enhance sustainability of results? LEARNING SITES INTERVIEW GUIDE: PRODUCERS AND PROCESSORS Q1: 1. What has been the extent of your involvement in the USAID Oceans Project? 2. Why did you become/not become involved in the Project? 3. Are you part of a formal PPP agreement? If so, how satisfied are you with this partnership? Satisfied, Partially Satisfied, Not Satisfied? Please explain. a. What have been the main achievements through that PPP? b. In what areas was the PPP less successful? How could it be strengthened? c. Which of the activities do you expect will continue following the conclusion of the PPP? 4. How satisfied are you with the level of support received from the Project? Are there any gaps in support or issues that need to be addressed? 5. What factors affect private sector acceptance and use of eCDT technology? On a scale of 1 – 4, please rate the importance of the following factors that have or may in the future affect your decision about eCDT. Are there any other important factors? For a handout to be used with respondents, see below. 66 1 = Very Important; 2 = Moderately Important; 3 = Minor Importance; 4 = Not Important, N/A=Not sure/don’t know Adequate knowledge about the benefits and costs of eCDT Adequate evidence and examples of eCDT technology effectiveness Industry or community support and advice on eCDT Technical assistance and training on eCDT installation and operation Costs of purchasing and maintaining eCDT technology Availability of financial assistance to reduce costs of eCDT Availability of financing schemes to purchase eCDT technology Better catch data to assist ongoing fishing/processing operations Government directives or regulations requiring use of eCDT Seafood buyers market requirement for eCDT reporting of catch Extent of privacy and confidentiality of the catch data Home telephone and at sea safety benefits of eCDT satellite communications Vessel monitoring and management information 6. To what extent have social welfare aspects – gender equity, labor protection been integrated into the seafood supply chains as a result of the Project? Have you or your family benefitted from the social and human welfare aspects of the Project? Can you give an example? Q2: 1. What factors do you feel have been most important in affecting, positively or negatively, IUU fishing and fisheries management at a regional level? Please provide any evidence available. 2. What factors do you feel have been most important in affecting, positively or negatively, IUU fishing and fisheries management at the national level in the country you are most familiar with? Please provide any evidence available. 3. Are there any key gaps at the regional level that still need to be addressed by the Project or others? Please elaborate. 4. Is there anything specifically needed to improve the performance of the USAID Oceans Project in its final year? This could include things specifically related to support to your organization or more broadly in terms of achieving the larger objectives. 5. How important do you think each of the following factors are in reducing IUU fishing and strengthening fisheries management? Please provide a rating of 1-5 (with 1 being very little effect and 5 being very significant effect) and explanation of your rating: (for handout to be used by respondents, see below) a. A demonstration of eCDT adoption and use among private sector “early mover” partners within seafood supply chains at both USAID Oceans learning sites; b. EAFM among national, provincial, and local fisheries management authorities at both USAID learning sites; 67 c. Regional coordination among ASEAN and CTI-CFF member countries supported by SEAFDEC and the CTI-CFF Regional Secretariat; d. PPP among private sector “early mover” partners within seafood supply chains at both USAID learning sites; e. Capacity building and institutionalization of eCDT approaches and guidance within SEAFDEC and CTI-CFF Regional Secretariat; and f. Fisheries supply chain-related human welfare and gender equity awareness and actions being taken by stakeholders within the seafood supply chains at both USAID Oceans learning sites. 6. What other factors do you think have important influences on IUU fishing and fishery management? Q3: 1. How likely is it that you will use the eCDT system after the Project closes in May 2020? What means/measures/instruments have been put in place to support you to continue to use the system? 2. What could be done by the Project to sustain progress on eCDT? 68 HANDOUT 1 How important do you think each of the following factors are in reducing IUU fishing and strengthening fisheries management? Please provide a rating of 1-5 (with 1 being very little effect and 5 being very significant effect) and explanation of your rating. N/A=Not sure/don’t know 1 2 3 4 5 NA Notes A demonstration of eCDT adoption and use among private sector “early mover” partners within seafood supply chains at both USAID Oceans learning sites EAFM among national, provincial, and local fisheries management authorities at both USAID learning sites Regional coordination among ASEAN and CTI-CFF member countries supported by SEAFDEC and the CTI￾CFF Regional Secretariat PPP among private sector “early mover” partners within seafood supply chains at both USAID learning sites Capacity building and institutionalization of eCDT approaches and guidance within SEAFDEC and CTI-CFF Regional Secretariat Fisheries supply chain-related human welfare and gender equity awareness and actions being taken by stakeholders within the seafood supply chains at both USAID Oceans learning sites 69 HANDOUT 2 What factors affect private sector acceptance and use of eCDT technology? On a scale of 1 – 4, please rate the importance of the following factors that have or may in the future affect your decision about eCDT. Are there any other important factors? 1 = Very Important; 2 = Moderately Important; 3 = Minor Importance; 4 = Not Important, N/A=Not sure/don’t know 1 2 3 4 N/A Notes Adequate knowledge about the benefits and costs of eCDT Adequate evidence and examples of eCDT technology effectiveness Industry or community support and advice on eCDT Technical assistance and training on eCDT installation and operation Costs of purchasing and maintaining eCDT technology Availability of financial assistance to reduce costs of eCDT Availability of financing schemes to purchase eCDT technology Better catch data to assist ongoing fishing/processing operations Government directives or regulations requiring use of eCDT Seafood buyers market requirement for eCDT reporting of catch Extent of privacy and confidentiality of the catch data Home telephone and at sea safety benefits of eCDT satellite communications Vessel monitoring and management information 70 ANNEX 6: LIST OF EVALUATION RESPONDENTS BY CATEGORY Respondent Type Total number of Respondents USAID Washington, USAID RDMA, and staff from bilateral Missions including USAID Indonesia and USAID Philippines 11 US Department of Interior 1 USAID Oceans implementing partner staff 10 USAID Sustainable Ecosystems Advanced (USAID SEA) Project 2 SEAFDEC staff 7 CTI-CFF 3 Malaysia Department of Fisheries (DOF) 3 Thailand Department of Fisheries (DOF) 8 Bureau of Fisheries and Aquatic Resources, Philippines, including NFDRI, 12, EAFM, National, and National Gender 6 MMAF Indonesia 8 Cambodia MAFFA 1 LGU Fisheries, Philippines 2 LGU Alabel, Philippines 2 LGU Maasim, Philippines 1 Sta Cruz 2 Thai Union 1 AP2HI 4 MDPI 1 BMB 1 Nutrindo 1 Sig 4 BOGI 2 Altermyth 1 FAME, including Director and local 2 Celebes 1 SFAII 2 Philmic 1 71 Respondent Type Total number of Respondents Fishwise 1 ATH 1 Bitung Port 3 Bitung Fisheries 5 Bitung Fishermen 2 Manado Port 4 Manado Captain 1 Manado Fishermen 2 General Santos Fishermen 2 Kawas Fishermen 4 Maasim Fishermen 2 Total 115 72 ANNEX 7: LIST OF DOCUMENTS REVIEWED The below documents were reviewed during the desk review for this PE. Documents Reviewed Burges Herbert, Sarah. “Overview for new COR and Activity Managers.” Presentation from USAID/RDMA. April 2017. Courtney Catherine, Supol Singhapoom, and John Parks. “USAID Oceans Mid-Term Review: Findings and Recommendations.” Lecture, from USAID/RDMA. May 22, 2018. CTI-CFF Regional Secretariat. “Report of the Learning Exchange for Coral Triangle Member Countries to The USAID Oceans and Fisheries Partnership’s Learning Site (Bitung, Indonesia) on the Application of Catch Documentation and Traceability (CDT) System for Fisheries and Seafood Products.” CTI-CFF. 2018. DeRigger, Kim j and Santi Nindang, Southeast Asia’s Fisheries Near Collapse from Overfishing, The Asia Foundation, March 28, 2018. Dow, Allan. "ASEAN Member States and UN FAO Agree Five-year Plan to Strengthen Cooperation in Agriculture and Forestry and Combat Illegal, Unreported and Unregulated (IUU) Fishing." FAO, December 10, 2018. http://www.fao.org/asiapacific/news/detail-events/en/c/1157013/. FAO/GEF/SEAFDEC/REBYC-II CTI Project; Training Course on Essential Ecosystem Approach to Fisheries Management (EAFM) in Myanmar 14 to 18 December 2015 IUU WATCH. "As EU Lifts Warning, EJF Cautions That Thailand Must Continue Fighting Illegal Fishing and Abuse at Sea." IUU Watch, January 9, 2019. http://www.iuuwatch.eu/2019/01/eu-lifts￾warning-ejf-cautions-thailand-must-continue-fighting-illegal-fishing-abuse-sea/. Marine Change, Thai Union CDT and Crew Communications Pilot Assessment Report, March 2018 Marud, Farid, From Bait to Plate: Designing Comprehensive and Integrated Technology Solutions, Feb. 16, 201. Morgan, Gary, Derek Staples, and Simon Funge-Smith. "Fishing Capacity Management and IUU Fishing in Asia." Asia-Pacific Fishery Commission, 2007. Regional Plan of Action to Promote Responsible Fishing Practices including Combating IUU Fishing in the Region (RPOA-IUU), “Cooperation in the Southeast Asia.” February 2014. Regional Workshop on Piloting E-EAFM LEAD Materials on 27-29 June 2016 in Rayong Province, Thailand Schuttenberg, HZ. “USAID Oceans Pause and Reflect Workshop Summary Report.” USAID/RDMA. 2018. Schuttenberg, HZ Pause and Reflect Workshop for the Oceans and Fisheries Partnership, 5-7 February 2018 SEAFDEC, Onsite Training for Pilot Testing of the eACDS in Brunei Darussalam and the meeting of ACDS Committee, 19-22 June 2017 SEAFDEC, Project Document Achievements for the Year 2016 and Proposed Activity for Year 2017, Nov. 2016 SEAFDEC, Report of the Fortieth Meeting of the Program Committee, November 2017. SEAFDEC, Report of the Forty-first Meeting of the Program Committee, 5-7 November 2018 SEAFDEC/Sweden, Policy Brief: Applying Human Rights-based and Gender Equality Approaches to Small- 73 Documents Reviewed scale Fisheries in Southeast Asia, 2018 Section C “Performance Work Statement C.1." January 2016. Somboon Siriraksophon, Report on the 2nd Technical Consultation Visit to Brunei Darussalam on Development of the electronic ASEAN Catch Documentation Scheme, 29, 2017 Somboon Siriraksophon, Report on the Baseline Survey towards Development of the electronic ASEAN Catch Documentation Scheme and Pilot-testing in Brunei Darussalam, 16-18 August 2016 Table X, “Target and Achievement in FY19.” The Nation. "EU Rap a Result of 'flawed Policies'." The Nation. May 17, 2018. http://www.nationmultimedia.com/detail/national/30345668. The Oceans and Fisheries Partnership. Data Requirements for Catch Documentation and Traceability in Southeast Asia: Critical Tracking Event and Key Data Element Framework and Glossary. Report. 2017 The Oceans and Fisheries Partnership. Fisheries Catch Documentation and Traceability in Southeast Asia: Technical Specifications. Report. 2017 USAID Oceans and Fisheries Partnership 2nd Annual Technical Working Group Planning Meeting Report, 2017 USAID Oceans and Fisheries Partnership, Inception Workshop Report, 2015 USAID Oceans and Fisheries Partnership, Mid-Term Partnership Review, May 2018 USAID Oceans and Fisheries Partnership, Monitoring and Evaluation Plan, November 2016. USAID Oceans and Fisheries Partnership, Partnership Appraisal Report, January, 2017 USAID Oceans and Fisheries Partnership, Partnership Prioritization Workshop Report, 2017 USAID Oceans Contact Database, Key Stakeholders. January 2019 USAID Oceans Current Projected Partnerships. December 2018 USAID Oceans, Activity Report, Learning Exchange for Coral Triangle Member Countries, Manado-Bitung, Indonesia, 25-28 June 2018 USAID Oceans. “Annex J.10, General Santos City Results Chain.” USAID/RDMA. 2018. USAID Oceans. “Annex J.11, 3rd Technical Working Group Planning Workshop, Participants List.” USAID/RDMA. 2018. USAID Oceans. “Annex J.12, USAID Oceans 2020 Endgame.” USAID/RDMA. 2017. USAID Oceans. “Annex J.13, Internal Midterm Review: Key Findings and Recommendations.” USAID/RDMA. 2018. USAID Oceans. “Annex J.6, Oceans Situation Model.” USAID/RDMA. 2016. USAID Oceans. “Annex J.7, Integrated Theory of Change.” USAID/RDMA. 2016. USAID Oceans. “Annex J.8, USAID Oceans Pause & Reflect.” Training to SEAFDEC Staff, from USAID/RDMA. June 29, 2016. USAID Oceans. “Annex J.9, Bitung Results Chain.” USAID/RDMA. 2018. USAID Oceans. “Baseline Survey and Communications Field Trip Report.” USAID/RDMA. 2018. USAID Oceans. “Event Report, Catch Documentation and Traceability Learning Exchanges and ASEAN Tuna 74 Documents Reviewed Ecolabeling Consultative Workshop.” USAID/RDMA. 2018. USAID Oceans. “FY15 Annual Progress Report.” USAID/RDMA. 2015. USAID Oceans. “FY15 Annual Work Plan.” USAID/RDMA. 2015. USAID Oceans. “FY16 Annual Progress Report.” USAID/RDMA. 2016. USAID Oceans. “FY16 Q1 Progress Report.” USAID/RDMA. 2016. USAID Oceans. “FY16 Q2 Progress Report.” USAID/RDMA. 2016. USAID Oceans. “FY16 Q3 Progress Report.” USAID/RDMA. 2016. USAID Oceans. “FY16 Q4 Progress Report.” USAID/RDMA. 2016. USAID Oceans. “FY17 Annual Progress Report.” USAID/RDMA. 2017. USAID Oceans. “FY17 Q1 Progress Report.” USAID/RDMA. 2017. USAID Oceans. “FY17 Q2 Progress Report.” USAID/RDMA. 2017. USAID Oceans. “FY17 Q3 Progress Report.” USAID/RDMA. 2017. USAID Oceans. “FY18 Annual Progress Report.” USAID/RDMA. 2018. USAID Oceans. “FY18 Q1 Progress Report.” USAID/RDMA. 2018. USAID Oceans. “FY18 Q2 Progress Report.” USAID/RDMA. 2018. USAID Oceans. “FY18 Q3 Progress Report.” USAID/RDMA. 2018. USAID Oceans. “Mid-Term Partnership Review.” USAID/RDMA. 2018. USAID Oceans. “Monitoring and Evaluation (M&IE) Plan.” USAID/RDMA. 2016. USAID Oceans. “Monthly Summary Report, November.” USAID/RDMA. 2018. USAID Oceans. “Revised Year 4 – Annual Work Plan.” USAID/RDMA. 2019. USAID Oceans. “Year 2 – Annual Work Plan.” USAID/RDMA. 2017. USAID Oceans. “Year 3 Annual Work Plan.” USAID/RDMA. 2017. USAID Oceans. Mid-Term Partnership Review. March 2018. USAID, E3 Bureau Capacity Development Assessment: From Capacity Development to Sustainable Development, June 2017. USAID, Private-Sector Engagement Policy, Executive Summary USAID, The Oceans and Fisheries Partnership Activity, “First Annual Workplan”, May 2015 – September 2016. USAID, The Oceans and Fisheries Partnership, "Annex 3 Gender Strategy.” USAID, The Oceans and Fisheries Partnership, 3rd Annual Regional Technical Working Group Planning Workshop, Meeting Report, July 2018. USAID, The Oceans and Fisheries Partnership, Annual Progress Report 2015 USAID, The Oceans and Fisheries Partnership, Communications Strategy and Implementation Plan (2018- May 2020), Sept. 2018. USAID, The Oceans and Fisheries Partnership, Gender Analysis of the Fisheries Sector, General Santos 75 Documents Reviewed City, Philippines, 2018 USAID, The Oceans and Fisheries Partnership, Gender Research in Fisheries and Aquaculture: A Training Handbook, Oct. 2018. USAID, The Oceans and Fisheries Partnership, Malaysia Capacity Gap Analysis and Partnership Appraisal, November 2018. USAID, The Oceans and Fisheries Partnership, Mid-term Review: Key Findings and Recommendations, July 2018. USAID, The Oceans and Fisheries Partnership, Technology Solutions. USAID, The Oceans and Fisheries Partnership, The Value Chain Assessment Report for Bitung, Indonesia, 2017 USAID, The Oceans and Fisheries Partnership, Vietnam Capacity Gap Analysis and Partnership Appraisal, October 2018. Worawit Wanchana, et.al., Addressing Trans-boundary Issues and Consolidating Bilateral Arrangements to Combat IUU Fishing, Fishing for the People, 2016 76 ANNEX 8: ACHIEVEMENTS AND EXPECTED OUTCOME/ OUTPUTS, 2016-2018 REPORTS ACHIEVEMENTS AND EXPECTED OUTCOME/OUTPUTS OF THE ACTIVITY 2018 Planned activity Expected outcome/output Achievements Activity 1: Regional Sub-activity 1.1: Third TWG Planning Workshop Enhanced coordination, capacity, work plan development Inputs received from TWG members on regional priorities; Increased capacity and coordination for Year 4 Work Plan Sub-activity 1.2: Workshop for the Finalization of Sulu￾Sulawesi Seascape Sub￾region EAFM Plan Regional EAFM Plan, Enhanced Capacity Finalization and socialization of draft Regional Sustainable Fisheries Management Plan Sub-activity 1.3: e-CDT learning exchange and ASEAN Tuna Ecolabelling Consultative Workshop Enhanced capacity for eCDT Learning site lessons learned shared with regional partners; capacity increased on eCDT development and implementation Activity 2: Philippines Sub-activity 2.1: EAFM Planning Workshop for Crafting the Sarangani Bay and Sulawesi Sea Fisheries Management Plan with the Technical Working Group and Stakeholders Sustainable Fisheries Management Furthered development of learning site Sustainable Fisheries Management Plan Sub-activity 2.2: BFAR eCDTS Development workshop Enhanced capacity for eCDT Capacity increased on eCDT development and implementation Sub-activity 2.3: Seafood Watch Standard and Methodology workshop Capacity building, Sustainable Fisheries Management Engaged stakeholders; built capacity and understanding of importance of data-drive sustainable fisheries management Sub-activity 2.4: Workshop for the Use of FAME Technology and Simplified Catch Report Capacity building for eCDT implementation Capacity increased on eCDT development and implementation 77 Activity 3: Indonesia Sub-activity 3.1: Formulation of Management Measures and Quantitative References to Develop Harvest Strategy for Fisheries in the FMA 716 Sustainable Fisheries Management Furthered development of learning site Sustainable Fisheries Management Plan Sub-activity 3.2: Seafood Watch Standard and Methodology workshop Capacity building, Sustainable Fisheries Management Engaged stakeholders; built capacity and understanding of importance of data-drive sustainable fisheries management Activity 4: Expansion Sub-activity 4.1: Validation Meeting eCDTS in Viet Nam Stakeholder Engagement, CDT Development and EAFM Planning Engaged stakeholders, furthered development of eCDT planning ACHIEVEMENTS AND EXPECTED OUTCOME/OUTPUTS OF THE ACTIVITY YEAR 2017 Planned activity Expected outcome/output Achievements Activity 1 Sub-activity 1.1: 2nd Annual TWG Meeting and Planning Workshop Enhanced coordination, capacity, work plan development Increased capacity and coordination for Year 3 Work Plan Sub-activity 1.2: Regional EAFM Workshop Regional EAFM Plan, Enhanced Capacity Draft Regional Sustainable Fisheries Management Plans Sub-activity 1.3: Regional Gender Workshop Enhanced Capacity, Advance Gender Equity and Equality interventions Enhanced capacity and coordination, progress on advancing Gender equity and equality objectives Sub-activity 1.4: CDT 201 Development CDTS Legacy Publication Established CDTS technical requirements, engaged Technical Advisory Group to conduct expert review Sub-activity 1.5: KDE Manual Development CDTS Legacy Publication Established KDE requirements for CDTS along with complete definitions, engaged Technical Advisory Group to conduct expert review Sub-activity 1.6: Oceans and Fisheries Partnership Web Portal External Communications Development of www.seafdec￾oceanspartnership.org 78 Sub-activity 1.7: The Oceans and Fisheries Partnership eNewsletter External Communications Development of Making Waves eNewsletter; two distributions to over 900+ stakeholders Activity 2 Sub-activity 2.1: Philippines Stakeholder Consultation Workshop Stakeholder Engagement Validation results of learning site studies, increased capacity and coordination, initiated development of Sustainable Fisheries Management Plan Sub-activity 2.2: Rapid Appraisal of Fisheries Management Systems (Philippines) Sustainable Fisheries Management Furthered development of learning site Sustainable Fisheries Management Plan Sub-activity 2.3: Crafting for the Fisheries Component of the Protected Area Management Plan of Sarangani Bay Protected Seascape Sub-activity 2.4: CDTS Planning and Design Workshop CDTS Development Development of the CDTS back-end and interface, increased capacity within the Philippines Bureau of Fisheries and Aquatic Resources Sub-activity 2.5: Technical Workshop on CDTS Development Sub-activity 2.6: Agile Training for Development of the CDTS Sub-activity 2.7: Exhibition at 19th National Tuna Congress Stakeholder Engagement Presented learning site CDTS, increased general awareness of Partnership activities Activity 3 Sub-activity 3.1: Indonesia Stakeholder Consultation Workshop Stakeholder Engagement Validation results of learning site studies, increased capacity and coordination, initiated development of Sustainable Fisheries Management Plan Activity 4 Sub-activity 4.1: Thailand Stakeholder Consultation Workshop Stakeholder Engagement, CDT Development and EAFM Planning Engaged stakeholders, furthered development of CDTS planning and development of Sustainable Fisheries Management Plan 79 Sub-activity 4.2: Viet Nam Country Consultation Stakeholder Engagement, CDT Development and EAFM Planning Engaged stakeholders, furthered development of CDTS planning and development of Sustainable Fisheries Management Plan ACHIEVEMENTS AND EXPECTED OUTCOME/OUTPUTS OF THE ACTIVITY 2016 Planned activity Expected outcome/output Achievements Activity 1 Theory of Change Workshop CDT 101 Development CDT 201 Development Theory of Change, Situation Analysis, revised indicators CDTS Foundational Publication CDTS Foundational Publication Developed refined Theory of Change, Situation Analysis, and indicators on which to guide strategic planning Established conceptual overview of CDTS and Oceans objectives, solicited and received feedback from leading sector organizations, industry, and academia Developed technical overview of CDTS, technical requirements, and specifications for review by Technical Advisory Group and subsequent publication Sub-activity 1.1 – Partnership Prioritization Workshop Strategy for Partner Engagement Partnerships formalized with GFTC, Monterey Bay Seafood Watch, coordination with Thai Union and leaders to support CDTS implementation in Indonesia and the Philippines Sub-activity 1.2 – TWG Training and Workshop Networking, Country￾Specific Work Plans, Consistent Knowledge Baseline for Capacity Building Baseline established for future capacity building activities, country-specific work plans drafted, and information gathered for country profiles Sub-activity 1.2 – SEAFDEC EAFM Training, Siem Reap Increased program visibility, Capacity building Overview of Cambodia EAFM priorities, facilitation of 7 out of 17 presented modules, strengthened regional networking Sub-activity 1.3 – 6th Global Symposium on Gender in Aquaculture and Fisheries Increased program visibility, Capacity building Visibility increased for gender mainstreaming in fisheries, capacity building, strengthened regional collaboration 80 Activity 2 Sub-activity 2.1 – Philippines Country Consultation (multiple meetings) Raised Awareness/ Government Support and TWG Participation Provided overview of Oceans, secured stakeholder buy-in, and identified priority challenges and opportunities for collaboration Sub-activity 2.1 - 2.2 – Mainstreaming EAFM Finalized EAFM Planning Process Handbook, Planned activity Expected outcome/output Achievements BFAR National EAFM TWG Workshop (additional activities are included in Activity 3 detail) Planning Process Handbook Developed work plan for testing EAFM Planning Process Training Modules Activity 3 Sub-activity 3.1 – General Santos City Learning Site Launch and Technical Session Formal launch of site￾level activities Secured stakeholder buy-in, cemented program commitment to support of General Santos City and relevant government agencies (BFAR, DENR) Sub-activity 3.1 – Value Chain Analysis (General Santos City) Analysis Report Identified learning site supply chain issues and opportunities, profiled supply chain components, guidance for learning site supply chain strategies Sub-activity 3.3 – Planning for Rapid Appraisal for Fisheries Management (General Santos) Appraisal Report Identified fisheries management issues and opportunities, profiled fisheries and fishing communities, guidance on the development of CDT and human welfare interventions. Sub-activity 3.4 – Labor Rapid Appraisal (General Santos) Appraisal Report Identified labor rights issues and opportunities, guidance on learning site strategy and interventions Sub-activity 3.4 – Planning for Gender Assessment (General Santos) Assessment Report Identified gender concerns and opportunities, guidance on learning site strategy and interventions 81 Activity 4 Sub-activity 4.1 – Indonesia Country Consultation (multiple meetings) (additional activities are included in Activity 5 detail) Raised Awareness/Governme nt Support and TWG Participation Provided overview of Oceans, secured stakeholder buy-in, and identified priority challenges and opportunities for collaboration Activity 5 Sub-activity 5.1 – Planning for Rapid Value Chain Analysis Sub-activity 5.1 – Regional Fisheries Summit, Meetings with technology partners, USAID/Indonesia Analysis Report Strengthened networking; CDTS design development Identified potential Value Chain Analysis subcontractors; Identified potential technology solutions for the CDTS; coordinated with USAID/Indonesia; increased visibility and networking at Economist Regional Fisheries Summit Activity 6 Sub-activity 6.1 – Country Consultation – Thailand Raised Awareness/ Government Support and TWG Participation Provided overview of Oceans, secured stakeholder buy-in, identified priority challenges and opportunities for collaboration, discussed Songkla as Oceans learning site Sub-activity 6.2 – Country Consultation – Malaysia Raised Awareness/ Government Support and TWG Participation Provided overview of Oceans, secured stakeholder buy-in, identified priority challenges and opportunities for collaboration, discussed Kelantan as Oceans learning site Sub-activity 6.3 – Country Consultation – Brunei Raised Awareness/ Government Support and TWG Participation Provided overview of Oceans to Department of Fisheries, secured buy-in, and identified priority challenges and opportunities for collaboration; Conducted visits to fish landing sites and met with port officers. Sub-activity 6.3 – Country Consultation – Cambodia Provided overview of Oceans, secured buy-in, and identified priority challenges and opportunities for collaboration. Sub-activity 6.3 – Country Consultation - Mynamar Provided overview of Oceans, secured buy-in, and identified priority challenges and opportunities for collaboration. Sub-activity 6.3 – Country Consultation – Laos Provided overview of Oceans to Fisheries Administration, secured buy-in, and identified priority challenges and opportunities for collaboration; Conducted visits to fish markets and landing sites. 82 Sub-activity 6.3 – Country Consultation – Singapore Provided overview of Oceans to Government of Singapore, secured buy-in, and identified priority challenges and opportunities for collaboration. Sub-activity 6.3 – Country Consultation – Viet Nam Provided overview of Oceans to USAID/Viet Nam, Embassy representatives, Viet Nam Fisheries Department; secured buy-in; and identified priority challenges and opportunities for collaboration. Sources: Reports of the 39th, 40th and 41st Meetings of the Program Committee of the Southeast Asian Fisheries Development Center 83 ANNEX 9: EVALUATION TEAM COMPOSITION The ET includes a Senior Team Leader, Alan Ferguson, an accomplished ET leader. Over his 30-year career, he has conducted 40 evaluations in Southeast Asia and Africa for donors such as the United Nations (UN), AusAid, and DFAT. As Mr. Ferguson brings demonstrated technical expertise on issues surrounding fisheries management, climate change, and biodiversity in Southeast Asia, he will also serve as the Senior Fisheries and Marine Management Expert. Michael Duthie, Senior M&E Specialist, demonstrates 12 years of experience designing, guiding, and conducting performance and impact evaluations, primarily in Southeast Asia. He has expertise in USAID evaluation policy, rules, and regulations, and a long-term relationship conducting evaluations with USAID/RDMA. Mr. Duthie has also implemented evaluations for USAID in environmental protection and climate change sectors. He will provide overall Quality Assurance (QA), and support to the TL on all aspects of the evaluation. Senior Fisheries and Marine Management Expert, Dadang Setiawan, is an Indonesian citizen and has over 15 years of experience providing technical assistance to USAID, government agencies, and other bilateral donors in marine and fisheries management in Indonesia. He will serve in a key advisor role, ensuring local context is incorporated into the evaluation. Further details of each team member can be found below. Senior Team Leader, Mr. Alan Ferguson demonstrates 30 years of evaluation leadership experience, serving as TL on 40 evaluations of projects addressing factors affecting fisheries management, climate change finance/adaptation, and environment/biodiversity worldwide. He has worked across South East Asia, including Philippines, Lao PDR, Vietnam, Thailand, and Timor-Leste. Mr. Ferguson brings proven technical knowledge and understanding of the complexities, challenges, and structures of biodiversity conservation, climate change, and fisheries management. He has worked with Canadian government agencies and ministries to develop strategic plans, reviews, and strategy plans at the donor and project levels, on coastal resources management, fisheries conservation, fish sustainability, issues and organization for integrated management of coastal areas, climate change practices, and climate change integration. He is deeply knowledgeable on donor rules and regulations with donor agencies such as the UN, World Bank, GIZ and AusAid. His impressive career as an evaluation TL includes leading a PE of several climate change adaptation initiative demonstration projects across the Mekong region, including Thailand. In Bangladesh, he served as TL of a mid-term evaluation of climate change adaptation through a coastal afforestation project, involving technical, socio-economic, and policy issues. He also recently served as TL of a mid-term evaluation for UNDP Myanmar, addressing climate change risks on water resources, and similarly, led a final evaluation on climate-resilient water management practices in Cambodia. Additionally, he served as TL for a final review of a $10 million Cambodia Climate Change Alliance program of policy and capacity development, including 20 adaptation demonstration projects, jointly funded by a variety of donors. A frequent TL, he designs approach, methodology, data collection tools, and works with team members to synthesize and present data into comprehensive reports including recommendations for clients on outcomes for future programming. He is knowledgeable on the Open Standard tool and is skilled in qualitative methods, from KIIs to FGDs and simple surveys. Mr. Ferguson holds an M.A. in Geography from the University of Victoria and has completed resource management science courses from the University of British Colombia. Senior M&E Specialist, Mr. Michael Duthie is a Senior Technical Advisor at SI and has led or supported more than 50 PEs and impact evaluations throughout Asia (primarily Southeast Asia) and Africa. Mr. Duthie has implemented evaluations covering a variety of sectors, including environment, climate change, conservation, and biodiversity. Having supported evaluations with USAID Missions including USAID/RDMA, Mr. Duthie is deeply familiar with USAID evaluation policy, rules, and regulations. He has extensive experience in evaluation of environmental protection and climate change projects, having led a team in the evaluation of the USAID CTI project and overseen evaluations of the USAID climate change portfolio in the Pacific, a community forestry project in 84 Liberia, and a renewable energy project in Indonesia. Prior to joining SI, he worked as a research assistant on population biology, modeling, and mapping population dynamics of endangered species and on research related to environmental policy in the U.S. Currently based in Kentucky, Mr. Duthie also resided in Thailand and Indonesia, bringing regional knowledge to the evaluations he conducts. Mr. Duthie speaks English and has excellent writing, presentation, and interpersonal skills, with proven relationships within USAID/RDMA from his prior evaluations in the region. He holds an M.A. in Social Policy in Developing Countries with Merit from The London School of Economics and Political Science, and a B.A. in Environmental Biology from the University of Virginia. Fisheries and Marine Management Expert, Mr. Dadang Setiawan is an Indonesian citizen, with over 15 years of experience providing technical assistance and advisory services in marines and fisheries, climate change, natural resources management, and environmental development. He served as a Climate Change Adaptation Researcher on a vulnerability assessment for a USAID-funded project in Indonesia, carrying out the desk review and contributing key findings and conclusions to the final report. As a technical expert, Mr. Setiawan worked closely with government agencies and other stakeholders, assisting to establish Local Marine Management Areas (LMMAs) and local regulations on marine conservation areas in eight districts in Indonesia. He was the lead author of key guidelines and manuals that helped establish Marine Management Areas/Marine Protected Areas (MMA/MPAs), as well as the design of a community-based trough LMMA. With the ADB, Mr. Setiawan led the inception report of sustainable forestry and biodiversity management in Borneo. He has authored publications on climate change impact on oceans, management for sustainable fisheries in Indonesia, and community development in the biodiversity conservation and climate change framework. Mr. Setiawan holds an M.A. in Environmental Science from the University of Indonesia and speaks fluent Indonesian and proficient English. Additionally, the ET is supported by logisticians and interpreters for in-country fieldwork. At Head Quarters (HQ), ET members will be supported by a Project Director (PD) Amanda Stek, Project Manager (PM) Soham Banerji and Project Assistant (PA) Jennifer Elkins. 85 ANNEX 10: EVALUATION DESIGN MATRIX EVALUATION DESIGN MATRIX EVALUATION QUESTIONS EVALUATION INDICATORS DATA SOURCES/COLLECTION DATA ANALYSIS METHODS 1. To what extent have the USAID Oceans’ approaches achieved and/or advanced the USAID Oceans’ objectives on a) regional coordination, b) PPP and c) capacity building? a. Regional coordination: i) To what extent has USAID Oceans’ work with SEAFDEC and CTI-CFF enhanced their ability to: counter IUU fishing and strengthen fisheries management across ASEAN member countries? 1.1 Changes in SEAFDEC and CTI-CFF organization reflecting capacity development to counter IUU fishing and strengthen fisheries management 1.2 Changes in resources available for IUU and FM work within SEAFDEC and CTI-CFF 1.3 HRD related to IUU and Fisheries Management that is attributed to the Activity including ability to complete eCDT gap analysis 1.4 SEAFDEC, CTI-CFF, USAID and other participant satisfaction with the capacity development role in ASEAN countries Response to regional interviews and FGDs with SEAFDEC and CTI-CFF and ASEAN country representatives Completed SEAFDEC and CTI-CFF activities and progress related to eCDT and EAFM with USAID Oceans support including Activity progress reports See Figure 10/11/12 of MTR on approaches and mechanisms, and capacity needed to promote adoption of electronic catch documentation and traceability systems Review of organizational changes in SEAFDEC and CTI-CFF in relation to the EQ to determine extent of capacity development Review of gap analysis of ‘capacity needs assessment’ noted in the MTR Analysis of interview data on capacity development in relation to the EQ a.: ii) To what extent has USAID Oceans’ work with SEAFDEC and CTI-CFF enhanced their ability to: effectively support ASEAN member countries’ adoption and use of eCDT systems to counter IUU Fishing and strengthen fisheries management? 1.5 Changes in SEAFDEC and CTI-CFF organization to support eCDT 1.6 Changes in resources available for support to eCDT 1.7 HRD related to eCDT promotion that is attributed to the Activity including ability to complete eCDT gap analysis 1.8 SEAFDEC, CTI-CFF, USAID and other participant satisfaction with the capacity development and the level of support to countries 1.9 Evidence of eCDT activity results (Annex 8); Activity Indicators 2,3,4,5,9 data Response to Regional interviews with SEAFDEC and CTI-CFF and ASEAN country reps Completed SEAFDEC and CTI-CFF activities and progress related to eCDT with USAID support Review of available Activity M&E data on eCDT activity results Analysis of interview data on capacity development for eCDT in relation to the EQ 86 b. Public-private partnership and engagement: i)To what extent does USAID Oceans effectively engage national governments and private sector partners within learning sites and expansion sites (Kelantan, Songkhla) in formalizing partnerships for a fully sustainable eCDT system? 1.10Project indicator 12. Number of new USG￾supported public-private partnerships (PPPs) formed 1.11 Scope of government partnership agreements and effectiveness of engagements on eCDT at Activity Learning Sites 1.12 Partner and USAID satisfaction with partnership results to date 1.13 Activities under partnership agreements are likely to be continued following conclusion of the agreements 1.14 Partner implementation of comprehensive eCDT system Scope of partnership agreements (9) ‘scope’ may involve i) consultation/info sharing, ii) cost-sharing, iii) technology development, iv) policy development, v) contracted testing, and vi) other activities) Response to Learning Site interview questions regarding effectiveness, conditions, implementation and sustainability of engagements Analysis of PPP Indicator 12 data in relation to the EQ Content analysis of partnership agreements Analysis of interview data on partner satisfaction, implementation, and sustainability to determine the effectiveness of engagements b. Public-private partnership and engagement: ii) Which incentives for adopting eCDT systems are the most appealing to private sector partners to encourage adoption of eCDT systems? 1.15 Types of Incentives established by the Activity 1.16 Extent of producer/processor participating – not participating in the incentives 1.17 Stakeholder rating of the Incentives (which ones have been most/least effective and why?) List of and analysis of Incentives – financial, regulatory, non-regulatory Response to Learning Site interview questions Examine reasons why producers (fishers) are under-represented in PPPs (MTR observation) Analysis of incentives data and interview responses on preferences c. Capacity building and institutionalization: i) How promising have USAID Oceans’ capacity building efforts been in helping key targeted beneficiaries to achieve the following results? i. Strengthening SEAFDEC capacity as the regional champion for moving ASEAN member countries to adoption and use of an eCDT system; and ii. Building local and national government agencies to develop and implement an eCDT system. 1.18 Beneficiary perceptions and other evidence/data on the results of SEAFDEC and local/national government support 1.19 Data from Activity Indicators #2, 3, 4, 5, and 9 as relevant to measure movement of ASEAN countries and local/national agencies to adoption and use of eCDT [see also Question 2e below] SEAFEC activities completed in support of eCDF at the regional level SEAFEC activities completed in support of eCDF at the local/national level Compilation of Activity Indicator data and results achieved from capacity development activities Responses to Learning Sites interview questions Internet survey as needed Review of available Activity data on the relevant Activity Indicators and the activities completed to assess how ‘promising’ the capacity building has been. Analysis of interview data on beneficiary perceptions of capacity building efforts in relation to the EQ 87 c. Capacity building and institutionalization: ii). To what extent have human welfare, particularly gender equity and labor considerations, been addressed within the promotion and adoption of the eCDT systems being supported by USAID Oceans? 1.20 Activity indicator #8: Number of legal instruments drafted, proposed, or adopted with USG assistance designed to promote gender equality or non￾discrimination against women or girls at the regional, national or sub-national level 1.21 Gender equity provisions in eCDT support measures in work plans and reported results on enhanced gender equity (e.g., enhanced awareness and commitment) 1.22 Labor considerations in eCDT support measures in work plans and reported results on enhanced human welfare (e.g Human Welfare KDEs developed by the Activity) Disaggregated gender data on activities and beneficiaries Compilation of Activity Indicator #8 data Response to Regional and Learning Site interview questions Review of consolidated Activity data in relation to the EQ Analysis of interview data on gender and labor considerations in relation to the EQ and the extent of their consideration 2 What factors (both internal and external) have enhanced, diminished, or have no effect on achievement of USAID Oceans expected results in reducing IUU fishing and strengthen fisheries management? Specific factors assess include, but are not limited to, the following: a. eCDT adoption and use among private sector “early mover” partners within seafood supply chains at both USAID Oceans learning sites 2.1 Level of eCDT adoption by early movers; see Activity Indicators 2,3,4,5 2.2 Identification of eCDT adoption and use as important to IUU fishing and fisheries management 2.3 Identification of a demonstration effect (by first movers) as an important factor Early eCDT movers identified in seafood supply chain. Response to Learning Site interview questions Review of Activity Indicator data in relation to the EQ Analysis of Learning Sites interview data in relation to the EQ b. EAFM among national, provincial, and local fisheries management authorities at both USAID learning sites 2.4 Development and passing of EAFM plans 2.5 Identification of EAFM plans developed by the project as a factor influencing IUU fishing and fisheries management 2.6 Identification of EAFM more generally as an important factor in reducing IUU fishing EAFM activities completed at Learning Sites Response to Learning Site interview questions Analysis of Learning Sites interview data in relation to the EQ c. Regional coordination among ASEAN and CTI-CFF member countries supported by SEAFDEC and the CTI￾CFF Regional Secretariat 2.7 Identification of a regional response as an important factor 2.8 Identification of SEAFDEC and/or CTI-CFF coordination and/or support as an important factor Regional coordination activities completed by SEAFDEC and CTI-CFF. Responses to Regional interview questions Analysis of Regional interview data in relation to the EQ 88 d. PPP among private sector “early mover” partners within seafood supply chains at both USAID learning sites? 2.9 Identification of PPPs as an important factor 2.10 Identification of private sector engagement as an important factor Response to Learning Site interview questions (see also Activity Indicator 1.14 - extent of producer/processor participation in incentives) Analysis of Learning Sites interview data in relation to the EQ e. Capacity building and institutionalization of eCDT approaches and guidance within SEAFDEC and CTI￾CFF Regional Secretariat 2.11 Identification of increased capacity of SEAFDEC or CTI-CFF as an important factor 2.12 Identification of institutionalization of eCDT in regional organization (SEAFDEC or CTI-CFF) as an important factor 2.13 Identification of guidance information provided from a regional level as an important factor Scope and characteristics of eCDT approaches in SEAFDEC and CTI-CFF Response to Regional interview questions Analysis of Regional Sites interview data in relation to the EQ f. Fisheries supply chain-related human welfare and gender equity awareness and actions being taken by stakeholders within the seafood supply chains at both USAID Oceans learning sites 2.14 Characteristics of human welfare and g.e. aspects of eCDT and EAFM in activities 2.15 Identification of human welfare or gender equity actions as important factors Response to Learning Site interview questions Any gender and welfare assessments conducted by the Activity stakeholders Disaggregated gender data on activities and beneficiaries Compilation of Activity Indicator #8 data. Analysis of Learning Sites interview data in relation to the EQ 3. What evidence(s) exists that the Association of Southeast Asian Nations (ASEAN) and CTI countries, especially in the learning and expansion sites would potentially sustain the adoption and implementation of the Electronic Catch Documentation and Traceability (eCDT) systems to counter IUU fishing and strengthen fisheries management as a result of USAID Oceans' support? a. What aspects of sustainability (legal, institutional, financial, cultural, etc.) can be observed in the Activity implementation and results? 3.1 Institutional sustainability indicators (laws, regulations, codes, best practices, technical support of TWGs and others) related to eCDT and EAFM 3.2 Financial sustainability indicators (direct subsidies, loan guarantees, tax incentives) related to eCDT and EAFM 3.3 Socio-cultural indicators (industry sector support, community support) related to eCDT and EAFM Response to Regional interview questions Response to Learning Site interview questions Internet survey as needed Analysis of Learning Sites interview data in relation to the EQ b. What specific actions by which stakeholders (i.e., USAID/RDMA, USAID Oceans, SEAFDEC, CTI-CFF and beyond) will be required to 3.4 Formal commitment made by SEAFDEC, CTI-CFF and ASEAN partners to sustain results 3.5 Formal commitment made by Learning Site partners to sustain results Response to Regional interview questions Response to Learning Site interview questions Internet survey as needed Analysis of Learning Sites interview data in relation to the EQ 89 ensure future sustainability of eCDT, EAFM after the USAID Oceans ceased. 3.6 Budgets allocated for ongoing support to eCDT/EAFM by partners 3.7 USAID and other donor planned commitments for ongoing support 3.8 Budgets allocated for ongoing support by ASEAN members