July 2020 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Pacific Rim Innovation and Management Exponents, Inc. USAID/INDONESIA’S SUSTAINABLE ECOSYSTEMS ADVANCED (SEA) PROJECT: MID-TERM PERFORMANCE EVALUATION (MTPE) FINAL MTPE REPORT USAID/INDONESIA’S SUSTAINABLE ECOSYSTEMS ADVANCED (SEA) PROJECT: MID-TERM PERFORMANCE EVALUATION FINAL MTPE REPORT Prepared under Task Order: AID-497-C-16-C00006 Submitted to: USAID/Indonesia: 11 July 2020 Prepared by: Mr. Geronimo T. Silvestre, Team Leader/Marine Conservation and Sustainable Fisheries Specialist Mr. Luis P. Eleazar, Senior Evaluation Specialist Dr. Imelda G. Pagtolun-an, Senior Survey Specialist/Statistician Contractor: Pacific Rim Innovation and Management Exponents, Inc. (PRIMEX) 502 Manila Luxury Condominium 30 Pearl Drive, Ortigas Center, Pasig City, Philippines Tel: (632) 8633-9052, 8633-3717, 8634-7338, 8635-2924 Fax: (632) 8634-7340 Email: primexinc@primexinc.org Website: www.primexinc.org Photo Credits: @Ms. Fitria Wahid, USAID Indonesia. Taken during site visits to Pasinalo, Taniwel, 25 January 2020. DISCLAIMER. The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. i | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV ABSTRACT The USAID Sustainable Ecosystems Advanced (SEA) Project (AID-497-C-16-00008) is a five-year (March 2016-March 2021) program of activities designed to support the Government of Indonesia (GOI) in improving the governance of fisheries and marine resources and conserving biological diversity. Funded by USAID/Indonesia at a cost of US$31.9 million, the project operates at the national level (principally with the Ministry of Marine Affairs and Fisheries [MMAF]), Fisheries Management Area (FMA) 715 level, and provincial/site levels (in North Maluku, Maluku, and West Papua provinces). The USAID SEA Project consists of five strategic approaches (SAs) or components, namely: (i) SA1: create demand for marine and fisheries resources management and marine conservation through awareness raising and advocacy; (ii) SA2: improve ecosystem management of FMA 715 and marine protected areas (MPAs); (iii) SA3: increase incentives for marine stewardship; (iv) SA4: advance the development of marine and fisheries policies and regulations; and (v) SA5: institutionalize training and capacity building. A Mid-term Performance Evaluation (MTPE) was commissioned to an independent third party (Pacific Rim Innovation and Management Exponents, Inc. [PRIMEX]) to provide USAID/Indonesia and GOI with an assessment of project achievements at mid-term in relation to selected SAs, including factors, conditions, and challenges that enhanced or diminished the achievement of project results. The MTPE was conducted during the period, October 2019–July 2020 and assessed project performance over the period, 21 March 2016–30 September 2019. The MTPE utilized three data collection methods, namely: (i) desk review of available project documents; (ii) key informant interviews (KIIs); and (iii) focus group discussions (FGDs). The project’s mid-term performance was assessed based on progress indicators reported in its Monitoring, Evaluation, and Learning (MEL) Plan (supported by project reports/documents) and validated using the KII and FGD outputs. The MTPE concluded that, overall, the project has satisfactorily achieved its planned activities and targets and is largely on track in attaining its life of project (LOP) targets, as of mid-term. Of its 11 MEL performance indicators directly relevant to the MTPE, three have been substantially exceeded, six are on track, and two need improvement to achieve the set LOP targets. The MTPE found that the project has been largely responsive to challenges encountered in project implementation, as of mid-term. Facilitating and hindering factors impacting project performance and recommendations to improve implementation effectiveness in selected project SAs are outlined in this report. The MTPE proposed the following recommendations on thematic areas requiring strengthening or expansion: (i) explicit incorporation of SA4 in the project’s Theory of Change (TOC) results chain for SA3 and SA5; (ii) strengthening and streamlining of the policy development process to facilitate adoption of enabling legal instruments; (iii) provision of incentives and greater visibility for model marine stewardship efforts for wider replication; (iv) improved communication, coordination, and collaboration among implementing partners in various elements comprising the EAFM approach; (v) incorporation of ecosystem approaches in research studies; (vi) addressing gender empowerment; and (vii) improving the sharing of best practices. It also recommends that the project concentrate on ongoing and unfinished activities (and the thematic improvements noted above), rather than expand its geographic coverage, in order to maintain the focus on achieving sustainable impacts in its target areas/sites within the remaining project life. Finally, the MTPE report provides recommendations for the consideration of USAID/Indonesia when designing similar projects in the future. ii | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV TABLE OF CONTENTS Abstract................................................................................................................................................................................. i Acronyms............................................................................................................................................................................ iv Executive Summary ........................................................................................................................................................ viii 1. Introduction....................................................................................................................................................................1 Project Background......................................................................................................................................................1 Alignment of SEA Project Objectives and Activities with US-Indonesia Priorities.......................................2 2. Evaluation Objectives and Questions.......................................................................................................................3 Evaluation Objectives...................................................................................................................................................3 Evaluation Questions....................................................................................................................................................4 3. Evaluation Methods and Limitations.........................................................................................................................4 Evaluation Methods......................................................................................................................................................4 Data Analysis Methods................................................................................................................................................7 Limitations......................................................................................................................................................................7 4. Findings, Conclusions, and Recommendations.......................................................................................................8 5. General Conclusions and Recommendations ......................................................................................................40 Overall Project Performance ...................................................................................................................................40 Challenges and Influencing Factors for Selected Mid-term Results................................................................42 Recommendations for the Remaining Project Period........................................................................................43 Recommendations to USAID for Future Project Design..................................................................................45 Annex A. Statement of Differences.............................................................................................................................47 Annex B. Theory of Change of USAID SEA Project...............................................................................................51 Annex C. USAID SEA Project Areas..........................................................................................................................52 Annex D. The Evaluation Statement of Work (SoW)............................................................................................53 Annex E. Team Organization, Composition, and Assigned Tasks.......................................................................62 Annex F. List of Documents Reviewed......................................................................................................................66 Annex G. KII Guide Questions....................................................................................................................................78 Annex H. List of KII Informants................................................................................................................................ 155 Annex I. FGD Guide Questions................................................................................................................................ 157 Annex J. List of FGD Participants............................................................................................................................. 199 Annex K. FGD Photo Documentation.................................................................................................................... 201 Annex L. Summary of Data Triangulation Matrix ................................................................................................. 204 Annex M. Summary of Training Courses Provided by the Project to Key Stakeholders, FY2016 – FY2019 ........................................................................................................................................................................................... 253 iii | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV FIGURE Figure 1. Simplified TOCs of SA4, SA5, and SA3, and their Functional Linkages.............................................34 TABLES Table 1. The Five Components/SAs of the Project...................................................................................................1 Table 2. Number of KII Participants, by Organization, and Number of KII Sessions........................................5 Table 3.FGD Participants, by Gender and Project Intervention ............................................................................6 Table 4. Types and Status of Laws, Policies, Regulations, Strategies, and Plans Introduced by the USAID SEA Project, 09/30/2019 ..................................................................................................................................................9 Table 5. Progress on Adoption of Laws, Policies, Strategies, Plans, or Regulations, 09/30/2019 ................10 Table 6. Progress on Number of People Trained, Innovations Supported, and People Applying Improved Conservation Law Enforcement Practices, 09/30/2019 .........................................................................................15 Table 7. Number of Trainings Completed by the Project, April 2016-September 2019...............................16 Table 8. Progress on Number of People Demonstrating Behavior Change, as of 09/30/2019 ....................19 Table 9. Results of Behavior Change Assessments Conducted by USAID SEA, 2017-2019.........................20 Table 10. Evaluation Findings on Dominant Characteristics of Behavior Change in Galo-galo, North Maluku, January 2020......................................................................................................................................................21 Table 11. Key Activities and Regulations in LMMA and TURF Establishment/Management .........................26 Table 12. Progress on Number of People who have More Secure Tenure and Apply Improved Conservation Law Enforcement Practices Indicators, as of 09/30/2019............................................................28 Table 13. Results of Selected Assessments of Target Villagers Involved in Conservation Activity Implementation, FY2017–FY2019................................................................................................................................28 Table 14. Evaluation Findings on Sustainable Fishing Methods Applied by Fishers in Target Villages with More Secure Marine Tenure, Maluku and West Papua, March 2020 .................................................................30 Table 15. Summary of MTPE Recommendations based on Responses to EQ1–EQ6.....................................43 iv | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV ACRONYMS AP2HI Asosiasi Perikanan Pole and Line dan Handline Indonesia (Indonesian Pole and Line and Handline Fisheries Association) AMORE Alliance for Mindanao Off-Grid Renewable Energy BCC Behavior change communication BEEP Best Environmental Equitable Practices BRSDM Bureau of Research and Human Resources BS Bachelor of Science CBO Community-based organization CDCS Country Development Cooperation Strategy CEO Chief Executive Officer CFA Customary fisheries area CI Conservation International COC Chain of custody COR Contracting Officer’s Representative COP Chief of Party CPUE Catch per unit effort CSO Civil society organization CTC Coral Triangle Centre DCOP Deputy Chief of Party DG Directorate General DKP Dinas Kelautan dan Perikanan (Marine and Fisheries Agency) E-KKP3K Efektivitas Pengelolaan Kawasan Konservasi Perairan, Pesisirdan Pulau-pulau Kecil (Technical Guidelines for Evaluating the Management Effectiveness of Aquatic, Coastal, and Small Island Conservation Areas) EAFM Ecosystem approach to fisheries management ECOFISH Ecosystems Improved for Sustainable Fisheries ETP Endangered, threatened, and protected EQ Evaluation question FA Fishers’ association FCMC Fisheries co-management committee FGD Focus group discussion FMA Fisheries management area FMU Fisheries management unit FT Fair Trade FTF Feed the Future GOI Government of Indonesia GTZ German Technical Cooperation Agency HHS Household survey ICM Integrated coastal management ICM PH Integrated Coastal Management in the Philippines ICR Implementation completion result v | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV ILMMA Indonesia locally managed marine area IMACS Indonesia Marine and Climate Support IUUF Illegal, unregulated, and unreported fishing JICA Japan International Cooperation Agency KII Key informant interview KKHL Konservasi dan Keanekaragaman Hayati Laut (Conservation and Marine Biodiversity) KKNI Kompetensi Kerja Nasional Indonesia (Indonesian National Work Competency) KKP Kementerian Kelautan dan Perikanan (Ministry of Marine Affairs and Fisheries) KPI Key performance indicator KSOP Kantor Kesyahbandaran dan Otoritas Pelabuhan (Harbor Master Office and Port Authority) LGU Local government unit LMMA Local marine management area LOP Life of project M&E Monitoring and evaluation MC Marine Change MEL Monitoring, evaluation, and learning MDPI Masyarakat dan Perikanan Indonesia (Indonesian Society and Fisheries Foundation) MFRM Marine and fisheries resources management MMAF Ministry of Marine Affairs and Fisheries MOHA Ministry of Home Affairs MOU Memorandum of Understanding MPA Marine protected area MS Master of Science MSC Marine Stewardship Council MSP Marine spatial planning MTPE Mid-term Performance Evaluation NGO Non-government organization NOAA National Oceanic and Atmospheric Administration NRM Natural resource management OOC Our Ocean Conference OSPC Open Standards for the Practice of Conservation P3D Participatory three dimensional PEMSEA Partnerships in Environmental Management for the Seas of East Asia PerDa Peraturan Daerah (local regulation) PerMen Peraturan Menteri (ministerial regulation) PES Payment for environmental (or ecosystem) services PKK Pemberdayaan Kesejahteraan Keluarga (Family Welfare Guidance Program) PMI Performance management indicator POKDARWIS Kelompok Sadar Wisata (Community Tourism Group) POKJA Kelompok Kerja (Working Group) POKMASWAS Kelompok Masyarakat Pengawas (Community-based Surveillance Group) PPP Public-private partnership PRIMEX Pacific Rim Innovation and Management Exponents, Inc. PSDA Pengelolaan Sumber Daya Alam (natural resource management) vi | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV PSDI Direktorat Pengelolaan Sumber Daya Ikan (Directorate of Fisheries Resource Management) PSM Port state measure PSMA Port State Measure Agreement PTSP Pelayanan Terpadu Satu Pintu (One-Stop Integrated Services) PURISKAN Pusat Riset Perikanan (Fisheries Research Center) RCFI Reef Check Foundation Indonesia REA Rapid ecological assessment RPJMD Rencana Pembangunan Jangka Menengah Daerah (Provincial Medium-term Development Plan) RUM Resource use monitoring RZWP-3-K Rencana Zonasi Wilayah Pesisir dan Pulau-Pulau Kecil (Zoning Plan for Coastal Areas and Small Islands) SA Strategic approach SCW Stakeholder consultation workshop SDI Spatial data infrastructure SEA Sustainable Ecosystems Advanced SKKNI Standar Kompetensi Kerja Nasional Indonesia (Indonesian National Work Competency Standard) SOP Standard operating procedure SOW Statement of work SPAG Spawning aggregation SSF Small-scale fisheries TA Technical approach TCBFMA Traditional community-based fisheries management area TL Team Leader TNC The Nature Conservancy TOC Theory of Change TOT Training of trainers TPA Third party assessment TSP Trade Support Programme TURF Territorial use rights in fisheries UKIP Universitas Kristen Papua (Christian University of Papua) UNDP United Nations Development Programme UNIPA Universitas Negeri Papua (University of Papua) UNKHAIR Universitas Khairun (Khairun University) UP University of the Philippines UPLB University of the Philippines at Los Baños USA United States of America USAID United States Agency for International Development USG United States Government VIIRS Visible Infrared Imaging Radiometer Suite WB World Bank WCS World Conservation Society vii | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV WG Working group WWF World Wide Fund for Nature WFD World Fisheries Day WPP Wilayah Pengelolaan Perikanan (Fisheries Management Area) viii | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV EXECUTIVE SUMMARY BACKGROUND The USAID Sustainable Ecosystems Advanced (SEA) Project (AID-497-C-16-00008) is a five-year (March 2016-March 2021) program of activities designed to support the Government of Indonesia (GOI) in improving the governance of fisheries and marine resources and conserving biological diversity. Financed by a grant from the US Government (USG) in the amount of US$31.9 million, the project operates at the national level (principally with the Ministry of Marine Affairs and Fisheries [MMAF]), at the Fisheries Management Area (FMA) 715 level, and provincial/site levels (in North Maluku, Maluku, and West Papua provinces). The SEA Project consists of five strategic approaches (SAs) or components, namely: (i) SA1: create demand for marine and fisheries resources management and marine conservation through awareness raising and advocacy; (ii) SA2: improve ecosystem management of FMA 715 and marine protected areas (MPAs); (iii) SA3: increase incentives for marine stewardship; (iv) SA4: advance development of marine and fisheries policies and regulations; and (v) SA5: institutionalize training and capacity building. A Mid-term Performance Evaluation (MTPE) was commissioned to an independent third party (Pacific Rim Innovation and Management Exponents, Inc. [PRIMEX]) to provide USAID/ Indonesia and GOI with an assessment of project achievements at mid-term. Specifically, the MTPE aimed to: (i) review the project’s mid-term performance and analyze factors and conditions that enhanced or diminished the achievement of these results; (ii) identify the factors that constrained and/or influenced the achievement of selected mid‐term results (policies/regulations, training, livelihoods, and marine tenure); (iii) provide recommendations to improve project implementation effectiveness and to be considered by USAID in the design of future projects that will build Indonesia’s capacity and ability to effectively manage its own resources. To meet these objectives, the MTPE focused on six evaluation questions (EQs) based on the main elements of the project’s Theory of Change (TOC). The MTPE was conducted from October 2019–July 2020 and assessed project performance over the period, 21 March 2016–30 September 2019. The evaluation utilized three data collection methods, namely: (i) desk review of available project documents; (ii) key informant interviews (KIIs) involving 57 respondents from 24 organizations at national, provincial, and target site levels; and (iii) focus group discussions (FGDs): two FGD sessions with 18 respondents involved in non-fishing livelihoods in North Maluku and four FGD sessions with 41 respondents in Maluku and West Papua. The project’s mid-term performance was assessed based on progress indicators reported in its Monitoring, Evaluation, and Learning (MEL) Plan, supported by project reports/documents and validated using the KII and FGD outputs. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS EQ1: Under what conditions are the regulations and/or tools being introduced by USAID SEA to relevant authorities leading to the adoption of these policies, regulations, and tools? Triangulated data from the desk review and KII/FGD outputs revealed that National Law 23/2014 (Local Government Law) served as the key condition, which enabled and motivated national and provincial ix | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV governments to draft or adopt laws, policies, regulations, and tools in support of the project’s four technical approaches (TAs) under SA2 (viz., EAFM, marine protected areas [MPAs], marine spatial planning [MSP], and law enforcement). In line with the assignment of powers and responsibilities from district governments to provincial governments, as provided in the Local Government Law, a Governor’s Decree (Rencana Zonasi Wilayah Pesisir dan Pulau-Pulau Kecil [RZWP-3-K], or Marine Spatial Planning for Coastal and Small Islands) was passed in North Maluku and Maluku provinces. Other enhancing conditions cited by respondents include the recognition of the key threats to marine biodiversity and fisheries and the urgent need to develop institutional capacity and relevant policies, regulations, and tools to address existing threats to marine biodiversity and fisheries resources (e.g., destructive and unregulated fishing activities, indiscriminate disposal of waste). The project interventions directly responded to the conditions/needs of the project provinces according to the TOC for SA4. At the time of evaluation, two decrees out of 19 draft instruments introduced by the project have reached Stage 3 (authorized), seven were transitioning from Stage 1 (initiated) to Stage 2 (endorsed), and another ten reached the final review process of Stage 1, based on the project’s three-stage policy development process. Success of the SA4 project component will depend on the final adoption of the 15 policy instruments targeted by the project at the national and provincial levels. Analysis of the identified challenges suggests the need for: (i) an accessible and reliable database of existing legal and policy instruments that could expedite the review process; and (ii) a more effective coordination mechanism among government units for efficient functional relationships and review. These were evident in the delayed adoption of the local regulation (PerDa) in North Maluku and Maluku due to discrepancies between the adopted RZWP-3-K and existing national laws. It is, therefore, recommended that: (i) the project work closely with the Ministry of Home Affairs (MOHA) to help the provincial governments to prioritize the complete adoption of draft policy instruments within a timetable agreed with MMAF in the remaining project life; and (ii) the project seek MMAF leadership support for concerned MMAF units to complete the adoption of recommended instruments within an agreed schedule. EQ2: Under what conditions do people trained to manage marine and fisheries resources and/or advocate for sustainable management of these resources, as a result of USAID SEA interventions, apply their skills in marine and fisheries resources management or advocacy? Project reports showed that, from April 2016–September 2019, the project completed 74 training sessions benefitting 1,739 individuals/stakeholders at the national, provincial, district, and village levels. The training-related performance indicators indicated that the project performed well with respect to SA5.1 (number of people trained in sustainable natural resource management (NRM) and/or biodiversity conservation as a result of US Government [USG] assistance) and SA5.2 (number of innovations supported with USG assistance), registering 89% and 92% against the respective life of project (LOP) targets. In contrast, the project posted a low achievement of 4% against the LOP target for SA5.3 (number of people applying improved conservation and law enforcement practices as a result of USG assistance), although the project has started validating and updating the reports on this indicator. Current evidence indicates that a large number of trained people have applied their learnings and skills (e.g., in community advocacy, community surveillance, sustainable fishing practices, MPA management, and non-fishing livelihood activities). However, the most compelling evidence for the application of trainees’ acquired knowledge/skills is the remarkable progress that the project has made in its various x | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV SAs, TAs, and activities. Such progress would not have been possible without the application of the knowledge/skills acquired by participants in project-supported trainings. The content and process of project training activities reflect current best practices in learning/training. Attributes of the learning process ensured relevant knowledge/skills acquisition and subsequent application. The trainings included innovations designed to improve functional effectiveness and application, such as catch data collection and vessel registration, handheld devices for fisheries data collection, an MPA alert system to detect/record boat activity within the core zone of MPAs, design of Kelompok Masyarakat Pengawas or Community-based Surveillance Group (POKMASWAS) institutional development, and process for vessel registration. Other conditions, which enabled trainees to use their knowledge/skills effectively, include: (i) engagement of experienced non-government organization (NGO) partners; (ii) deployment of champions; (iii) influence and support of village, customary, and religious leaders; and (iv) national/local institutional support. Respondents, who are directly involved in law enforcement and community surveillance and who were trained under the project, identified the following constraints to the application of skills: (i) remote location of target villages; (ii) inadequacy of patrol equipment; and (iii) safety protection against violators. To address these constraints, the project helped coordinate the provision of handheld communication equipment and patrol boats by national and local governments and provided law enforcement support to POKMASWAS to enhance safety protection. The other constraints pertain to the need for: (i) adoption of remaining policy instruments to provide legitimacy to regulations and related knowledge/skills application; (ii) completion of POKMASWAS training, according to the three-stage project surveillance development program, and assisting these groups to produce post-project action plans for sustained operations; and (iii) completion of Free Trade (FT)/Marine Stewardship Council (MSC) certification process for fishers’ associations (FAs) to increase the quality, availability, and price of fishery products and increase the fishing income of FAs in target villages. Project action is needed to address these constraints in the remaining project period. EQ3. To what extent do non‐fishing livelihood activities introduced or strengthened by USAID SEA change behavior, reduce threats, and improve marine conservation (e.g., livelihood program in Galo-Galo and Morotai)? Under what conditions are these activities effective? The most common responses of FGD participants in Galo-galo village in Morotai, North Maluku to this EQ suggest that local people have complied with waste management and fishery regulations and participated in regulation development and non-fishing livelihood activities. With respect to the latter, the facilitating factors cited by these participants include the project’s training activities (SA5), regulations (SA4), marine tenure activities (SA3), and behavior change campaigns (SA1). The desk review of behavior change assessments conducted in Galo-galo and Mare in Morotai, North Maluku indicated similar progress in the number of people exhibiting positive behavior change, as evidenced by compliance with waste management and fishery regulations and their participation in livelihood activities and the maintenance of public infrastructure. The project achieved 85% of the LOP target for the SA1.1 indicator (i.e., number of people demonstrating behaviors that contribute to biodiversity conservation as a result of USG assistance). As a requirement of the project’s six-stage behavior change model, it is necessary to provide the target villages with continuing facilitation and guidance to enable them to reach the maintenance stage in order to achieve the goals of sustainable MPA and fisheries management. In Galo-galo and Mare villages in xi | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV North Maluku Province, for instance, Kelompok Sadar Wisata or community tourism group (POKDARWIS) members and other local entrepreneurs should be supported in their community businesses (e.g., tourism, salted fish production, soap and pottery making) in partnership with private companies (e.g., CV Taksa) to ensure the sustainability of the livelihood enterprises after the end of the project. They should also be provided with additional training and technical support to help improve their compliance with the fishery and MPA regulations that have been put in place with project support. EQ4: To what extent do USAID‐supported fishermen with more secure marine tenure to fisheries resources apply sustainable fishing practices and reduce destructive fishing practices (e.g., territorial use rights for fishing [TURF] in Raja Ampat and local marine management area [LMMA] in Maluku)? Under what conditions are these activities effective? In the last three years, the project has supported local governments and FAs to establish more secure marine tenure through the establishment of LMMAs and TURFs integrated into traditional sacred grounds in the project sites. These tenure arrangements were accompanied by village regulations on traditional fishing grounds, no-take zones (reserves), and coastal areas. Two performance indicators were found relevant to answer EQ4, where the project achieved 89% and 4% of the LOP targets, respectively, as of September 2019. These indicators are: (i) SA3.3: number of people with more secure tenure or managed access; and (ii) SA5.3: number of people who apply improved conservation law enforcement practices as a result of USG assistance. As cited earlier, progress on SA5.3 needs updating to reflect the evidences found in triangulated data. Both project data and MTPE evaluation findings showed an increasing number of people applying improved conservation law enforcement practices. The number has far exceeded the LOP target of 240 people for SA5.3 (the key project performance indicator relevant to EQ4) and includes 408 POKMASWAS members, 403 FA members, POKDARWIS, and other women’s groups whose members have yet to be counted and verified by the project. As in EQ3, all four project SAs (SA1, SA3, SA4, and SA5) have enabled project-supported fishers/FAs to apply improved conservation law enforcement practices in target villages with more secure tenure or managed access to fisheries resources. The fishers/FAs were ably supported by the existing local governance institutions (i.e., “tiga tungku”) in the effective application of these improved practices. These local institutions are: (i) village, customary, and religious leaders; (ii) district offices of fisheries and marine affairs (Dinas Kelautan dan Perikanan [DKP]); and (iii) POKMASWAS. As previously mentioned, the latter organization is required to complete a three-stage community surveillance capacity building program and secure protection against the unlawful entry of fishers from other villages, districts, and provinces through the adoption of relevant policies and regulations. EQ5: Given that USAID SEA addresses key marine threats and drivers through a complex project made up of multiple components/strategic approaches implemented by a large number of partners, how well are all the components/approaches working together? What aspects of overall management are working well, and what can be improved? To what extent are there capacity gaps among the partners that might be hindering effective implementation? The evaluation findings in EQ2, EQ3, and EQ4 revealed that SA1, SA3, SA4, and SA5 worked well together to achieve the specific objectives of each of the components/SAs based on their respective TOCs. For example, SA4 (provision of appropriate policies, regulations, and tools to legitimize the xii | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV activities of trained people and groups in local governments and communities) was essential for the application of skills in sustainable MPA and fisheries management. With multiple partners, a relatively large project organization, and spatially dispersed and remote target coastal villages, a strong communication and coordination system was key to the effective management of project implementation. The triangulated data showed that the management aspects, which worked well for the project within this context, include: (i) establishment of institutional mechanisms at the national and local levels (i.e., committees and task forces, joint planning workshops, etc.); (ii) strengthening of provincial fisheries co￾management committees (FCMCs) and provincial marine and fisheries agencies (DKP); (iii) socialization and capacity building processes (i.e., coaching, mentoring, apprenticeship, cross-learning visits, etc.); (iv) engagement of experienced NGO partners; (v) training and deployment of champions; and (vi) establishment of web-based and social media communication and information systems as well as and print and online publications. However, despite these various institutional mechanisms, the lack of coordination between NGO partners and concerned local government agencies, as well as among NGO partners, especially those working in the same project sites, has remained a key issue for local governments and requires improvement to maximize resource complementation and avoid duplication of effort. Other management aspects that need improvement include the requirement for progress reporting and the disbursement scheme (based on progress billing or reimbursement), which was cited by the NGO partners as a major concern. Capacity gaps were manifested in the different levels of understanding of the concepts and practices of EAFM, MPA, and MSP, which were new to most key stakeholders and which resulted in delays in the review and adoption of draft policies and regulations (EQ1). The other major capacity gap pertained to the conflicting priorities between national and local governments, which also contributed to the delayed adoption of these policies and regulations. EQ6: Based on the answers above about the conditions under which the USAID SEA Project strategies are or are not effective, what are the recommended areas (thematic and/or geographic) for expansion/contraction for USAID? The thematic areas proposed by the MTPE for strengthening or expansion include: (i) explicit incorporation of SA4 in the project’s TOC results chain for SA3 and SA5; (ii) strengthening and streamlining of the policy development process to facilitate adoption of enabling legal instruments; (iii) incentives and higher visibility of model marine stewardship efforts for wider replication; (iv) improved communication, coordination, and collaboration among implementing partners in various elements comprising the EAFM approach; (v) incorporation of ecosystem approaches in research studies; (vi) strengthening gender empowerment; and (vii) improving the dissemination of best practices. The MTPE also recommends that the SEA Project concentrate on ongoing and unfinished activities and the thematic improvements noted above, rather than expand its geographic coverage, in order to maintain the project’s focus on achieving sustainable impacts in its target areas/sites within the remaining project life. OVERALL CONCLUSIONS AND RECOMMENDATIONS Overall, the SEA Project has satisfactorily achieved its planned activities and targets and is largely on track in attaining its LOP targets as of mid-term. Of the 11 MEL performance indicators directly relevant to the MTPE, the project has already substantially exceeded three, is on track in six, and needs to xiii | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV address challenges in two to achieve the set LOP targets. The project has been largely responsive to challenges encountered in project implementation as of mid-term. The following recommendations are put forward for the consideration of USAID/ Indonesia in the design of similar projects in the future: • Implementation arrangements for a project with components that will involve national and local government should include the government agency mandated to oversee the operations of local government (in this case, MOHA), in addition to the national technical agency (in this case, MMAF), as active, co-implementing partner. • A project with complex, multiple components and which will involve multiple stakeholders, should provide adequate time for the partners to have a common understanding of the project scope, components/SAs/TAs, including their responsibilities in the project. • Good practices and key lessons from this project should be captured and compiled in a Best Practice Manual for the benefit of implementing partners/ stakeholders and future project design. • Having multiple, experienced NGO partners is an advantage, but it requires effective mechanisms for coordination, collaboration, information exchange, and lessons/ knowledge sharing. Note: USAID/Indonesia does not fully agree with all of the findings and conclusions in this report. A Statement of Difference, included as Annex A, provides commentary on areas of difference between the views of USAID/Indonesia and those of the evaluation team. 1 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 1. INTRODUCTION PROJECT BACKGROUND This mid-term performance evaluation (MTPE) of the USAID Sustainable Ecosystems Advanced (SEA) Project (AID-497‐C‐16‐00008) is aimed at an independent mid-term assessment of SEA’s achievements in relation to selected strategic approaches (SAs), including factors, conditions, and challenges that enhanced or diminished the achievement of results. With funding from the US Government (USG) in the amount of US$31,857,614, the project aims to address the key threats to Indonesia’s rich fisheries resources and marine biodiversity, viz., overfishing; destructive fishing practices; illegal, unregulated, and unreported fishing (IUUF); open access regime and weak governance of fisheries resources; and adverse climate change. Designed for implementation over a five-year period (March 2016–March 2021), the project supports the Government of Indonesia (GOI) in improving the governance of fisheries and marine resources and conserving biological diversity. More specifically, the project aims to: (i) enhance conservation and sustainable use of marine resources by reforming fisheries management and promoting marine protected areas (MPAs) to improve fisheries productivity, food and nutrition security, and sustainable livelihoods within the target areas; and (ii) strengthen the leadership role and capacity of the Ministry of Marine Affairs and Fisheries (MMAF) and local governments in promoting conservation and sustainable fishing through an ecosystem approach to fisheries management (EAFM) with strong multi￾stakeholder engagement. The implementation of this project was awarded to Tetra Tech ARD, the main contractor, in partnership with MMAF, local governments, 11 non-government organizations (NGOs), and two universities.1 The project has five components/SAs), as shown in Table 1 below, which are reflected in the project’s Theory of Change (TOC) (Annex B). The TOC is the key management tool for planning and implementation of activities to achieve the project’s goals and objectives according to the performance indicators defined in the Monitoring, Evaluation, and Learning (MEL) Plan. Table 1. The Five Components/SAs of the Project SA1. Create demand through awareness raising and advocacy Task 1. Change behaviors to increase demand for conservation and management. Task 2. Build coalitions and leadership to advocate for, and publicly monitor, transparent and accountable decision-making. SA2. Improve ecosystem management of FMA 715 and MPAs - 2a. Ecosystem Approach to Fisheries Management (EAFM) - 2b. Marine Protected Area (MPA) Management - 2c. Marine Spatial Planning (MSP) - 2d. Law Enforcement Task 3. Implement effective fisheries and marine ecosystem management in the target area. Task 4. Provide assistance to MMAF and key stakeholders on the development and implementation of an action plan to improve the national fisheries statistics and stock assessment system. 1 The implementing partners include: Asosiasi Perikanan Pole & Line dan Handline Indonesia (AP2HI), Conservation International (CI), Coral Triangle Centre (CTC), Indonesia Locally-Managed Marine Area (ILMMA) Network, Marine Change, The Nature Conservancy (TNC), RARE, Reef Check Foundation Indonesia (RCFI), Universitas Kristen Papua (UKIP), Universitas Negeri Papua (UNIPA), World Conservation Society (WCS), World Wide Fund for Nature (WWF), and Yayasan Masyarakat dan Perikanan Indonesia (MDPI). 2 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Task 5. Develop financing mechanisms to sustain efforts. SA3. Increase incentives for marine stewardship Task 7. Engage partners and leverage resources to achieve conservation, management and market reform objectives. Task 8. Increase the ancillary benefits available to stakeholders to enhance their participation and commitment to stewardship within the target area. SA4. Advance the Development of Marine and Fisheries Policies and Regulations Task 6. Facilitate dialogue on further policy, legal, and regulatory needs for the marine and fisheries sector. SA5. Institutionalize Training and Capacity Building Task 10. Design and implement training on conservation, MPAs and fisheries management, and strengthen the capacity of marine extension programs. Task 13. Build the capacity of stakeholders to assess performance and become a “learning community”. Management and Administration Task 7. Engage partners and leverage resources to achieve conservation, management and market reform objectives. Task 9. Develop and implement Feed the Future (FTF) intervention and mechanism to improve fisheries productivity. Task 11. Augment project results through the use of science, technology, and innovation. Task 12. Undertake baseline assessments to develop a contextual understanding of the target area. Task 14. Coordinate and communicate with other USAID programs, GOI programs, and other donor programs. The project operates at three levels of governance: national, fisheries management area (FMA) (FMA￾715), and provincial, with special focus on the provinces of North Maluku, Maluku, and West Papua, which lie within FMA 715 (Annex C). The activities in the first three years (FY2016–FY2019) of the project are summarized as follows. Year 1 started with baseline studies and the setting up of institutional arrangements at the national level. Year 2 focused on: (i) completion of institutional arrangements at the sub-national level; (ii) support to the development and/or adjustment of policy/regulatory frameworks, plans, models, and strategies related to the five SAs and activities; (iii) public awareness campaigns; (iv) start of capacity-building activities; and (v) development of EAFM database and data integration with the existing Pusat Riset Perikanan (PURISKAN) database. Year 3 included village-level work to develop non-fishing livelihood activities and implementation of more secure tenure or managed access to fisheries as well as continuing training and technical assistance to ongoing and planned activities at the national and provincial levels. ALIGNMENT OF SEA PROJECT OBJECTIVES AND ACTIVITIES WITH US-INDONESIA PRIORITIES The project was designed to contribute to GOI’s priorities, such as MMAF’s Ministerial Regulation No. 25/2015 and Ministerial Regulation No. 45/2015 on Strategic Planning, 2015-2019, particularly with respect to the following key performance indicators (KPIs): • KPI No. 1: Marine and fisheries community welfare index; 3 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV • KPI No. 2: Compliance of fisheries businesses with fisheries laws and regulations; • KPI No. 5: Effective management value of marine areas and sustainable fisheries; • KPI No. 12: Government policy effectiveness index; • KPI No. 13: Governance effectiveness of sustainable, fair, and competitive marine and fisheries resources utilization; and • KPI No. 15: Surveillance effectiveness at border areas. Project design was aligned with USAID’s Country Development Cooperation Strategy (CDCS) for GOI, with the goal of fostering a “self-reliant, advanced, just, and prosperous” nation. The SAs focused on the achievement of CDCS’ three intermediate results, i.e., global development priorities of mutual interest advanced, democratic governance strengthened, and collaborative achievement in science and technology increased. It also supported the Conservation of Global Biodiversity; the U.S. Presidential Initiative on Food Security, Nutrition, and Poverty; Executive Order on Combating Wildlife Trafficking; and Presidential Memorandum on Establishing a Framework to Combat Illegal, Unreported Fishing, and Seafood Fraud. 2. EVALUATION OBJECTIVES AND QUESTIONS EVALUATION OBJECTIVES The MTPE covered the implementation period, 21 March 2016 to 30 September 2019, and had the following objectives (Annex D): i. To provide a review of the USAID SEA Project’s mid-term performance and analyze the factors and conditions that enhanced or diminished the achievements of these results; ii. To identify the challenges and influencing factors of selected mid‐term results (policies/regulations, trainings, livelihoods, and marine tenure) by analyzing achievement against selected SAs and the TOC and to what extent the project has been or is able to address these challenges; iii. Based on findings in (i) and (ii), provide recommendations to improve the SEA Project’s implementation effectiveness; and iv. Based on findings in (i) and (ii), provide recommendations to USAID for future project design in building Indonesia’s capacity and ability to effectively manage its own resources. Within this context, the MTPE analyzed the performance of the USAID SEA Project at mid-term to serve as a (i) guide to project management for the remaining implementation period and (ii) reference for the design of similar future projects to be pursued by USAID in Indonesia and in other countries or by GOI (specifically MMAF) or by implementing partners. 4 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV EVALUATION QUESTIONS To achieve the evaluation objectives, the Statement of Work (SOW) for this MTPE provided the following six evaluation questions (EQs): 1. Under what conditions are the regulations and/or tools being introduced by USAID SEA to relevant authorities leading to the adoption of these policies, regulations, and tools? 2. Under what conditions do people trained to manage marine and fisheries resources and/or advocate for sustainable management of these resources, as a result of USAID SEA interventions, apply their skills in marine and fisheries resources management or advocacy? 3. To what extent do non-fishing livelihood activities introduced or strengthened by USAID SEA change behavior and reduce threats and improve marine conservation (e.g., livelihood program in Mare, Ternate and Galo-galo, Morotai)? Under what conditions are these activities effective? 4. To what extent do USAID-supported fishermen with more secure marine tenure to fisheries resources apply sustainable fishing practices and reduce destructive fishing practices (e.g., territorial use rights for fishing [TURF] in Raja Ampat and local marine management area [LMMA] in Maluku)? Under what conditions are these activities effective? 5. Given that USAID SEA addresses key marine threats and drivers through a complex project made up of multiple components/SA implemented by a large number of partners, how well are all the components/approaches working together? What aspects of overall management are working well, and what can be improved? To what extent are there capacity gaps among the partners that might be hindering effective implementation? 6. Based on the answers above about the conditions under which the USAID SEA project strategies are or are not effective, what are recommended areas (thematic and/or geographic) of expansion/contraction for USAID? 3. EVALUATION METHODS AND LIMITATIONS EVALUATION METHODS The MTPE used three data collection methods, namely: desk review, key informant interview (KII), and focus group discussion (FGD). The MTPE Team reviewed initial project documents during the inception phase, which served as the basis for determining the most appropriate and efficient data collection methods for the MTPE, taking into account the limited time available. To get responses to the six EQs from the key project stakeholders, the MTPE team developed guide questions for interviews with the lead contractor (Tetra Tech ARD); government counterparts (MMAF and Provincial Marine and Fisheries Agencies in North Maluku, Maluku, and West Papua; NGO and university partners; and selected project sites involved in non-fishing livelihood activities and more secure marine tenure. There were questions to measure the fishing community’s experiences in non-fishing livelihood activities at the site level and more secure marine tenure in Maluku, North Maluku, and West Papua. 5 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV The MTPE was conducted by Pacific Rim Innovation and Management Exponents, Inc. (PRIMEX) (Philippines) (see Annex E for the team organization, composition, and assigned tasks) from 28 October 2019 to 25 March 2020. It determined the extent by which selected mid-term results (policies/regulations, trainings, livelihoods, and marine tenure) were achieved by the USAID SEA Project and the conditions that enhanced or diminished such results. Desk Review The MTPE conducted a desk review of relevant documents from USAID/Indonesia, the project, and other key stakeholders at all levels. These were cross-referenced with information from KIIs and FGDs. The documents included the MEL Plan, work plans, and annual reports for the past three years, as well as technical studies and operational guidelines (see Annex F for the list of documents reviewed). Activity reports were reviewed to document and analyze experiences and identify data gaps in the KIIs and FGDs. The desk review helped identify the following: (i) qualitative or quantitative data indicating progress towards achieving selected mid-term results; (ii) facilitating or constraining factors; and (iii) data gaps. Key Informant Interviews KIIs were held with subjects who have first-hand knowledge and experience in the SAs and TAs under evaluation. The effectiveness of SAs was gauged through a validation of the achievements of selected SAs and the identification of (i) conditions under which the SAs were effective, (ii) factors that enhanced or diminished the achievement of selected mid-term results, and (iii) challenges to effective project implementation. Guide questions (Annex G) were designed to obtain information during the KIIs. The MTPE Team interviewed 57 key informants from 24 organizations at the national, provincial, and site levels from 13 January to 7 February 2020. The number of key informants interviewed was 84% higher than the target of 31 key informants (from 21 organizations). The number of KII sessions also increased from 31 to 39. Table 2 shows the breakdown of the 57 key informants by organization. Table 2. Number of KII Participants, by Organization, and Number of KII Sessions Participating Organizationsa Number of KII Sessions Number of Key Informants USAID 1 2 Tetra Tech 5 10 MMAF 8 9 MOHA 1 1 Private Sector - Harta Samudra in Morotai, North Maluku 1 2 Provincial Governments - North Maluku Province - Maluku Province - West Papua Province 3 3 1 3 5 2 6 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV NGO/University Partners - NGO Partners - University Partners 12 4 16 7 Total 39 57 a See Annex H for list of key informants. Focus Group Discussions The FGDs analyzed the following aspects of the SAs: (i) increasing incentives for marine stewardship through non-fishing livelihood activities and more secure marine tenure; (ii) improving ecosystem management of FMA 715 and MPAs through sustainable fisheries and LMMA establishment; and (iii) institutionalizing training and capacity building through improved conservation and law enforcement. The FGDs also documented the change in conditions of fishing villages in the selected project sites in terms of human behavior, reduction in threats to marine and fisheries resources, and improvement in conservation of biodiversity focal interests as well as non-fishing livelihood activities as a result of project interventions. Guide questions for those involved in non-fishing livelihood activities and in more secure marine tenure were designed (Annex I) and used in the FGDs. Eight FGDs were planned, with separate sessions for male and female participants in each of the three project sites (viz., Galo-galo, Morotai, Mare Island, Ternate in North Maluku; Taniwel, Seram Island in Maluku; and Waisai, Raja Ampat in West Papua). Those who joined the FGDs had similar background and direct project involvement. FGDs were not held in Mare Island and Ternate, North Maluku due to reported village conflicts during the data gathering period. Table 3 shows that only six FGD sessions were held, with a total of 59 participants (30 males and 29 females). (Details are in Annex J, and the photo documentation is in Annex K.) Table 3.FGD Participants, by Gender and Project Intervention Gender Project Interventions Total Non-Fishing Livelihoods More Secure Tenure or Managed Access to Fisheries Galo-galo, Morotai, North Maluku Taniwel, Seram Island in Maluku Waisai, Raja Ampat in West Papua Number Frequency Number Frequency Number Frequency Number Frequency Male 8 44 14 54 8 53 30 51 Female 10 56 12 46 7 47 29 49 Total 18 100 26 100 15 100 59 100 7 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV DATA ANALYSIS METHODS Processing of KII and FGD Outputs All field notes from KIIs and FGDs were translated, transcribed into English (if needed), and consolidated in the form of tables in MS Excel format. The spreadsheets showed direct verbatim quotations and translations of specific questions to help with interpretation and analysis of data. Before encoding the field notes, the MTPE team held meetings to ensure a clear understanding of the context of the answers. The encoded data were compiled by theme and analyzed for each EQ. Data Summation by Data Gathering Method The MTPE team processed and analyzed the information from each method prior to cross verification with the results from the other methods. Data analysis covered questions on the conditions and the extent to which the project components/SAs have achieved their results at mid-term toward improved ecosystem goods and services and increased benefits to communities. The analysis focused on stakeholders’ experiences in implementing the project SAs, singly and/or collectively, to get answers to the EQs. Project documents were reviewed to extract evidence of project performance and results at mid-term. Data Triangulation Triangulation was used to derive the answers to the six EQs. It confirmed or supplemented the consistency of information gathered and analyzed from the desk review, six FGD sessions, and 39 KIIs. If there was a major inconsistency in findings between these methods, the dominant response/s took precedence. The data triangulation matrix is shown in Annex L. LIMITATIONS This MTPE was not designed as a full and comprehensive evaluation; rather, it focused on six EQs, which were most fundamental to the project TOC. The assessment of selected SAs (i.e., policies/regulations, trainings, livelihoods, and marine tenure) reviewed the relevant project performance indicators indicated in the MEL Plan, examined indicators from secondary data, and validated them with the KII and FGD outputs. The MTPE scope and methodology did not allow direct validation of progress on the MEL Plan performance indicators cited in the desk review. Hence, the reliability and validity of progress data for selected indicators depended on the veracity of data contained in the project’s annual and other related reports. With the full cooperation and support of stakeholders at all levels, the MTPE field work went well, although some factors limited the conduct of the KIIs and FGDs. Some respondents were not available for the KII due to sudden tasks or more pressing needs. This meant that the MTPE team had to reset the KII, which used up more time and travel resources.2 Two FGDs were also cancelled due to reported conflicts in Mare Island. The MTPE Team expanded the time allocation for KIIs occasioned by cancellations and sought the assistance of USAID/MMAF/colleagues to firm up appointments or identify 2 Interviews with key informants from the same office were scheduled on the same day, but the plan did not materialize due to the factors cited above. As a result, the KII interviews were extended up to 7 February 2020 instead of the original completion date of 31 January 2020. 8 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV alternate respondents. Due to the cancellation of the Mare FGD, the MTPE team had to rely on desk reviews and relevant KIIs. There was a small number of KII participants from the private sector. The MTPE team found only a few private sector respondents in Harta Samudra, Morotai. The other private sector entity, A2PHI, which was scheduled for interview, was found to have stopped operating in Maluku and is maintaining offices outside Jakarta. Numerous attempts were made to conduct KIIs over the phone, but to no avail. Thus, key insights on the private sector’s participation, activities, and accomplishments were obtained from the review of the project’s annual and activity reports, which described the private sector’s involvement in the project. 4. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS EQ1: Under what conditions are the regulations and/or tools being introduced by USAID SEA to relevant authorities leading to the adoption of these policies, regulations, and tools? Findings The project introduced 19 marine and fisheries laws, policies, regulations, strategies, and plans supporting the formal adoption of its four TAs (i.e., EAFM, MPA, MSP, and law enforcement) and the improved marine and fisheries governance at the national and provincial government levels. The development of these legal, policy, regulatory, and planning instruments involves three stages: initiation (stage 1), endorsement (stage 2), and authorization3 (stage 3), which signals formal adoption for implementation. As defined in the project’s MEL Plan, “adopted” laws, policies, regulations, strategies and plans are documents that have been “officially codified or enacted by the government entity with decision￾making authority in their legal, regulatory, or policy system.” 4 As of 30 September 2019, two of the 19 draft or recommended marine and fisheries laws, policies, and regulations were authorized (Table 4). These pertain to the Governor’s decree on Rencana Zonasi Wilayah Pesisir dan Pulau-Pulau Kecil (RZWP-3-K or Marine Spatial Planning for Coastal and Small Islands), which was issued in North Maluku and Maluku provinces. These decrees were instrumental in MPA establishment in North Maluku and the managed access to fisheries resources under LLMA and TURF management Maluku (as discussed in EQ3 and EQ4). Seven proposed policies, regulations, and tools reached the process of transitioning from Stage 1 to Stage 25, while another ten reached the final review process of Stage 1. 3 Stage 1: Recommended or draft law, policy, regulation, and so on are initiated and beginning to move through a formal process for consideration, discussion, and eventual decision on the recommended draft. Stage 2: Recommended or draft laws, policies, regulations, decisions/executive orders, strategies, or guidance documents or plans from authorized bodies at national and provincial/district levels) endorsed and presented to an authorized regulatory or implementing organization for trial and further development. Stage 3 - Endorsed or accepted policies, laws, regulations, decisions/executive orders or guidance/positions from authorized decision-making bodies passed to a regulatory or implementing organization for trial, implementation or further development. Executive orders/decrees/regulations registered at the community, district municipal levels, legislation passed at the national, provincial, or district levels to ensure perpetuity of the policy. Source: USAID SEA Project (2018). Monitoring, Evaluation and Learning (MEL) Plan. Fifth Version (Updated). pp. 71-72. 4 USAID SEA Project (2018). Monitoring, Evaluation and Learning (MEL) Plan. Fifth Version (Updated), p. 71. 5 These include: (i) Amendment to Ministerial Regulation (Peraturan Menteri - PerMen) No. 48/2014 (Logbook Policy for Small Scale Fishers); 9 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Table 4. Types and Status of Laws, Policies, Regulations, Strategies, and Plans Introduced by the USAID SEA Project, 09/30/2019 Focus Area Legal, Policy, Regulatory, and Planning Instruments Level Type1 Stage2 Ecosystem Approach to Fisheries Management (EAFM) 1 Amendment of Fisheries Law (SSF Definition and other inputs) National 1 1 2 Harvest Strategies National / Provincial 4 1 3 Amendment to PerMen 48/2014, Logbook Policy for Small-scale Fishers National 2 1 Marine Protected Area (MPA) 4 Amendment of Ministerial Regulation, MMAF No. 30/2010 on the MPA Management and Zoning Plan National 3 1 5 Technical Guidelines for Ministerial Regulation, MMAF No. 13/2014 on MPA Network National 3 1 Marine Spatial Planning (MSP) 6 PerDa No. 2 on RZWP-3-K in North Maluku, 2018-2038 North Maluku 1 3 7 Governor Decree of North Maluku on the Implementation of the RZWP￾3-K PerDa North Maluku 1 1 8 PerDa No.1 on RZWP-3-K in Maluku, 2018-2038 Maluku 1 3 9 Governor Decree of Maluku on the Implementation of the RZWP-3- K PerDa Maluku 1 1 10 PerDa on RZWP-3-K in West Papua, 2019-2039 West Papua 1 1 11 Governor Decree of West Papua on the Implementation of the RZWP-3- K PerDa West Papua 1 1 Law Enforcement 12 Ministerial Regulation of MMAF on Community-based Marine and Fisheries Surveillance System National 3 1 13 Marine and Fisheries Compliance Strategy and its Management Framework for National and Provincial Level National 3 1 14 Ministerial Regulation of MMAF on Implementation of Guidelines for the Port State Measure Agreement (PSMA) in MMAF National 3 1 15 Governor Regulation of North Maluku on Combating Destructive Fishing North Maluku 3 1 16 Governor Regulation of Maluku on Combating Destructive Fishing Maluku 3 1 17 Governor Regulation of West Papua on Combating Destructive Fishing West Papua 3 1 Governance 18 RPJMD North Maluku and DKP Strategic Plan, 2019-2024 North Maluku 5 1 19 RPJMD Maluku and DKP Strategic Plan, 2019- 2024 Maluku 5 1 Source: USAID SEA Project (2019). Annual Report, October 2018 to September 2019. pp. 89-95. Note: 1 Type of document: 1. Law; 2. Policy; 3. Regulation; 4. Strategy; 5. Plan. 2 Stage of development: 1- Initiated; 2 – Endorsed; 3 - Authorized. In the project’s MEL Plan, the life of project (LOP) target under the four TAs is 15 laws, policies, regulations, strategies, and plans proposed or adopted as a result of USG assistance. These are reflected in performance indicator SA4.1 (Table 5). SA4.1 targeted five laws, policies, regulations, strategies, and plans to be authorized each year from FY2018 to FY2021. As of September 2019, only two provincial regulations have been enacted. (ii) Technical Guidelines for PerMen No. 13/2014 (on MPA Networks); (iii) PerDa Finalization for the Marine Spatial Plan for West Papua Province; (iv) MMAF Ministerial Regulation on Community-based Marine and Fisheries Surveillance Systems; (v) MMAF Ministerial Regulation on the Implementation of Guidelines for the Port State Measures Agreement (PSMA); (vi) Regional Medium-Term Development Plan (Rencana Pembangunan Jangka Menengah Daerah - RPJMD) and DKP Strategic Plan for North Maluku (2019-2024); and (vii) RPJMD and DKP Strategic Plan for Maluku (2019- 2024). 10 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Table 5. Progress on Adoption of Laws, Policies, Strategies, Plans, or Regulations, 09/30/2019 Key Indicators LOP Target (1) Actual Status (2) Change (%) (2/1) SA4.1: Number of laws, policies, strategies, plans, or regulations addressing biodiversity conservation officially proposed, or adopted as a result of USG assistance (disaggregated by type/stage) 15 2 13 Source: USAID SEA Project (2019). Annual Report, October 2018-September 2019. p. 146. Note: The two statutes that have reached Stage 3 - Authorized are the RZWP-3-K for North Maluku and Maluku. A review of the latest USAID SEA Project Quarterly Report covering the period, 01 October 2019 to 31 December 2019, showed that three laws, policies, regulations, strategies, and plans had been adopted, for an equivalent increase of one or a 20% achievement of the LOP target. The triangulated data identified the conditions that facilitated the adoption of RZWP-3-K PerDa at the provincial level and of regulations at the village and district levels. At the provincial level, the conditions include: i. National Law No. 23/2014, which requires the partner provincial governments to develop institutional capacity in planning and managing the country’s marine and fisheries and produce RZWP-3-K as a requirement under the same law; ii. Recognition of key threats to marine biodiversity and fisheries resources (i.e., destructive and unregulated fishing activities and indiscriminate disposal of wastes), which provide the primary source of income and livelihood in coastal villages; and iii. Recognition of the need for enabling policies, regulations, and strategies to solve the threats in order to conserve marine biodiversity and sustain fisheries management and support the livelihood of coastal villages within their territories in a sustained manner. At the national level, the same enabling law also served as the key facilitating condition for amendments to, and development of, laws, policies, regulations, strategies, and plans to support the transfer of mandates over marine and fisheries governance from district to provincial governments. The project came at the right time, with well-conceived SAs/TAs to take advantage of the enabling laws and respond to the pressing conditions and needs through the provision of technical assistance for: i. Reactivation and strengthening of fisheries co-management committees (FCMCs) in North Maluku6 and Maluku provinces. The FCMCs spearheaded EAFM-related activities (i.e., fishing vessel registration and spot trace deployment) with active support from the Provincial Agencies for Marine Affairs and Fisheries (e.g., DKP North Maluku); ii. Establishment of MPA working groups. These groups are made up of provincial and district government agencies, universities, and NGOs per province, which develop MPA management and zoning plans; 6 Including active engagement of Fisheries Extension Officers (Penyuluh) in EAFM activities (i.e., logbooks, vessel registration, and behavior change communication [BCC] campaigns) in North Maluku Province. 11 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV iii. Strengthening capacity of POKMASWAS (Kelompok Masyarakat Pengawas or community surveillance group), Port State Measures (PSM) Office, and judiciary institutions, among others. These offices strengthen law enforcement and marine and fisheries surveillance to increase compliance with MPA, MSP, and fisheries management policies and regulations; iv. Socialization and consultation workshops and training activities that build the capacity, confidence, and experience of agencies and partner villages to develop and adopt marine and fisheries policies, regulations, and tools; and v. Design and conduct of science-based marine and fisheries resources as well as socio-economic baseline studies to generate accurate and reliable data for policymaking, planning, and management. All these interventions were consistent with the project’s TOC for SA4.7 Although the scope of SA4.1 focused on national and provincial levels, the triangulated data also showed that marine and fisheries resources regulations were adopted and enforced at the village level. The conditions that contributed to the adoption of these regulations include the following: i. Formal recognition and engagement of village, customary, and religious leaders in the policy and lawmaking process. This facilitated the integration of customary laws (“adat”) and practices (“sasi” or respect for sacred grounds) in MPA and LMMA/TURF management and the acceptance of local people’s involvement in the project TAs; ii. Organization and training of champions, POKMASWAS, and POKDARWIS (Kelompok Sadar Wisata or community tourism groups). They volunteered to announce, popularize, and adopt relevant regulations, designated “no-take” zones, and sanctions for violators in target villages; iii. Local government assistance to adopt and enforce village regulations. The support of local government legitimized and helped the community surveillance of POKMASWAS and the fishing activities of target villages in the project sites; iv. Development of alternative non-fishing livelihood activities anchored on the practice of improved marine conservation through MPA establishment and sustainable fisheries management under LMMA and TURF arrangements for target villages; and v. Conduct of science-based research by academic and research partners, which informed the development of policies, regulations, and tools. 8 Although this was not identified by key informants as a key condition for the adoption of such instruments, the MTPE observations and desk reviews imply that these were important enhancing conditions. The research covered fish 7 USAID SEA Project (2018). Monitoring, Evaluation, and Learning (MEL) Plan. Fifth Version (Updated). p. 92. 8 For example, research findings informed policies on harvest strategy and controls for flying fish and anchovy fisheries in West Papua Province for the sustainability of both fisheries; encouraged Fakfak and South Sorong regencies to establish MPAs and develop plans for their management; and supported Raja Ampat regency and local customary groups to establish customary fisheries areas (CFAs) in Raja Ampat in FY2018. 12 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV stock assessment, marine biodiversity threat assessments, fishing effort, and other subjects for fisheries management purposes. At the start of the project, there was no common and clear understanding of EAFM, MPA, and MSP among GOI and project implementing partners at the national, provincial, district, and village levels. This led to lengthy discussions, numerous training and awareness-raising activities, and closer coordination. The reported low achievement for the SA4.1 indicator can be attributed to the following factors: i. The considerable time spent in Year 1 to support the national government (through MMAF) and provincial governments in North Maluku, Maluku and West Papua (through DKPs) in improving institutional arrangements and capacities at the provincial level (e.g., development of norms, standards, and procedures as well as the transfer of personnel, financing infrastructure, and documents to provincial governments, among others) in compliance with National Law No. 23/2014; ii. Long review period and unification of existing regulations to amend ministerial decrees (e.g., Amendment of Ministerial Regulation of MMAF no. 30/2010 on the MPA Management and Zoning Plan) to streamline MPA-related policies; iii. Conflicts in planning schedules between national and provincial offices in the development of Marine and Fisheries Compliance Strategy and its Management Framework for National and Provincial Levels9, which required the project to focus on supporting the provinces for eventual inputting into the national strategic plans; and iv. Discovery of contradictory laws at the national level, which impeded the implementation of RZWP-3-K PerDa authorized under the Governor’s decree in North Maluku and Maluku provinces. This led to a new round of dialogue between the provincial governments and relevant national agencies. The scope of the SA4.1 indicator (i.e., national and provincial levels) required strict observance of formal and rigid processes for proposing, reviewing, and adopting laws, policies, and regulations, which all took time. The triangulated data showed that specific regulations in support of EAFM, MPA, MSP, and law enforcement have been authorized and enforced at the provincial, district, and village levels. This occurred despite the relatively slow adoption process for the proposed amendment or enactment of marine and fisheries laws, policies, regulations, and tools at the national and provincial levels. There are several examples of such regulations. One, in support of EAFM, the project: (i) helped local governments hold dialogues, trainings, and outreach activities on small-scale vessel registration; (ii) helped fishers develop logbooks; (iii) conducted advance fish catch monitoring; and (iv) established LMMA and TURF￾managed areas. Two, in support of MPA management, the project provided training and TA to local governments and communities for the development of 15 MPAs in the three project provinces (six in North Maluku, five in Maluku, and four in West Papua), including promoting and supporting non-fishing livelihood activities (i.e., tourism, salted fish production, soap production, and pottery production) and private sector participation (i.e., Fair Trade for Tuna, and public-private partnership (PPP) in selected 9 MMAF aimed at finalizing its strategic plans, including the compliance strategy, by April 2020, while provincial offices aimed to complete their provincial strategic plans, including the provincial compliance strategy, by November 2019. 13 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV business enterprises). Three, the RZWP-3-K PerDa was adopted in support of MSP. Finally, in support of law enforcement, local governments and the project organized, trained, and engaged 29 POKMASWAS to carry out community surveillance and support MPA management (see further details in EQ4). Village regulations were legitimized through village decrees with the agreement of traditional and religious leaders in target villages in Maluku and West Papua provinces. District governments also ordained them through relevant decrees. However, the SA4.1 indicator only counted laws, policies, regulations, and tools adopted at the national and provincial levels. The long policy review process at the national level, as well as the conflicting planning schedules between national and provincial government offices, were the key constraints to the timely adoption of draft proposed policies, regulations, strategies, and plans. This suggests the lack of an accessible and reliable database of legal and policy instruments for quick review. Also, there is no coordination mechanism between national and provincial governments for efficient functional review relationships. This was the case in the delayed adoption of the Governor’s Decree on the Implementation of RZWP-3-K PerDa in North Maluku and Maluku, which was attributed by the KII respondents to the lengthy process of producing the adoption instruments at the national level. This suggests that the process itself is a key constraint to the adoption of policies, regulations, and tools, particularly for MPA establishment. Thus, policies, regulations, and tools enacted at the provincial level have not been supported by appropriate national decrees. Therefore, the project needs to find ways to facilitate the adoption of the eight recommended policies, regulations, and tools at the national level, as shown in Table 3. Conclusions For EQ1, the major conditions that facilitated the development and adoption of policies, regulations, and tools introduced by the project include the following: (i) National Law No. 23/2014; (ii) the key threats to marine and fisheries resources, particularly in target project sites and the need for clear laws, policies, regulations, strategies, and plans (Table 3) to address those threats; and (iii) the results of science-based research, which informed the development of policies, regulations, and tools. The project’s interventions have largely responded to these conditions and needs in line with the project’s TOC. The success of SA4 will depend on the final adoption of the remaining 15 draft or proposed laws, policies, regulations, strategies, and plans at the national and provincial levels, prior to project closing. Two key constraints need to be addressed immediately to complete the adoption and start the implementation of the remaining 15 draft or proposed policies, regulations, and tools before the project ends. First, a realistic timetable for completion of all the remaining proposed policies, regulations, and tools should be agreed between MMAF and the provincial government offices to avoid conflicts in planning or work schedules. MMAF should gather and collate all the required data/information on existing legal and planning documents and give them to the concerned task forces/working groups (WGs) prior to the scheduled consultation and planning workshops. The adoption of eight recommended policies, regulations, and tools at the national level is critical to legitimize the existing local government regulations and activities, which are being enforced in the project sites, to sustain MPA and fisheries management and protect law enforcers and POKMASWAS members from violators. Second, as local governments are under the control and supervision of MOHA under Law 23/2014, the project can negotiate with MOHA to influence the provincial governments to prioritize the adoption of draft or recommended policies, regulations, and tools, within the timetable agreed between MMAF and the project in the remaining implementation period. 14 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Recommendations It is highly recommended that the project, in collaboration with MMAF, focus its efforts on the following: i. Legitimization of MPAs, fisheries management, and law enforcement through the adoption of the remaining draft or proposed laws, policies, regulations, and strategies, and plans at the national and provincial levels in order to achieve the LOP targets of SA4.1 (and SA2.1b). This would require building consensus on priorities/schedules with MMAF and local governments (with MOHA assistance) and work on a database of instruments to facilitate the review/adoption process; ii. Enhancement of provincial, district, and village government capacity with focus on law enforcement and monitoring at the project site/target village level; and iii. Negotiations with MMAF and its Bureau of Research and Human Resources (BRSDM), as well as with DKPs, for the continuing operation of FCMCs, MPA WGs, and other EAFM and law enforcement groups for the sustainability of ongoing activities. EQ2: Under what conditions do people trained to manage marine and fisheries resources and/or advocate for sustainable management of these resources, as a result of USAID SEA interventions, apply their skills in marine and fisheries resources management (MFRM) or advocacy? Findings The project defines “people trained” as “those who have successfully completed a training course and have attended 16 or more continuous contact hours.”10 Training curriculum development and course implementation focused on sustainable natural resources management (NRM) and biodiversity conservation, directed at the project’s TAs, SA3, and marine and fisheries governance to reduce overfishing, destructive fishing, and mangrove cutting in FMA 715 in accordance with the project’s TOC for SA5.11 Along with these trainings, the project supported innovations that will enable national and local government offices, universities, NGOs, the private sector, and communities to achieve the goals of sustainable MPA and fisheries management within FMA 715. These innovations include “new” technologies, tools, practices, processes, approaches, or otherwise that can achieve significant, not incremental, improvements compared to the status quo,12 which were also integrated in the training courses and socialization for proper use in implementing TAs/SA3. The indicators relevant to trainings and to responding to EQ2, as of 30 September 2019, are presented in Table 6. Overall, the project performed well with respect to SA5.1 and SA5.2, registering 89% and 92% against the respective LOP targets. However, it posted a low achievement of 4% against the LOP 10 A standard training course includes: (i) defined learning objectives, (ii) a formal learning program (or curriculum) and (iii) educational materials. Source: USAID SEA Project (2018), Monitoring, Evaluation and Learning (MEL) Plan, Fifth Version. (Updated). p. 76. 11 USAID SEA Project (2018). Monitoring, Evaluation and Learning (MEL) Plan. Fifth Version (Updated). p. 93. 12 USAID SEA Project (2018). Monitoring, Evaluation and Learning (MEL) Plan. Fifth Version (Updated). p. 79. 15 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV target of SA5.3.13 This is due to the fact that the indicator has not yet counted or verified the number of trained MPA managers, PSM inspectors, fisheries extension officers (Penyuluh), and other law enforcers at the provincial, district, and village levels, as well as the POKMASWAS, POKDARWIS, and women community members who have applied improved MPA and fisheries resources management systems, law enforcement and surveillance devices, and alternative non-fishing livelihood activities. Private fisheries companies (e.g., PT. Harta Samudra) should also be counted for SA5.3 as they have benefited from trainings and workshops on Fair Trade (FT) and Marine Stewardship Council (MSC) certification programs, including fish trade data collection and traceability of fishery products, and have started to apply FT/MSC standards in post-harvest processing and trading. Table 6. Progress on Number of People Trained, Innovations Supported, and People Applying Improved Conservation Law Enforcement Practices, 09/30/2019 Key Indicators LOP Target (1) Actual Status (2) Change (%) (2/1) SA5.1 Number of people trained in sustainable natural resources management and/or biodiversity conservation as a result of USG assistance 2,000 1,786 89 SA5.2 Number of innovations supported with USG assistance 12 11 92 SA5.3 Number of people that apply improved conservation law enforcement practices as a result of USG assistance 240 9 4 Source: USAID SEA Project (2019). Annual Report, October 2018-September 2019. pp. 146-147. The most compelling evidence of the application of trainees’ acquired knowledge/skills is the very good progress that the project has made in its various SAs, TAs, and activities. Such progress would not have been possible without the application of knowledge/skills acquired by participants in project-supported trainings. This is shown by the MEL indicators for SA1.1, Benchmark 2: number of capacitated champions (121%, as of September 2019); SA1.1, Benchmark 4: number of forums (387%); SA3.3, people with secure marine tenure or managed access (89%); and SA4.1, number of policy instruments (2 adopted, 17 proposed). Moreover, the behavior change assessments done by the SEA Project in 2017- 2019 (see Table 9 under EQ3 below) provide evidence of knowledge/skills application by trainees at the village/target site level, notably participation in regulation development and compliance with waste management, fishing, and MPA regulations. From April 2017–September 2019, the project completed 87 training sessions,14 which benefited 1,739 individual stakeholders (1,323 males and 416 females) at the national, provincial, district, and village levels (see Annex M). Table 7 shows the summary number of trainings completed by the project.15 A large proportion of trainings was on MPA (37%) and EAFM (28%), followed by law enforcement (14.5%) and MSP (6%). Other training courses (14.5%) covered topics on communication, marine tourism, 13 In the project’s last 18-month Work Plan with updated progress on MEL indicators, the project reported achievements of 109%, 92%, and 59% for SA 5.1, 5.2, and 5.3, respectively. Source: USAID SEA Project (2019). USAID SEA Project FY20/2118-Month Work Plan (October 1, 2019 to March 22, 2021). p. 147. 14 In FY2019, the project doubled the recording of four training courses (e.g., Organizational and Bookkeeping Training, and Champion Training conducted in North Maluku on 22-23 August 2019, 26-27 August 2019, 29-30 August 2019, and 14-16 August 2019, benefitting a total of 44 individuals. Source: USAID SEA Project (2019). Annual Report, October 2018-September 2019. Annex A, pp. 121-123. 15 The slight difference in number of trained people reported in Annual Report, FY2018-2019 (Table 5) against that calculated from three Annual Reports, FY2016-2019 (Table 6) can be explained partly by the double entry of four training courses conducted in FY 2019 and partly by counting error. 16 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV fisheries product processing, food industry certification, and organizational development and bookkeeping. Most of the EAFM, MPA, and law enforcement trainings were provided to target villages (i.e., champions, POKMASWAS, POKDARWIS, fishers’ associations, women’s groups, village/customary/ religious leaders, provincial- and district-level trainers) in the project sites. Table 7. Number of Trainings Completed by the Project, April 2016-September 2019 Level Number of Training Sessions by TA EAFM MPA MSP LE Others Total National 5 6 1 2 14 North Maluku 3 7 2 5 17 Maluku 8 12 2 1 5 28 West Papua 4 4 5 2 15 Total 201 292 33 104 125 74 (+13, level of training not stated) Male 350 518 42 262 151 1,323 Female 148 130 34 27 77 416 Source: USAID SEA Project. Annual Reports FY2016-FY2019. Notes: 1 4 courses in FY1 with no indicated level of training; 2 3 courses in FY1 with no indicated level of training; 3 2 courses with no indicated level of training; 4 3 courses in FY1 with no indicated level of training; 5 1 course in FY1 with no indicated level of training. The triangulated data showed that the project applied three key concepts in designing and conducting training courses, namely: knowledge of target area, its problems, and solutions.16 These basic concepts enabled knowledge to be learned, integrated, and used. As reflected in the TOC for SA4, the project conducted a parallel assessment of the effects of managing marine and fisheries resources on threats to marine biodiversity, fisheries resources, and local livelihoods at the national and community/site levels. The results of the baseline assessment were shared with local stakeholders during socialization activities to deepen their knowledge about the existing situation and enlist their commitment to participate in the project’s training activities. The socialization process helped in developing the training curriculum and modules, which sought competency standards for MPA and fisheries management and aided the development of criteria for the selection of champions and trainers. The trained champions and trainers included local government personnel and community members who were exposed to various learning methods, i.e., coaching, mentoring, apprenticeship, cross-learning visits, competitions, games, etc. They echoed the knowledge and behavior change process at the community level. This learning process provided relevant training and served as a key enabling condition for trainees to apply skills in MPA and fisheries management. The training also introduced innovations, such as handheld devices for fisheries data collection (elaborated in collaboration with the US National Oceanic and Atmospheric Administration [NOAA]), an MPA alert system to detect and record boat activity within the core zone area of an MPA, design of institutional steps for POKMASWAS development, and vessel registration to help community surveillance, among others. The development, adoption, and enforcement of policies, 16 This statement was reached by combining/triangulating several documents: USAID SEA Project (2019). Annual Report, 01 October 2018-30; USAID SEA Project (2017). Annual Report, April 2016-September 2017. p. xii; and the responses of KIIs with the SEA Project Chief of Party (COP), Deputy Chief of Party (DCOP), MEL, Regional, and Provincial Technical Directors, and other Tetra Tech ARD technical personnel. 17 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV regulations, and tools demonstrated the extent of trainees’ application of their learnings from the courses listed in Table 7. The KII and FGD outputs revealed that other conditions facilitated the application of knowledge and skills by the trainees. These include the following: i. Engagement of partner NGOs with local experience in fisheries management and marine conservation; ii. Deployment of trained champions, who shared their learnings with friends and neighbors in villages in the local dialect and passionately advocated desired behavior changes; iii. Support provided by village, customary, and religious leaders to POKMASWAS and fishers; iv. Introduction of incentives for marine stewardship at the community level (through SA3); and v. Institutional support from national and local governments, which motivated trainees to apply their learnings on MPA and fisheries management as well as non-fishing livelihood activities. The development and strengthening of local government and community-based organizations (CBOs) contributed to the enabling conditions for providing trainees with organized institutions, where they can collectively and regularly learn, integrate, and use their knowledge and skills. This is exemplified by the POKMASWAS training and the organization and training of champions. POKMASWAS Stage 1 was delivered to eight groups, with post-training follow-up assessments conducted to measure the efficacy of the training. Concurrently in each province, support was provided for the establishment and strengthening of Coordination Fora for Fisheries Crime Handling, comprised of groups with the authority to undertake arrests/sanctions and support judicial processes related to fisheries crimes. The project also works closely with local behavior change champions or influential figures who act as conduits for messaging and promoting behavior change. The training of champions conducted by NGO partners is crucial in achieving behavior change goals. However, trainees in law enforcement and community surveillance were constrained in the use of their newly acquired skills by the remote location of the target villages, the inadequacy of patrol equipment, and the safety risks posed by violators. To address these issues, national and local governments provided the target villages with patrol boats and communication equipment (mobile phones). At the national level, and consistent with the assumptions of TOC for SA4, the project supported MMAF in the development of the MMAF’s Ministerial Regulation on Community-based Marine and Fisheries Surveillance System and Marine and Fisheries Compliance Strategy and its Management Framework for National and Provincial Level, among others, for institutional development and the protection of POKMASWAS and other law enforcers. However, these instruments have remained at Stage 1 of development, as discussed in EQ1. The application of acquired knowledge/skills cannot proceed without the enabling legal instruments, which legitimize the interventions (e.g., law enforcement, MPAs, etc.) for which people have been trained. 18 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV The USAID SEA Project has partnered with NGOs in implementing the capacity building and training under the SAs and TAs. However, as these NGOs utilize their own training manuals and protocols, there is an urgent need to develop an integrated manual, which will incorporate good practices in terms of training design, content, and techniques. The manual should include lessons learned from, and experiences gained by, the project’s NGO partners in the course of providing training under the project. Conclusions The content and process of the project’s training activities were consistent with best practices in learning and satisfactorily reached the LOP target for the SA5.1 indicator (Tables 6 and 7). The project also introduced a number of innovations, which enhanced the capacity of local governments and communities to improve MPA and fisheries management. The evaluation likewise showed that the delivery of formal training courses, socialization, and expert inputs, followed by coaching and mentoring, has been effective, and the learning process has been well appreciated by the implementing partners and villagers. The trainees have been able to apply their newly acquired knowledge/skills, facilitated by the engagement of NGO partners with experience in MPA and fisheries management, deployment of champions, influence and support of local leaders, and the support of local institutions. However, there is a risk that the success of the project’s capacity building and training activities to date could be undermined by the delayed adoption of proposed policies, regulations, and tools. For example, while the project has designed a three-stage surveillance development program for 29 POKMASWAS, not all have reached the first stage, and all have yet to complete the last two stages. This intervention is critical to advancing local surveillance skills and systems and sustaining the cooperation of local stakeholders beyond the project period. Recommendations To enhance the effectiveness of SA4 for trained people to use their skills in marine and fisheries resources management, the following are recommended: i. Ensure that all 29 POKMASWAS complete the three-stage surveillance development program and help these groups produce post-project action plans and leverage support funds. This is necessary for surveillance to be carried out by the provincial, district, and village governments and by the private fisheries companies; ii. Facilitate and support the completion of FT/MSC certification of FAs in target villages in North Maluku (e.g., 11 FAs in North Maluku, Maluku, and West Papua). This will help improve the quality and price of fishery products, ensure the sustainability of fish stocks, and increase the income of FAs from fishing; iii. Institutionalize the existing forums set up by the project and negotiate with MMAF and the provincial governments for the establishment of a permanent platform. There are four major groups which could form an MPA and fisheries management community of practice in FMA-715. These are: (i) WGs with 60 members; (ii) POKMASWAS with 29 groups and 408 members; (iii) 17 FAs with 403 members; and (iv) POKDARWIS, Adat, and Indonesian National Work Competency Standard (Standar Kompetensi Kerja Nasional Indonesia [SKKNI]), with members still to be counted by the project; and 19 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV iv. Prioritize the production of integrated training manuals based on the existing manuals, which were found effective during the first three years of the project. EQ3: To what extent do non-fishing livelihood activities introduced or strengthened by USAID SEA change behavior and reduce threats and improve marine conservation (e.g., livelihood program in Galo-galo, Morotai)? Under what conditions are these activities effective? Findings Non-fishing livelihood activities formed part of the second work stream17 by creating incentives designed to change behavior among communities in the target coastal villages in line with the goals of sustainable marine resource use and fisheries management. Behavior change is defined as, “compliance with relevant regulations (e.g., MPA and TURF regulations, including participation in community patrols) and implementation of sustainable fishing (e.g., fishermen’s ethical code of practice) to reduce threats to local biodiversity interests and improve marine conservation.” 18 The project developed key indicator SA1.1, supported by four benchmarks, to measure the outcome of selected project approaches/SAs to change behavior, reduce threats, and improve marine conservation, 19 with direct relevance to EQ3. Table 8 presents the progress on these indicators as of 30 September 2019. Table 8. Progress on Number of People Demonstrating Behavior Change, as of 09/30/2019 Key Indicators LOP Target (1) Actual Status (2) Change (%) (2/1) SA1.1: Number of people demonstrating behaviors that contribute to biodiversity conservation as a result of USG assistance 1,200 1,023 85 SA1.1a: Number of people exposed to USAID SEA Project messages through communication and media outreach 5,000 44,547 891 SA1.1b: Number of champions that have the capacity to deliver USAID SEA Project messages 400 484 121 SA1.1c: Number of people influenced by champions 1,200 642 54 SA1.1d: Number of fora established and functional 15 58 387 Source: USAID SEA Project (2019). Annual Report, October 2018-September 2019. The reported achievement of 85% for the LOP target for this indicator, with three of four benchmarks exceeding the corresponding LOP targets, as of the end of September 2019, indicates that the project’s behavior change activities have been largely successful. The project created awareness and knowledge about overfishing and destructive and illegal fishing practices, developed analytical thinking to take 17 The USAID SEA Project implemented activities to create incentives for marine stewardship through three work streams, namely: (i) developing markets for sustainable fisheries; (ii) promoting and developing sustainable marine tourism and alternative livelihood opportunities; and (iii) promoting preferential, managed, and secure access to marine resources, in accordance with the TOC. 18 See Annex 1 of the USAID SEA Project’s Initial PMI (Performance Management Indicators) Baseline Report dated 16 November 2018 (pp. 44-45) for the specific target villages and behavior change activities of the project. 19 USAID SEA Project (2018). Monitoring, Evaluation and Learning (MEL) Plan. Fifth Version (Updated). p. 37. 20 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV responsibility for solving those threats, mobilized innovators and champions, planned concrete actions, and took steps to change behavior in target villages based on the six stages of the behavior change model. 20 In FY2019, the project conducted three behavior change assessments in North Maluku and West Papua provinces. The assessments showed that some villages shifted from the “pre-contemplation” to the “contemplation” stage (i.e., Konda and Kokoda in West Papua). Others advanced to the “preparation” and “action” stages (i.e., Galo-galo and Mare villages in North Maluku, and Mayalibit Bay and Dampier Strait in West Papua) of the behavior change continuum, as summarized in Table 9. Table 9. Results of Behavior Change Assessments Conducted by USAID SEA, 2017-2019 Province & District Village/Site Study Period Focus and Findings BS AS North Maluku - Pulau Morotai Galo-galo, Morotai 2019 Galo-galo (MPA management and non-fishing livelihood activities) Focus: Household perceptions of change in the community at pre- and post-partner involvement. Findings: a) 100% increase in respondents (23 households) worked in an eco￾friendly enterprise; b) 100% increase in respondents received training to assist them in their enterprise; and c) No change (95%) in respondents involved in beach clean-ups. North Maluku - Kota Tidore Kepulauan Maregam, Mare 2018 2019 Maregam (MPA management and non-fishing livelihood activities) Focus: Methods of waste management (i.e., throwing garbage into the sea compared to burying it on land); levels of willingness to maintain public infrastructure to improve ecotourism area. Findings: a) 6% increase in respondents (94 to 100%) who proactively handled garbage on land and do not throw it into the sea; b) 100% increase in respondents (0 to 100%) who were upset when they see garbage being thrown into the sea; and c) 27% increase in respondents (73 to 100%) who felt it is a civic duty to take care of communities’ public facilities in order to promote ecotourism and community well-being. West Papua￾South Sorong Konda (2 villages) Kokoda (1 village) 2019 Konda (POKMASWAS and TURF regulations) Focus: Assessment of behavior of communities and emerging POKMASWAS members in terms of level of compliance and willingness to promote compliance with fisheries/TURF regulations. Findings: a) % awareness of what POKMASWAS is (20% in Konda; 70% in Kokoda); b) % interest and willingness to engage in surveillance activities (80%; 90%); c) % self-stated fully compliant with existing fishing regulations (64%; 69%); d) % self-stated not compliant or compliant only in the presence of others (36%; 31%); 20 Kushardanto H, K. Tjandra, R. Jakub, S.J. Campbell (2019). Progress Report: Behavior Change Practices at Mayalibit Bay and Dampier Strait. USAID SEA Project. 18 p.; USAID SEA Project (2019). Annual Report: 01 October 2018-30 September 2019. pp. 63-73. 21 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Province & District Village/Site Study Period Focus and Findings BS AS e) % willingness to promote compliance with fishing regulations (80%; 70%). West Papua - Raja Ampat Mayalibit Bay (5 villages) Dampier Strait (4 villages) 2017 2019 Mayalibit Bay (TURF regulations) Focus: Assessment of broad progress of behavior change from pre- to post-campaign on TURF regulations. Findings: Mayalibit Bay: 37% point increase, resulting in 89% compliance of respondents; and Dampier Strait: 44% point increase, resulting in 44% compliance of respondents. Note: BS = baseline study; AS = assessment study. Source of basic data: USAID SEA Project (2019). Annual Report, 01 October 2018-30 September 2019. pp. 69-72. The evaluation focused on the documentation and analysis of the characteristics of target villages using the six-stage behavior change model. Galo-galo Village was chosen as a sample area for non-fishing livelihood activities.21 The project supported marine tourism development, copra-based soap production, and salted fish production, etc., as incentives for local people to change their behavior to support MPA management. The 18 participants (8 males and 10 females) who joined two FGDs in the village in January 2020 were all members of POKDARWIS. Table 10 shows the observed changes in behavior of local people as a result of the introduction or strengthening of non-fishing livelihood activities by the project. Table 10. Evaluation Findings on Dominant Characteristics of Behavior Change in Galo-galo, North Maluku, January 2020 Characteristics Ranking of Participants’ Responses1 Commonly Observed Behavior Change Compliance with waste management regulations22 1 • Practice of proper waste disposal • Involvement in beach clean-up operations • Reduced use of plastics Compliance with MPA regulations 3 • Protecting fisheries and marine resources within the MPA • Planting of mangrove trees • Stoppage of use of corals and sand for construction Compliance with fishery regulations 2 • Catching fish only in designated fishing zones • Use of proper fishing gears/nets • Stoppage of destructive fishing methods (bombs and poisons) Participation in regulation development 2 • Involvement in formulating village regulations on MPA and sustainable fishing Participation in regulation implementation 4 • Encouraging other community members to be more organized in protecting the fisheries and marine resources 21 As noted in the limitations of this MTPE, the reported conflicts in the village in Mare Island, North Maluku at the time of field data gathering prevented the conduct of the two planned FGD sessions. 22 These regulations are legitimized through village decrees supported by traditional, customary, and village counsel, and leaders. As such, they are somewhat informal in nature and are, therefore, not counted as laws, policies, or regulations in indicator SA4.1. These include regulations on: ( i) no-take zones or Sasi area, that is, prohibition of marine or fishing activities or no marine and fishing activities on established “no-take” sasi zone; (ii) beach clean-up and sanitation, that is, regular and scheduled cleaning activities and no throwing of solid waste in the ocean; (iii) prohibition of using harmful fishing gears and other destructive practices; and (iv) many others, 22 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Characteristics Ranking of Participants’ Responses1 Commonly Observed Behavior Change Participation in non-fishing livelihood activities23 2 • Marine tourism (homestay management, tour guiding, driving boats for tourists, renting out boats for snorkeling, preparing food for tourists) (all the 18 participants are members of POKDARWIS (local tourism group) • Handicraft making (from bamboo, pandan leaves, grasses) • Soap production from copra Salted fish production Perceived benefits from MPA and non-fishing livelihood activities 1 2 3 • Increase in household income from non-fishing livelihoods • Greater incentives for local people to protect the MPA and its resources • Reduced pressure on fisheries and marine resources Note: The ranking of responses was based on the most frequent responses of the 18 participants in two FGD sessions, with first rank (1) given to the most frequent responses, and fourth rank (4) as the lowest. Multiple responses were allowed. Source: Outputs of two FGD sessions conducted on 20 January 2020. The FGD outputs showed that the most commonly observed behavior changes were related to increased compliance with waste management regulations, followed by compliance with fishery regulations, participation in regulation development, and participation in non-fishing livelihood activities. The latter reinforced the importance of creating incentives through non-fishing livelihood activities and motivated local stakeholders to support MPA management and other relevant regulations. As of 30 September 2019, the project reported an achievement of 380, or 84% of LOP target of 450 for SA3.1 (or the number of people with increased economic benefits derived from sustainable NRM and conservation as a result of USG assistance). The project also reported that efforts in livelihood activities in two villages in North Maluku (i.e., Galo-galo and Mare) “resulted in an increase in sales income to community members of more than IDR674 million (US$46,900). In addition, the initiative mobilized individual and business contributions to village welfare funds of about IDR16.8 million and led to private business investments in community members valued at IDR17.3 million (in the form of marine tourism equipment, snorkel gear, surfboards etc.).” 24 The third and fourth most commonly observed behavior changes were associated with compliance with MPA regulations and participation in regulation implementation, respectively. The analysis of FGD participants’ responses showed that most of the observed behavior changes can be attributed to project interventions, except for dried fish production, which is a traditional local activity. The low level of compliance with MPA regulations may be due to the fact that MPA management practices were introduced just recently. The low level of participation in community patrol or surveillance may be due to either or both of the following factors: (i) lack of previous training and engagement of local people in this activity; (ii) inability to participate because of other more important family obligations to fulfill, especially for women in the villages. It is also possible that the FGD output for this question may have been affected by the relatively large number of women who joined the FGD. The high level of compliance with waste management regulations, as indicated by 83% of FGD participants, is consistent with the results of the behavior change assessment conducted by the project in Galo-galo Village in FY2019 (Table 8). Similarly, all 18 FGD participants worked in different types of non-fishing livelihood activities that were introduced or strengthened by the project. This is also 23 100% of the households were involved. 24 USAID SEA Project (2019). Annual Report, October 2018-September 2019. p. 79. 23 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV consistent with the results of the earlier behavior change assessment in Galo-galo Village. One-third or more of the FGD participants identified other behavioral changes linked to compliance with fishery regulations (44%), compliance with MPA regulations (39%), and participation in regulation development (33%). Only one FGD participant noted behavior change relative to participation in the implementation of regulations. This can be explained by the fact that all FGD participants are expected to comply with relevant MPA and eco-friendly enterprise regulations as POKDARWIS members. Overall, the FGD outputs supported the results of the behavior change assessment conducted by the project. They showed that local people, particularly the 60 members of POKDARWIS, continued to advance to the “preparation” and “action” stages in relation to compliance with MPA, fishery, and waste management regulations, as well as participation in the development of regulations. The varying levels of observed compliance and participation, as implied in the available data, suggest the need for more coaching, mentoring, and confidence-building activities for local champions, regular participatory assessments, and best practice sharing among POKDARWIS and other community groups involved in MPA management and non-fishing livelihood activities. The observed behavior changes were attributed to non-fishing livelihoods (18/18), trainings (16/18), and regulations (11/18) introduced, provided, and/or supported by the project. (The numbers in parentheses indicate the proportion of FGD participants who identified the selected SAs, which served as effective enabling conditions for MPA development and management in this particular project site.) A review of project reports revealed that each NGO partner assigned to a specific project component/SA (i.e., RCFI for SA3 in Galo-galo Village) in tandem with other implementing partners (i.e., local governments and the World Conservation Society [WCS] assigned to SA1-SA5 in North Maluku) has developed and conducted a series of culturally appropriate behavior change activities25 based on the baseline assessment results. This led to the diversification and popularization of campaign materials (e.g., tote bags, t-shirts, small containers, calendars, aprons, etc.) for daily use of the audience, which served as reminders and easy-to-understand messages for local people. The campaign also included organizing or supporting community games (e.g., fishing, cooking, art and sports contests) to convey behavior change messages across a wider audience. These strategies appeared effective in villages, where people do not read much and prefer media that had practical use in their daily lives. The field staff of the project’s NGO partners succeeded in building trust and gaining the support of the local target populations at the start of their community deployment, particularly with village, customary, and religious leaders. The coaching and mentoring in post-training programs enhanced both local knowledge and confidence. At the individual level, the one-on-one outreach activities were critical to changing behaviors to align with sustainable MPA and fisheries management goals. These activities were the strategies applied by the project and its NGO partners (e.g., ILLMA, CTC, WWF, RARE, Marine Change, RFCI, MDPI, etc.) to reach individuals or small groups in the community, who do not participate or attend project orientations, meetings, or socialization and/or do not cooperate with the project and, instead, violate community policies/regulations.26 As in the other sites, the members of POKDARWIS in 25 Behavior change activities comprised four major work streams: (i) exposing target audience to USAID SEA messages through awareness raising, outreach, and socialization activities; (ii) identifying and engaging behavior change agents (or champions), including professionals, village leaders, religious leaders, traditional leaders, fishers, women, and the youth sector; (iii) training and building coalitions of champions; and (iv) establishing fora for promoting and advancing behavior change among their constituents (currently consisting of four groups: WGs, POKMASWAS, FAs and other fora, such as POKDARWIS, Adat, and Standar Kompetensi Kerja Nasional Indonesia (National Work Competency Standards or SKKNI). 26 One-on-one outreach activities are mostly conducted by champions (as post-coaching or post-training approach), community leaders 24 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Galo-galo Village valued the training activities coupled with behavior change campaigns to reach the “action” stage in the behavior change model. The FGD outputs noted several village regulations governing MPA management, sustainable fishing, and waste management, including specific rules or standard operating procedures (SOPs) for non-fishing livelihood activities (i.e., marine tourism, salted fish and soap production). The various campaign materials used to disseminate these regulations were assessed as effective, based on the self-stated behavior change identified by POKDARWIS participants. However, the level of local compliance varied with the regulations. Sample evidences of village regulations and campaign materials were provided during this evaluation. These were largely prepared and written in the local language/dialect by the project partners, while the activity reports prepared by the NGO partners contained and described the titles of the project’s annual reports, focusing on key policies and regulations that had to be initiated, endorsed, and authorized at the national and provincial levels, as discussed under EQ1. This made data validation challenging, given the limited time for field work. The desk review and KII outputs showed that local regulations on MPAs, fisheries, etc. as well as trainings and behavior change activities were key to the effective implementation of non-fishing livelihood activities to change behavior. The project used socialization involving expert inputs, consultations, coaching and mentoring, etc. and engaged champions to build the knowledge and confidence of local people. The project’s introduction of a Fair Trade Program for Tuna, with its potential to increase the selling price of tuna products and, hence, increase FAs’ income from fishing, incentivized the village FAs to adopt sustainable fishing practices. The FT program is a certification program for fishery products, which guarantees the safety of products for consumption and the availability of products for export. Champions played a critical role in helping the project improve the local understanding of the benefits from sustainable MPA and fisheries management, which could lead to higher stages of behavior change in Galo-galo Village and possibly other similar villages. Conclusions The triangulated data showed that the project has been largely effective in SA3 in terms of introducing or strengthening non-fishing livelihood activities as incentives for Galo-galo villagers to support sustainable MPA and fisheries management. Table 7 presents the high achievement of relevant indicators (SA1.1, SA1.1a–1.1d) for this component, while Tables 8 and 9 show the advancement of self-stated behavior change of local people from “pre-contemplation” to “contemplation,” “preparation,” and/or “action” stages. This positive shift can be attributed to the accumulated and combined effects of the project’s four enabling strategies (SA1, SA3, S4, and S5), which contributed to the development of knowledge, confidence, motivation, organization, and networks for Galo-galo and other villages in the project sites. However, most of the behavior change plans prepared by the NGO partners aimed to move target villages to the “preparation” and/or “action” stages by the end of their engagement due to the short implementation period of 2-3 years. In most project sites, behavior change activities started with baseline assessment, consultations, trust-building with communities, participatory preparation of (political and traditional leaders), CBOs, NGOs and NGO workers/ enumerators, and/or concerned community members. This is an approach that relies on person-to-person passing of information, which includes passing/explanation of written regulations, posters, works/activities to be done, discussion of “sasi” or no-take zone areas, logbook, life cycle of fish, motivation and encouragement to participate in the project’s community activities, etc. This process provides a sustained guidance and monitoring of the community done in a person-to-person basis in a more direct approach. 25 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV plans (including securing local support for MPA management), and identification and selection of champions. The behavior change campaigns began in the second year with the recruitment and training of more champions in order to meet campaign targets. It must be noted that the achievement of existing plan targets for behavior change activities does not guarantee that target villages will reach the “maintenance” stage in the model, where sustainability of MPA and fisheries management can be ensured. People have to see the perceived incentive benefits and behavior change results (see last row in Table 9) at the “action” and “maintenance” stages to convince them to sustain their actions beyond project life. It is essential to continue facilitation and guidance in target villages to enable them to reach the “maintenance” stage. For instance, in Galo-galo and Mare villages in North Maluku Province, POKDARWIS members and other local entrepreneurs should be supported in implementing the memorandum of understanding (MOU) they signed with a private company (CV Taksa), which has committed to support their businesses even after the project.27 It is further noted that the project TOC and relevant literature suggest that observed positive behavior changes lead to threat reduction and, thus, to improved marine biodiversity conservation. The MTPE methodology did not allow for direct field measurement of threat reduction and consequent improved biodiversity conservation directly resulting from livelihood interventions. Therefore, the MTPE had to rely on project reports/documents, through the desk review, for quantitative measures, which were validated by the qualitative responses from KIIs and FGDs. (However, quantitative assessments will be done toward the end of the project, e.g., reef fish abundance, CPUE, etc.). Recommendations Continuous learning among implementing partners and local communities, as well as targeted technical support for local communities, are critical to the success and sustainability of this particular component/SA 3. Therefore, the following actions are recommended: i. Extend the implementation period of behavior change activities and education campaigns on MPA and fishery regulations to enable the target villages to reach the “action” and “maintenance” stages of behavior change. Emphasize coaching, mentoring, and confidence￾building for local leaders and champions, POKDARWIS and POKMASWAS, and FAs, among others; ii. Provide more training for fishers in target villages (e.g., Galo-galo) on the proper use of fishing gear/net types to comply with fishery regulations. Link fisher associations with appropriate government agencies for provision of proper fishing gear/net types, among others; iii. Facilitate completion of business agreements (MOUs) between FT-certified FAs and the private sector to improve sustainable fishing and post-harvest handling and processing practices to reduce threats and improve ecosystem management of FMA 715 and MPAs; and iv. Develop or link with a more effective learning platform at the provincial level for regular participatory assessments and best practice sharing. This platform can be tapped by the existing 27 USAID SEA Project (2019). Annual Report: 01 October 2018-30 September 2019. p. 80. 26 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV fora consisting of four groups (WGs, POKMASWAS, FAs, and others such as POKDARWIS, Adat, and SKKNI) organized and supported by the project to sustain the learning process. The following actions are recommended for project sustainability and the design of similar projects: i. Ensure better documentation and sharing of livelihood activities and lessons learned, particularly in SA3, recognizing that many fishing communities are away from the mainstream of economic development and have limited livelihood options; and ii. Organize a viable national network of experts under MMAF to upgrade the manuals on integrated sustainable MPA and fisheries management as well as best practices on non-fishing alternative livelihoods, and ensure their wide dissemination through MMAF communication platforms and networks. EQ4: To what extent do USAID-supported fishermen with more secure marine tenure to fisheries resources apply sustainable fishing practices and reduce destructive fishing practices (e.g., Territorial Use Rights for Fishing [TURF] in Raja Ampat and Local Marine Management Area [LMMA] in Maluku)? Under what conditions are these activities effective? Findings Providing fishermen with more secure tenure or managed access to fisheries in target villages in Maluku and West Papua provinces is the third work stream under SA3. It involves the setting up of LMMA and TURF28 adapted to the local communities’ traditional “sasi” systems (i.e., customary-based seasonal closure of fishing area). Under LMMA and TURF arrangements, fishing grounds, no-take zones (reserves), and sasi areas are delineated and mapped to promote breeding nurseries for fish stocks, protect other important marine ecosystems and species, and ensure sustainability of fisheries and marine resources.29 This is done by the villagers with the guidance of their local leaders (village, customary, and religious) and with TA from the project. A series of activities to inform, capacitate, engage, and motivate local governments and communities to set up and carry out LMMA and TURF rules and regulations were carried out in the project sites, as shown in Table 11. Table 11. Key Activities and Regulations in LMMA and TURF Establishment/Management LMMA TURF Key activities • Outreach through socialization of LMMA management model, including discussions about resource issues and threats, and possible solutions; • Baseline assessments; • Socialization meetings; 28 In Mayalibit Bay and Dampier Strait in Raja Ampat District, West Papua Province, TURF is locally known as customary fisheries area (kawasan perikanan adat or KPA). 29 Under existing local governance systems (“tiga tungku”), “village governments, adat institutions and religious institutions are the key stakeholders to legitimize marine tenurial systems, and the District Office for Fisheries and Marine Affairs (DKP) ... has the responsibility and authority to ordain marine tenure from a government perspective, and strengthen political and societal adherence to these policies and regulations.” Source: Kushardanto H., J.T. Wibowo, R. Jakub, and S.J. Campbell (2017). Activity Report: Institutional Structures and Capacities in Dampier Strait, Raja Ampat, West Papua. 24 p. 27 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV LMMA TURF • Development of village regulations on natural resources management (Peraturan PSDA/Pengelolaan Sumber Daya Alam); • Establishment of no-take zone and participatory mapping of community traditional fishing grounds and no-take zone in villages that have developed regulations on natural resources management; • Socialization of enacted village regulations; • Monitoring of fish catch (name of fishers, duration of fishing, gear used, fishing areas, length size (in cm) and species (local and scientific names), and number of catch) • Participatory identification of threats and solutions including securing community commitments for TURF; • Participatory mapping; • Development of management plans and SOPs; • Legalization of area and management interventions; • Establishment of fishery management units (FMUs); • Fishery monitoring and evaluation (amount and type of fish caught, location of fishing activity, time of catch, and type of fishing gear used) Key regulations No-take zone (Sasi area) • People from inside and outside the village are prohibited to perform any type of fishery activity inside this area/zone Traditional fishing grounds • Prohibition on the use of environmentally unfriendly fishing gears, such as; poisonous substances (potassium cyanide, tuba roots), and nets with small mesh size; • Prohibition on destructive activities, such as; coral mining, and breaking and taking of corals; Coastal areas • Prohibition on collection of sand and stones; • Prohibition on disposal of domestic waste (or littering) • Input controls (i.e., fishing gear use and boat size); • Output controls (i.e., catch limits and fish size limits); • Temporal closures (i.e., short term closures of fishing); • Periodic closures (e.g. Sasi - long-term closures of fishery habitats); • Access restrictions on fishers entering target fisheries for protection (e.g. multi-species). Sources: ILMMA (2019). Deliverable Report 6. Capacity Building to Enhance Community Based Conservation Program through the Establishment of Locally Managed Marine Area (LMMA) in Kabupaten Maluku Tengah and Kabupaten Seram Bagian Barat; ILMMA (2019). Deliverable Report 5B: Community Organization of Locally Managed Marine Resources in Central Moluccas and West Seram Districts, April 2018-March 2019; Jakub, R., Wibowo JT, H. Kushardanto, S.J. Campbell (2017b). Management Plan and Standard Operating Procedures for the Customary Fisheries Area of Mayalibit Bay, Raja Ampat, West Papua. USAID-SEA Project, 24 p.; Kushardanto, H., A. Rukma, R. Jakub, K. Tjandra, H.A. Susanto, G.S. Arum, E. dela Rosa, W. Suheran, and S.J. Campbell (2019). Final Report: Improving Food Security and Livelihoods Through Rights-Based Management of Coastal Fisheries in West Papua. USAID SEA Project. 54 p.; USAID SEA Project (2018). Annual Reports: October 2017-September 2018; USAID SEA Project (2019). Annual Reports October 2018-September 2019. The achievement of these LMMA and TURF activities and regulations has direct relevance to two project MEL indicators as presented in Table 11. The project expected that small-scale fishers and fishing communities with more secure tenure or managed access over a given fishery, fishing ground, or territory have a greater incentive to act collectively in the sustainable management of their resources (SA3.3 indicator) and to apply improved conservation law enforcement practices as a result of USG assistance (SA5.3 indicator). Progress on SA3.3 is based on the number of people in target villages where tenure rights are documented through some form of administrative records, while progress in SA5.3 is measured based on the number of people who apply improved conservation law enforcement practices. As stated in the MEL Plan, “Improved conservation law enforcement practices include procedures, analyses, technologies, intelligence systems, or other means by which enforcement of laws that conserve biodiversity is documented or expected to be more effective and/or efficient than the status quo,” 30 as contained 30 USAID SEA Project (2018). Monitoring, Evaluation and Learning (MEL) Plan, Fifth Version (Updated). p. 82. 28 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV in village regulations (Table 10). The SA5.3 indicator includes all law enforcement personnel and units at the national, provincial, district, and community levels with direct participation in the implementation of relevant conservation policies and regulations in target villages. As shown in Table 12, the project has made good progress on SA3.3, but has lagged behind on SA5.3 in relation to the respective LOP targets, as of 30 September 2019. Table 12. Progress on Number of People who have More Secure Tenure and Apply Improved Conservation Law Enforcement Practices Indicators, as of 09/30/2019 Key Indicators LOP Target (1) Actual Status (2) Change (%) (2/1) SA3.3: Number of people with more secure tenure or managed access 6,000 5,353 89 SA5.3: Number of people that apply improved conservation law enforcement practices as a result of USG assistance 240 9 4 Source: USAID SEA Project (2019). Annual Report, October 2018-September 2019. Page 47 of this report indicates that the actual status of data for SA5.3 is subject to review due to some errors. Refer to footnote 24 for update on SA5.3. However, the project achievement for SA5.3 (Table 11) is very low and inconsistent with the activity reports prepared by the NGO partners. This implies that the number of people (or households) in target villages, who participated in the implementation of conservation activities and regulations under LMMA and TURF management, increased from 2017 to 2019, as shown in Table 13. This information was based on the results of selected assessments conducted by NGO partners in West Papua and Maluku. In Dampier Strait in Raja Ampat District, West Papua Province, the number of people who participated in CFA surveillance increased by 35.5%, based on 49 and 107 households surveyed between 2017 and 2019, respectively. This finding alone indicates that 38 households can be considered to have applied improved conservation law enforcement practices as a result of the project. In comparison, the reported decline in household participation in CFA surveillance activities in Mayalibit Bay in Raja Ampat District was attributed to the late conduct of community surveillance training activities, which took place only in the latter part of FY2019. Table 13. Results of Selected Assessments of Target Villagers Involved in Conservation Activity Implementation, FY2017–FY2019 Categories/Attributes Assessment Sites/Periods/Results West Papua Province1 Dampier Strait Raja Ampat District Mayalibit Bay Raja Ampat District 2017 2019 Change* 2017 2019 Change* Participation in CFA (Customary Fisheries Area - TURF) surveillance1 0 35.4 +/-9.2 35.5 36.7 +/- 36.4 35.9+/- 5.4 -0.7 Number of villages surveyed 4 4 5 5 Number of households in each village **277 **277 ***167 ***167 Number of households surveyed 49 107 103 102 Maluku Province2 Dusun Baru Raya**** Central Maluku District Negeri Rumah Sokat***** Central Maluku District 11/17 3/18 6/18 8/18 11/17 3/18 6/18 8/18 Number of fish catch after implementation of conservation activity 14,427 3,801 3,405 8,072 1,080 1,183 3,729 1,380 Catch per unit effort (CPUE) 0.16 0.30 0.32 0.28 0.80 0.14 0.09 0.06 29 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Categories/Attributes Assessment Sites/Periods/Results West Papua Province1 Dampier Strait Raja Ampat District Mayalibit Bay Raja Ampat District 2017 2019 Change* 2017 2019 Change* Number of total population 103 (48 males, 55 females) 568 (296 males, 272 females) Number of households 28 174 Sources:1 Data taken from Kushardanto et al. (2019). Final Report: Improving Food Security and Livelihoods Through Rights￾Based Management of Coastal Fisheries in West Papua. USAID SEA Project. 54 p. 2 Data taken from ILMMA (2019). Deliverable Report 5A: Catch Per Unit Effort (CPUE) Survey Results, November 2017, March 2018, June 2018 and August 2018 at Dusun Baru Raya, Kobisonta, Seram Utara Timur Seti and Negeri Rumah Sokat, Seram Utara.22 p. Notes: * percentage points I ** Statistics Indonesia 2017 I *** Statistics Indonesia 2016 **** Conservation activities implemented I ***** Conservation activities not yet implemented at the time of assessment In Maluku Province, a survey measured the fish catch and catch per unit effort (CPUE), among other parameters, after implementing conservation activities in Dusun Baru Raya and Negeri Rumah Sokat, both in Central Maluku District, from November 2017–August 2018. The overall results supported the assumption “that CPUE will tend to increase in areas where conservation activities are implemented and decrease in areas where conservation activities are not implemented.” 31 Conservation activities were implemented in Dusun Baru Raya, but not in Negeri Rumah Sokat. The survey aimed to understand the changes in fish catch as a result of the implementation of village regulations on NRM (Peraturan PSDA) and the establishment of a no-take zone in Dusun Baru Raya. These two project assessments indicated that local people in the target villages (i.e., Dampier Strait in West Papua and Dusun Baru Raya in Maluku) have started to actively participate in implementing village regulations on LLMA and TURF management. In addition, the current 408 members of the POKMASWAS,32 who are distributed all over the project sites, have already exceeded the LOP target of 240 for SA5.3. At the community level, POKMASWAS members serve as the “core group” that applies improved conservation law enforcement practices. It is important to note that, while POKMASWAS is the official name used for community-organized surveillance groups in LMMA and TURF areas, these groups include all other traditional and customary community groups that conduct surveillance. In Maluku and in West Papua, the existing local community groups, which maintain marine regulation surveillance, are supported by the project. Similarly, 16 of the 17 FAs, with a total of 403 members, have secured FT certification with project support. This suggests that these FT-certified FAs are applying improved conservation practices in compliance with FT standards, such as safety at sea, traceability of products, development of chain of custody (COC), etc. Even the POKDARWIS members, although focusing on marine tourism-related activities, as noted earlier, have also applied improved conservation practices for sustainable MPA management to support their enterprises. (However, the current number of POKDARWIS members has yet to be counted by the project.) In addition, some small community￾level groups have been organized and trained by the project and undertake conservation activities, although the number of data collectors in all the project sites has not yet been counted. The project also experienced a decline in the number of local people trained in enhancing community￾based conservation activities. For instance, in Maluku Tengah and Seram Bagian Barat districts in Maluku 31 ILMMA (2019). Deliverable Report 5A: Catch Per Unit Effort (CPUE) Survey Results, November 2017, March 2018, June 2018 and August 2018 at Dusun Baru Raya, Kobisonta, Seram Utara Timur Seti and Negeri Rumah Sokat, Seram Utara, p. 2. 32 USAID SEA Project (2019). Annual Report, October 2018-September 2019. p. 73. 30 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Province, the project trained 48 community members to collect data to monitor changes in fish catch due to conservation activities related to MPA and fisheries management. As of April 2019, only eight data collectors have continued work in five survey stations. The others have left for a variety of reasons, such as taking up studies in the capital city, moving out of the village and taking another job, and viewing the work as temporary.33 This indicates the need to carefully select community members for data collection and engage trained data collectors to train “second-liners.” Fishers in Taniwel and Waisai villages located in Maluku and West Papua provinces, respectively, have been involved in LMMA and TURF management. They joined trainings on the setting up of managed access to fisheries resources, sustainable fishing methods, fish measurement, development of village regulations, and community surveillance, among others (Table 14). These villages also developed and enforced village regulations governing LLMA and TURF-managed access areas. This evaluation conducted FGDs with 41 local residents (26 in Taniwel Village and 15 in Waisai Village) to determine the extent of application of improved conservation law enforcement practices in compliance with village regulations (Table 11) and the conditions under which these practices are effective based on their experiences and observations. Table 14. Evaluation Findings on Sustainable Fishing Methods Applied by Fishers in Target Villages with More Secure Marine Tenure, Maluku and West Papua, March 2020 Categories / Attributes Marine Tenure/Project Sites LMMA TURF Taniwel, West Seram, Maluku Waisai, Raja Ampat, West Papua Ranking of Participants’ Responses1 Ranking of Participants’ Responses1 Sustainable fishing methods • Use proper fishing gears/nets • Catch fish in designated fishing zones 1 2 1 2 Reduction of destructive fishing practices • Stop to the use of bombs, poisons, etc. • Participation in community surveillance 1 2 1 2 Protection of coastal areas • Participation in beach clean-up operations 1 Relevant trainings attended • Establishment of managed access to fisheries resources (LMMA) • Development of village regulations • Sustainable fishing methods • Community surveillance • Fish measurement 1 2 1 2 3 33 ILMMA (2019). Deliverable Report 6: Capacity Building to Enhance Community-Based Conservation Program through the Establishment of Locally Managed Marine Area (LMMA) in Kabupaten Maluku Tengah and Kabupaten Seram Bagian Barat. p. 10. 31 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Categories / Attributes Marine Tenure/Project Sites LMMA TURF Taniwel, West Seram, Maluku Waisai, Raja Ampat, West Papua Ranking of Participants’ Responses1 Ranking of Participants’ Responses1 Changes observed in coastal and marine resources • Reappearance of fish species in no-take (sasi area) zone in increasing numbers and sizes • Appearance of more birds near the seashore • Recovery of corals • Octopus observed swimming near the seashore 1 2 3 3 1 Number of FGD participants 26 (14 males, 12 females) 15 (8 males, 7 females) Note: 1 Ranking of responses was based on the most frequent responses of the 18 participants in two FGD sessions, with first rank (1) having the most frequent responses, and third rank (3), the lowest. Multiple responses were allowed. Source: Outputs of two FGD sessions in Taniwel, North Maluku conducted on 25 January 2020, and another two FGD sessions in Waisai, West Papua on 22 January 2020. All 41 participants claimed that local fishers and other residents have used sustainable fishing methods, reduced destructive fishing practices, and protected coastal areas in their villages. Table 14 shows that the use of proper fishing gears/nets was the most common response for sustainable fishing method used in both villages. Fishers also generally caught fish in designated fishing zones. In Taniwel Village, the male participants (six POKMASWAS members, six village government representatives, one religious, and one youth representative) said that fishers have limited their fishing activities to the designated fishing zone. Of the 12 female participants from the Pemberdayaan Kesejahteraan Keluarga (PKK) (Family Welfare Guidance Program), 11 said that fishers now use proper fishing gears/nets. The male responses can be explained by the fact that POKMASWAS members focus on community surveillance work and, as such, can easily observe the compliance of fishers in the designated fishing zone. The female participants also said that they have joined beach clean-up activities. In Waisai Village, both male and female participants from the FA and PKK reported that fishers have used proper fishing gears/nets and have been fishing only in designated fishing zones. In both villages, male participants claimed that fishers have discontinued the use of destructive fishing practices (i.e., bombs, poison, etc.) and have joined community surveillance work. KII and FGD results revealed that during the first few months of implementation, there was outright non-acceptance and doubt among the villagers. It took several months of socialization, convincing, and community organizing, adapting customary knowledge, and combining these with the presentation of scientific evidence, before the local communities understood the benefits of non-destructive, sustainable fishing practices. And as village regulations, policies, and tools were implemented and as the local communities saw the results of their efforts and shared these with other communities, the project gained wider acceptance among the local population in the project areas. The lack of communication equipment was identified by the male participants in Waisai Village as a major constraint to community surveillance, rendering their work only partly effective. Community surveillance also includes the protection of the no-take zone, along with billboards or posters with the village regulations in this zone. The selected project assessment results in Table 13 clearly show that more fishers have adopted improved conservation practices related to sustainable fishing methods and 32 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV reduction of destructive fishing practices. At least 14 fisher-participants from both villages have used one or more of the improved conservation practices highlighted in the shaded rows in Table 14. The two villages studied in Maluku and West Papua provinces demonstrated improved conservation practices applied by fishers in LMMA and TURF covered areas, respectively, to support sustainable fishing and reduce destructive fishing. In community surveillance, the POKMASWAS members, who joined the FGD sessions, raised the need for communication equipment to make their work more effective. In Waisai Village, the unlawful entry of fishers from other villages has not been totally prevented. In response, some POKMASWAS members said that they extended the socialization of their TURF regulations to other neighboring villages. In Taniwel Village, the male participants suggested the provision of training on sustainable fishing methods and fishing equipment/tools (e.g., trolling rods) to fishers so they could apply improved conservation practices. In the two villages, 14 of the 41 FGD participants (34%) said they have applied one or more improved conservation practices (Table 14). With respect to enabling conditions, the triangulated data showed that all four project SAs (SA1, SA3, SA4, and SA5) contributed to the application of improved conservation law enforcement practices. However, it is critical for local community leaders and the district DKP to “legitimize marine tenure systems and to ordain marine tenure from a government perspective, and strengthen political and societal adherence”34 to ensure that improved conservation practices are applied in the target villages. These conservation practices involve LMMA and TURF management incorporating traditional tenure rights (sasi systems), particularly in Taniwel and Saiwai villages. In most target villages, adat institutions play a key role in the integration of traditional tenure rights in LMMA and TURF arrangements and the eventual acceptance of village regulations by local people, particularly those belonging to tribal groups. In addition, the organization and training of POKMASWAS has resulted in the establishment of a functional core community conservation group in each target village/project site, which is tasked to conduct community surveillance and enforcement of relevant village regulations in support of sustainable fishing in LMMA, TURF, and MPA areas. Conclusions The evaluation found evidence of the increasing number of people applying improved conservation law enforcement practices, far exceeding the LOP target of 240 people for SA5.3, the indicator relevant to EQ4. As a major output and contribution of USAID SEA, all the 408 POKMASWAS members were reported by the project, in September 2019, to be willing to serve as law enforcement personnel and fulfil the requirements of SA5.3 according to the MEL Plan indicator definition stated above. Members of other CBOs, such as POKDARWIS and FT-certified FAs, also apply improved conservation practices and regulations and can qualify for the definition of SA5.3, The same can be said of local champions who have applied or advocated for improved conservation law enforcement practices in their daily work. Like in EQ3, all the four enabling SAs of the project contributed to the application of improved conservation law enforcement practices in target villages with more secure tenure or managed access to fisheries resources. However, it is the existing local governance institutions (i.e.,“tiga tungku”) – local leaders, DKP, and POKMASWAS – that have played a critical role in the effective adoption of these 34 See footnote 23. 33 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV practices. While the POKMASWAS began to carry out community patrol and surveillance in LMMA/TURF areas and MPAs, they have completed only the first module (initiation stage) of the three￾stage capacity building program for POKMASWAS provided by the project. Village regulations passed by the village and district governments have served their purpose in the target villages, but they will require formal recognition at the provincial and national levels for the protection of the POKMASWAS. Recommendations In the remaining implementation period, it is recommended that the USAID SEA Project: i. Intensify socialization and outreach activities on sustainable fisheries management options (i.e., catch quotas, closed seasons, and other management controls) to inform and engage more fishers to apply these improved conservation practices; ii. Address the needs of local fishers and POKMASWAS in Taniwel Village for equipment/tools (e.g., fishing rods) and training on sustainable fishing methods, as well as the needs of fishers in Waisai Village for communication equipment for community surveillance, by linking them with government, the private sector, and NGO partners; iii. Secure the legitimization of MPA, LMMA, and TURF authorities at the provincial and national levels (also recommended in EQ1); iv. Develop and establish more permanent institutional arrangements, including regular budget allocations, with district and village governments for continued operation of POKMASWAS in target villages/project sites after the project implementation period; and v. Upgrade existing manuals and package them into Best Practice manuals to include improved conservation law enforcement practices as contained in the Management Plans and Standard Operating Procedures (SOPs) for managed access to fisheries resources, as well as assessment reports prepared by the project and its partners. EQ5: Given that USAID SEA addresses key marine threats and drivers through a complex project comprised of multiple components/SAs implemented by a large number of partners, how well are all the components/SAs working together? What aspects of overall management are working well, and what can be improved? To what extent are there capacity gaps among the partners that might be hindering effective implementation? Findings As designed, the project has five components/SAs and 14 major tasks and targets 3 provinces, 13 districts, and numerous coastal villages within FMA 715. The project partnered with: (i) national and provincial governments under the leadership of MMAF and coordination with MOHA and BAPPENAS; (ii) several government offices at the national, provincial, district, and village levels; and (iii) 13 NGOs/universities for implementation of project components/SAs. The project also established and 34 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV strengthened institutional mechanisms and communication/ advocacy activities together with experts and partners at the national and local levels. With multiple partners having distinct legal mandates and priorities in marine and fisheries governance and management and covering spatially dispersed target villages, the evaluation found that project implementation is more complex and challenging than anticipated during project design, as elaborated below. Effectiveness of the Project Components/Approaches Working Together The first four EQs concentrated on the three project components/SAs (i.e., SA4, SA5, and SA3), and the TOC of each of these SAs assumed the support of other SAs to achieve its specific objectives, leading to the attainment of the overall project goal. Figure 1 presents the functional linkages of the project components/SAs and the specific objectives as postulated in the respective TOCs according to the MEL Plan. The evaluation findings discussed in EQ1–EQ4 showed that the assumed functional linkages among the project components/SAs remain valid and have worked well together, especially in SA4. In SA5 and SA3, the evaluation revealed the critical role of SA4 in providing policies, regulations, and tools that enabled trained people in local governments and communities to apply their newly acquired knowledge and skills in sustainable MPA/fisheries management and non-fishing livelihood activities to achieve the respective objectives of SA3 and SA5 (Fig. 1). In reality, SA4 activities support SA5 and SA2, but such activities were not reflected in the TOCs of the latter two components/SAs, showing the importance of the project components/SAs working well together. Figure 1. Simplified TOCs of SA4, SA5, and SA3, and their Functional Linkages Note: Geographical focus (North Maluku, Maluku and West Papua) of specific SAs indicated in boxes is based on the MEL Plan. In general, the outputs/outcomes of each component/SA could not be achieved without the support of other SAs, including SA1, since the components/SAs/TAs at the project start were all new to the target local government personnel and coastal villagers. Thus, all the project components/SAs had to be 35 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV introduced as a package to enable the target groups to go through the behavior change process while building their capacity, developing regulations and tools, creating incentives, and applying knowledge/skills according to the activities planned for each target area (MPA or FMA) and group (local government personnel and community members). These processes and activities were documented by the project’s NGO partners in MPA establishment, MSP implementation, and LMMA/TURF management. However, the current process of adopting policies and regulations at the national level contributed to the delayed implementation of planned activities in SA5, SA3, and SA4. As the project progressed, policies and regulations were drafted, simultaneously at the national and local levels–a clear deviation from the normal government practice of initiating or drafting recommended policies and regulations at the village level for review and refinement at the district and provincial government and endorsement to the national government (MMAF) for approval/adoption. Management Aspects that Work Well or Need Improvement With multiple partners, a relatively large project organization (259 total staff as of 30 September 2019), and spatially dispersed and remote target coastal villages, it is essential for the project to have a strong communication and coordination system for effective project management. Based on the analysis of triangulated data, the following management measures appear to have worked well for the project: i. Establishment of institutional mechanisms at the national and local levels (i.e., committees and task forces, joint planning workshops, including the fora and community groups like FT-certified FAs, POKDARWIS, POKMASWAS, and women’s groups, and deployment of regional and provincial coordinators), which supported government offices in performing their tasks; ii. Strengthening of FCMCs and DKPs at the provincial level, which spearheaded planning and implementation of EAFM- and MSP-related activities; iii. Socialization and capacity building processes, which contributed to building desired behavior, capacity, and confidence of trainees to apply their acquired knowledge/ skills and adopt policies and regulations; iv. Engagement of NGO partners with working experience in fisheries management and marine conservation, which enabled them to “hit the ground running” and initiate project activities after just a short preparation period; v. Training and deployment of local champions, who played a critical role in socializing and sustaining behavior change at the community/village level; and vi. Setting up of web-based and social media communication and information platforms, as well as print and online publications, for the project’s behavior change campaign and capacity building activities. The above mechanisms and systems have resulted, among others, in functional relationships between and among government agencies at all levels and increased interaction between project partners and target villages; improving the database and capacity for EAFP, MPA, MSP, and law enforcement (reportedly “weak” prior to the project); completing RZWP-3-K; and synchronizing government planning processes. 36 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV The findings of the desk review and KIIs confirmed that the established mechanisms and systems have largely worked well. Some of the more specific management strategies that helped in facilitating project implementation include regular interactions among partners, person-to-person interactions with target groups, and scheduled work meetings to build common understanding, secure commitment, and achieve the objectives of planned activities. These types of project support were appreciated by the local governments as they helped their personnel develop and apply MPA and fisheries regulations (as in West Papua) and motivated them to take responsibility for encouraging communities to practice sustainable fishing methods and protect coral reefs (as in North Maluku). However, despite various institutional mechanisms, the interaction between and among NGO partners and between the NGOs and concerned government agencies, the NGOs have identified some issues related to their government engagement, which need to be resolved or improved upon. The project requires each NGO partner to focus on a specific scope of work and deliver certain outputs according to an agreed schedule. As such, they have been hard-pressed by set deadlines to meet with government counterparts and other NGO partners working in the same project sites, leading some to “work independently.” The NGOs find the reporting requirements of the project (by activity, monthly, quarterly, and annual) too frequent, cumbersome, and time consuming, especially considering that they need to spend a lot of time with the local governments and communities. They also expressed concern regarding the current disbursement scheme, based on progress billings or reimbursement, which has caused delays in the implementation of planned activities. Extent of Capacity Gaps among Partners Hindering Effective Implementation The project’s Third Annual Report, FY2018-2019 highlighted the nature and extent of current capacity gaps, which were largely confirmed by the KII outputs: “Government partner capacity continues to be a challenge across many of the strategic and technical approaches of the USAID SEA Project. Over the reporting period, limitations to government staff capacity presented challenges to advancing EAFM-related work, slowed progress in some of the MPA sites, and in policy development processes. In addition to capacity, the availability of government partners is often challenging, and waiting for available time periods can cause delays.”35 At the national level, the project identified “a wide variation of understanding about the laws and the technical language” among government staff and stakeholders, requiring greater effort to provide an adequate background on technical and legal aspects of draft or recommended policies to progress discussions based on a common understanding.36 At the provincial level, the project identified the lack of awareness of national guidelines for MPA planning and management among provincial government staff and the limited institutional capacity (i.e., personnel, offices, and other resources) for on-ground MPA establishment and management as constraints.37 While the project supported the provincial governments in the development of technical guidance for MPA management and provided training for provincial staff, continued project engagement with these offices/staff is required to encourage the adoption of standard MPA planning and management practices. At the district and village levels, where 35 USAID SEA Project (2019). Annual Report, October 2018-September 2019. pp. 117-120. 36 Ibid., p. 119. 37 Ibid., p. 118. 37 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV actual MPA management takes place, both the district government and local communities require continued capacity development and technical guidance on MPA management to support the provincial government and strengthen the coordination between district and provincial governments. The KII outputs confirmed the veracity of the content of project reports on key local government informants, who averred that their offices had limited capacity in EAFM, MPA, and MSP prior to the project. This evaluation identified the need for more specialized training and capacity building activities for provincial and district governments on the project’s TAs (EAFM, MPA, MSP, and law enforcement) and marine tourism in order to develop staff knowledge and confidence in sustaining these initiatives beyond the project period. Most key informants also stressed the need for continued training and coordination support. Examples of training needs of government stakeholders include MPA planning and management and facilitation skills in MPA establishment (provincial government), inter-agency coordination skills (provincial and district governments), marine tourism development (MMAF), and partnership building for marine and fisheries management (academe). Conclusions Synergy of project components/SAs. The findings in EQ1–EQ4 indicate that the project components/SAs worked well together, especially since these components/SAs/TAs were regarded as “new concepts” by the provincial and village governments in North Maluku, Maluku, and West Papua at the start of the project. The local governments needed the knowledge and skills, policies and regulations, planning and management tools, incentives for marine stewardship, and behavior change strategies for them to lead and steer the communities in their respective jurisdictions to practice sustainable MPA and fisheries management. Project experience in the last three years (FY2016–FY2019) confirmed the need for the integrated implementation of SA1, SA3, SA4, and SA5 for their respective objectives and the overall project goals to be achieved (see Fig. 1). Management aspects. The institutional mechanisms established by the project partners helped in improving coordination, communication, and learning processes, especially between national and local governments and between local governments and the local communities in the target villages. Other management measures, which facilitated effective project implementation, include the strengthening of local government offices and working committees/task forces, engagement of experienced NGO partners, deployment of champions, and socialization and capacity building. The need for stronger coordination among partners and between partners and target villages emerged as the main management issue that needs improvement. This will require the partners, local governments, and target villages to commit to regular coordination and consistent monitoring of the process and results of such coordination. This may be done through formal agreements to provide timely support to the target groups so that they can apply their acquired knowledge and skills and comply with adopted policies, regulations, and tools. Capacity gaps. It is necessary to provide specialized training for target groups at MMAF and DKPs in the remaining project period to facilitate the capacity building of concerned staff and the adoption of draft or recommended policies and regulations and sustain the implementation of project activities at the ground level. The trainings also need to focus on key technical and legal aspects in order to facilitate the adoption of proposed/draft legal instruments by the mandated agency/ies. The areas for training and capacity building include the following: (i) EAFM (e.g., concept/elements, FMA 715 council support, harvest control rules/measures, plans/budgets integration, FCMC strengthening); (ii) MPA management 38 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV (e.g., concepts/elements/ networks, management body formation, regulation development, budget integration); (iii) MSP (e.g., MSP MEL plan, SEANODE geoportal use); (iv) law enforcement (e.g., completion of POKMASWAS training, strengthening of collaboration with mandated law enforcement agencies); and (v) marine tourism (e.g., concept/elements, private sector linkages, marketing improvement). The identification of training courses should be done in close consultation with the concerned national and provincial project partners to ensure that the courses are responsive to their needs. The KII responses indicate that behavior change and the effects of capacity building may take longer than expected (i.e., 2-3 years). Recommendations Regardless of the overall satisfactory performance in project management and synergy, the following recommendations are made to address persistent capacity and management gaps: i. Enhance existing coordination and management platforms at the national and provincial levels. Establish formal institutional arrangements directing concerned local government offices to regularly meet and continue learning sessions with key government and non-government partners and stakeholders, who are expected to continue interventions at the local/site, provincial, and national levels. This can be done through an agreement among partners and villages or an administrative circular; ii. Conduct refresher courses for key government offices at the national and local levels on targeted capacity gaps. This will address differences in the level of understanding of concepts and practices on EAFM, MPA, MSP, and law enforcement; and iii. Develop and package a “legacy conservation model,” as suggested by one MMAF key informant. This model will showcase the effective and successful approaches, strategies, structures, and systems for fisheries management and marine conservation in Indonesia based on recent scientific and technical innovations that have been tested and applied by USAID SEA. EQ6: Based on the answers above about the conditions under which the USAID SEA project strategies are or are not effective, what are the recommended areas (thematic and/or geographic) for expansion/ contraction for USAID? Recommended Thematic Areas The project components/SAs/TAs were all found effective in improving fisheries management and conserving marine biodiversity in the context of FMA 715. The combined implementation of these SAs/TAs has produced concrete proofs of the desired results of working well together. It has led to improved fisheries, enhanced marine biodiversity, and early benefits for fishing villages in the project sites. On the other hand, the lack of legitimacy of MPA development and LMMA/TURF areas resulting from the delayed adoption of draft or recommended policies and regulations could reduce the potential sustainability of the early project benefits and gains. 39 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Recommendations Based on the evaluation findings, the following areas are recommended for expansion in the thematic areas: i. TOC results chain. Enhance the TOC for SA3 and SA5 by integrating SA4 in the results chain. This is needed to legitimize and protect the community surveillance activities of POKMASWAS and other law enforcement groups and support the certification of the marine extension program and other training courses on MPA, EAFM, and law enforcement. ii. Policy development process. Under this approach, a draft policy or regulation initiated by the village government is sent to the district government for review and validation and endorsement to the provincial government for alignment check with regional/national laws and policies, and finally to the national government (MMAF) for approval/adoption. iii. Incentives for marine stewardship. Give due recognition to villages that have been successful in their non-fishing livelihood activities and/or in securing marine tenure to serve as models for other villages to emulate. In this context, monitoring and evaluation (M&E) of components/SAs should also cover socio-economic development. iv. EAFM. Increase the frequency and effectiveness of coordination and communication between target villages and provincial and district governments. For example, in vessel registration, MPA and TURF management, and community surveillance, it is necessary to implement continuing behavior change communication campaigns and education activities targeting local communities/villages to ensure that they have a clear and full understanding of the procedures for, and benefits from, these interventions to ensure compliance. v. Research coverage. Complement research studies with ecosystem-level approaches, rather than on single species (flying fish, shrimps, anchovy, etc.), consistent with the project’s approach. Some key informants proposed research studies involving inter-regional connectivity of marine species, such as shrimp stocks in South Sorong, Bintuni, and Sorong to generate data for joint or coordinated stock management between and among these districts. vi. Gender equality. Improve local governments’ capacity to include gender and women empowerment in fisheries management and marine conservation through greater participation of women in planning and implementation of village plans and regulations, taking a stronger role in CBOs (e.g., POKDARWIS), and providing budgetary allocations for gender empowerment and development in provincial development plans; and vii. Best practice sharing. Encourage and document the establishment of more public-private partnerships (PPPs) at the local government and community levels for non-fishing livelihood activities and more secure marine tenure efforts using the “homestay management” experience in Taniwel and MDPI’s FT experience, among others, as models. 40 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Recommended Geographic Areas Key informants from national and provincial governments have recommended expanding project coverage to additional districts/sub-districts in the provinces of North Maluku, Maluku, and West Papua as well as to the three other provinces bordering FMA 715. This is in view of the “unit stock concept” and scale requirement in fisheries management, which dictates the full coverage of FMA 715 and adjacent areas for benefits to be realized. However, the evaluation findings identified capacity and coordination gaps among government agencies at all levels and weak interaction between local governments and local communities in target villages. Relevant policies, regulations, and tools at the national and provincial levels have not yet been fully adopted, and the FT certification of FAs and the signing of business agreements with private fisheries enterprises are yet to be completed. Therefore, it is recommended that, in the remaining project period, focus should be on these ongoing yet unfinished activities in order to achieve the project’s overall conservation and development goals. This will lead to a greater and lasting impact on the target groups and villages, which is far better than expanding geographic coverage but losing focus along the way. 5. GENERAL CONCLUSIONS AND RECOMMENDATIONS OVERALL PROJECT PERFORMANCE The overall evaluation findings showed that the project has satisfactorily achieved the planned activities and targets at mid-term. Of 11 project performance indicators, the evaluation found nine to be relevant, and the indicators were either exceeded (SA1.1a, SA1.1b, and SA1.1d) or on track (SA1.1, SA1.1c, SA3.1, SA3.3, SA5.1 and SA5.2) with their respective LOP targets. For the other two indicators (SA4.1 and SA5.3), the project showed sufficient evidence, as confirmed by FGD and KII outputs, that the LOP target for SA5.3 could be achieved before the end of the project. However, it is not likely that the project will meet the LOP target for SA4.1, with only 2 of 19 draft or recommended laws, policies, regulations and tools having been adopted at mid-term. This is due to the following factors: (i) long process of policy review and adoption and differences in level of understanding among framers or reviewers of draft or recommended instruments at the national level; (ii) conflicting planning schedules between national and provincial government offices; and (iii) discrepancies between the adopted RZWP￾3-K PerDa and existing national laws. The evaluation findings from selected mid-term results indicate that the project performed well in the design and conduct of trainings (SA5) and the introduction of incentives for marine stewardship (SA3), as discussed in EQ2, EQ3, and EQ4, respectively, but it lagged behind the LOP targets for policy/regulation adoption (SA4) due to the challenges mentioned above. The conditions that enhanced or diminished the achievement of results differed according to specific SAs, as discussed above and summarized below, with key actions taken by the project and its partners. 41 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV SAs Enhancing Factors Diminishing Factors SA4: Policies and regulations • Compliance with Law 23/2014 (Local Government Law) • Key threats to marine biodiversity and fisheries resources • Need for enabling policies, regulations. and tools to address the threats • Long process of policy review and adoption • Differences in level of understanding among policy framers or reviewers • Conflicting schedules between national and local government agencies • Discrepancies between adopted RZWP-3-K (provincial level) and existing national laws Actions taken: Continued work with MMAF and provincial governments and further discussions to provide a comprehensive background on legal and technical aspects and achieve a common understanding. SA5: Training • Best practice attributes of the project’s learning process • Engagement of experienced NGO partners • Training and deployment of champions and trainers • Development/strengthening of local government and community-based organizations • Influence and support of village leaders, customary leaders and religious leaders • Institutional support from national and local governments • Remote location of target villages • Inadequacy of patrol equipment • Safety risk of law enforcers and POKMASWAS members (vs. violators) Action taken: Provision of patrol boats and handheld radios/cellphones for community surveillance by national and local governments SA3: Livelihoods Extent of behavior change (SA1): • Compliance with waste management regulations (83%) • Compliance with fishery regulations (44%) • Compliance with MPA regulations (39%) • Participation in regulation development (33%) Facilitating conditions: • Trainings • Regulations • Behavior change campaigns • Non-fishing livelihoods • Deployment of champions • Limited fishing gear, as required in fishery regulations • Lack of knowledge on proper application of fishing gear • Local people, especially the women, in the target villages have important family obligations to fulfill Actions taken: Sustained socialization and training activities, and linking POKDARWIS with private sector partners for selected non-fishing livelihood activities. SA3: Marine tenure Extent of number of people applying sustainable fishing practices • 408 POKMASWAS members involved in community surveillance • 403 FT-certified FA members • 484 local champions • POKDARWIS members (number of members yet to be counted by the project) Facilitating conditions: • Trainings • Regulations • Behavior change campaigns • Secure marine tenure (LLMA, TURF) • Influence of village/ customary/religious leaders • Support from DKPs • Inadequacy of patrol equipment • Safety of law enforcers and POKMASWAS members against violators Actions taken: Provision of patrol boats and handheld radios/cellphones for community surveillance, and facilitation of FT certification of 16 FAs by the national and local governments. 42 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV CHALLENGES AND INFLUENCING FACTORS FOR SELECTED MID-TERM RESULTS From the table above and discussions under EQ1–EQ5, the diminishing factors associated with SA4 served as the major challenges to the attainment of positive mid-term results. The success and sustainability of the other project components/SAs (i.e., SA2, SA3, and SA5) will depend on the adoption of the remaining draft or recommended policies, regulations, and tools. The project clearly recognized that policy development in the current situation requires more time (and budget) for wider discussions, both at the national and provincial levels. As local governments are under the control and supervision of MOHA under Law 23/2014, the project can negotiate with MOHA to influence DKPs in North Maluku, Maluku, and West Papua provinces to prioritize the adoption of the draft or recommended policies, regulations, and tools according to a timetable that will be agreed between MMAF and the project within the remaining project period. For the other mid-term results (i.e., trainings, livelihoods, and marine tenure), the challenges are technical/capacity and logistical in nature and could be resolved before the end of the project. The major challenges to the attainment of selected mid-term results vis-à-vis the respective TOCs are described below. • SA4: Policies and regulations. The establishment of legal frameworks for EAFM, MPA, and MSP was set as the first major result in the TOC. It is the key input for improved cross-cutting knowledge and awareness and enhanced capacity for law enforcement (surveillance, prosecution, and penalty) to increase compliance with fisheries management, MPA and MSP regulations, leading to the achievement of specific objectives. The main issue is that only 2 of 19 draft or recommended policies, regulations, and tools have been finalized, and 17 are still awaiting adoption. As mentioned in EQ1, 7 of the remaining 17 drafts have transitioned from Stage 1 to Stage 2 of the three-stage policy development process, while the other 10 have reached the final review process in Stage 1. • SA5: Trainings. As discussed in EQ2, the planned training and capacity building activities have been mostly implemented, with SA5 posting an achievement of 89% at mid-term. This accomplishment was the result of two influencing or facilitating factors. First, the project used the results of competency assessments and prioritization to enrich the existing MMAF curriculum for fisheries management and to develop a new curriculum for MPA development. These served as bases for the development of training materials for a cadre of certified government trainers and NGOs. Second, these trainers trained and certified local government personnel and community members (POKMASWAS, FAs) in the use of skills in marine and fisheries resources management and promote environment-friendly behavior at the project sites and target villages. • SA3: Livelihoods. As noted in EQ3, all five indicators (SA1.1 and associated benchmarks) to measure behavior change as a result of the project showed that the targets for SA3 have been reached at mid-term. The findings discussed under EQ3 reflected the strategies and activities undertaken to change the behavior of target populations as well as the nature and extent of changes in behavior in coastal and marine resources management practices in compliance with fishing and MPA regulations, among others. 43 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV • SA3: Marine tenure. Under this work stream of SA3, the establishment of more secure marine tenure or managed access to fishing areas was set as the major result through LMMA and TURF. Traditional “sasi” systems were incorporated, and the roles of customary, village, and religious leaders were recognized to influence behavior change and obtain community support. The project also engaged the private sector through the FT/MSC certification program, which will result in better market prices and incomes for their fishery products. Fishers have been motivated to practice sustainable fishing methods and reduce the use of destructive fishing practices. At mid-term, the project achieved 89% of the LOP target for SA3.3 and 84% of the LOP target for SA3.1. EQ4 findings reflect the nature and extent of sustainable fishing methods applied by fishers in target villages in Maluku and West Papua. In both SA3 and SA 5, the timely adoption of remaining draft or recommended policies and regulations is critical to the success and sustainability of MPA and fisheries management (LMMA/TURF), particularly for the protection and safety of POKMASWAS and other law enforcers, including non-fishing livelihood activities. Continuing training and socialization are important as most concepts and practices of the project components/SAs/TAs introduced by the project (as reflected in the TOC) were considered “new” by local governments and communities and even by some key informants from the national government. Activities should focus on specific groups tasked to sustain the project interventions at the national, local, and community levels in the remaining implementation period. RECOMMENDATIONS FOR THE REMAINING PROJECT PERIOD Table 15 below presents the recommended actions for consideration by the USAID SEA Project for possible implementation during the remaining period of the project. Table 15. Summary of MTPE Recommendations based on Responses to EQ1–EQ6 EQ Number / Enabling Conditions and Constraints Recommended Actions for the Remaining Project Period EQ1: Adoption of policies, regulations, tools Key enabling conditions: • National Law No. 23/2014; • Need for clear laws, policies, regulations, and tools to address existing key threats to marine and fisheries resources; and • Good research outputs and technical inputs. Key constraints: • Long period of review and unification of existing regulations; • Differing priorities/schedules; and • Inconsistency with national laws 1. Focus efforts on legitimacy of MPAs, EAFM, and law enforcement to achieve targets (e.g., SA4.1 and SA2.1b); 2. Build consensus on work schedule/priorities with MMAF, MOHA, and local governments; 3. Work on the database of instruments to facilitate the review/adoption of draft/proposed policies, regulations, and other legal instruments; 4. Work with MMAF/BRSDM (Bureau of Research and Human Resources) and DKPs to sustain the FCMCs, MPA WGs, and other EAFM and law enforcement groups; and 5. Enhance LGU capacity-building activities with focus on MPAs, EAFM, and law enforcement. EQ2: Training and capacity building Key enabling conditions: • Consistency of training content and process with best learning practices; • Experienced and dedicated NGO partners; • Trained champions and trainers; • Local leadership support; and • National and local government support. Key constraints: • Delayed adoption of draft/proposed policies, regulations, and tools; and 1. Adopt draft/proposed policies/regulations on MPAs, MSP, and law enforcement; 2. Complete the three-stage development program for all POKMASWAS; 3. Complete FT certification of all FAs; 4. Institutionalize the existing forums under a MMAF platform/community of practice; and 5. Prioritize integrated training manual production. 44 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV EQ Number / Enabling Conditions and Constraints Recommended Actions for the Remaining Project Period • Incomplete three-stage POKMASWAS training and FT certification of FAs in target villages. EQ3: Behavior change resulting from non-fishing livelihoods Key enabling conditions: • Livelihoods (SA3), • Trainings (SA5); • Regulations (SA4); and • Behavior change campaigns/activities (SA1). Key constraints: • Lack of previous training and engagement of local communities in community surveillance • Inability to participate in project activities due to more important family obligations 1. Extend the implementation period of behavior change activities/campaigns on MPA and fishing regulations to enable villages to reach the “action” and “maintenance” stages of behavior change; 2. Provide more training for fishers (e.g., Galo-galo) on proper use of fishing gears to comply with fishery regulations and link with government agencies for provision of proper fishing gears, etc.; 3. Facilitate completion of business agreements (MOUs) between FT-certified FAs and the private sector; and 4. Develop a more effective learning platform at the provincial level for participatory assessment and best practice sharing. EQ4: More secure marine tenure or managed access to fishing areas Key enabling conditions: • Existing local governance institutions (local leaders, DKP, and POKMASWAS); • Involvement of local fishers in surveillance work; • Livelihoods (SA3), trainings (SA5), regulations (SA4), and behavior change campaigns/activities (SA1). Key constraints: • Lack of communication equipment for community surveillance; • Small number of local people trained in community surveillance. 1. Intensify socialization/outreach activities in sustainable fisheries management options (i.e., catch quotas, closed seasons, etc.) to enable more fishers to apply improved practices; 2. Address needs of fishers and POKMASWAS and fishers for fishing and communication tools; 3. Secure the legitimization of LMMA and TURF areas; 4. Establish permanent institutional arrangements with, and provide budget allocations to, local governments for continued operation of POKMASWAS; 5. Package Best Practice Manuals to include improved conservation law enforcement practices. EQ5: Synergy of project components/SAs Key enabling conditions: • Established institutional mechanisms at national and local levels (e.g., MMAF, DKP, POKMASWAS, POKDARWIS); • NGO partners with working experience in fisheries management and conservation; • One on one outreach activities; • Local champions. Key constraints: • Capacity gaps among partners hindering effective project implementation; • Issues related to engagement of NGO partners, e.g., reporting requirements, disbursement scheme. 1. Enhance existing coordination and management platforms at national and provincial levels; 2. Conduct refresher courses for key government offices at national and local levels with focus on identified capacity gaps; 3. Develop/package a legacy conservation model to showcase effective approaches, strategies, structures, and systems for MPA and fisheries management based on recent scientific and technical innovations tested and applied under USAID SEA. 45 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV EQ Number / Enabling Conditions and Constraints Recommended Actions for the Remaining Project Period EQ6: Recommended areas (thematic/ geographic) for expansion or contraction Thematic Areas Recommended for Expansion: 1. TOC results chain. Incorporate SA4 in the TOC results chain for SA3 and SA5; 2. Policy development process. Strengthen and streamline the policy development process to facilitate adoption of enabling legal instruments; 3. Incentives for marine stewardship. Replicate incentives and increase the visibility of model villages for marine stewardship efforts; 4. EAFM. Improve communication and coordination among various EAFM implementing partners; 5. Research coverage. Incorporate ecosystem approaches in research studies; 6. Gender equality. Strengthen efforts to address gender and women empowerment; and 7. Best practice sharing. Improve sharing of best practices in PPPs for non-fishing livelihood activities and more secure marine tenure efforts. Expansion in geographic areas: Not recommended. RECOMMENDATIONS TO USAID FOR FUTURE PROJECT DESIGN For the design of similar projects in the future, the following recommendations are put forward for USAID’s consideration: i. The implementation arrangements for a project with components that will be implemented at the national and local levels should include the government agency mandated to oversee the operations of local government agencies (in this case, MOHA) in addition to the national technical agency (in this case, MMAF). As the local governments are under MOHA (as stipulated in Law 23/2014, the Local Government Law), MOHA should be an implementing partner (more actively involved in field implementation and coordination apart from oversight) so that it could get the commitment of local governments in the policy development process, as well as in EAFM, MSP, MPA, and law enforcement-related activities. The MOHA Directorates-General (DGs) of Regional Autonomy, Regional Development, and Village Governments at the national level need to be included to influence and monitor local governments’ performance and institutionalize improved marine and fisheries management practices in line with their legal mandates. ii. Complex projects, particularly those involving NRM and livelihood development, take longer to show results or yield benefits. Because of the technical and complex nature of NRM projects, all the project implementers, including local communities, need to be fully oriented and adequately trained on project concepts and practices and provided with continuing technical advice, guidance, and support during project planning and implementation. It is necessary to ensure that all partners have a common understanding of the project approaches/SAs/TAs as well as their respective responsibilities and obligations in the project. While this should be the principal responsibility of the prime contractor, active participation and oversight by USAID is essential in this process, particularly during the initial project years/ stages. And all these take time. 46 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV iii. The combined implementation of the project components/SAs showed tremendous potential for the setting up of a comprehensive capacity building and ecosystem management model that will cover policy development, competency strengthening, incentives creation, and improved management and technical tools for EAFM, MPA, MSP, and law enforcement. Good practices and key lessons from this project should be captured and compiled in a Best Practice Manual, which could serve as a reference for USAID in the design and implementation of future projects. iv. The involvement of experienced NGOs as implementing partners (i.e., RARE, MPDI, ILMMA, and Marine Change) effectively reduced the learning curve and hastened community engagement. However, as NGOs have different levels of experience and understanding of the project components/SAs/TAs, it is necessary to establish a platform or a mechanism for knowledge sharing and information exchange, including best practices and lessons learned, to enhance their technical capacity and add to their wealth of experience in fisheries management and biodiversity conservation. 47 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV ANNEX A. STATEMENT OF DIFFERENCES Statement of Differences USAID Indonesia appreciates the efforts of PRIMEX in conducting the midterm evaluation (MTE) for the USAID Sustainable Ecosystems Advanced (SEA) project, admittedly a complex activity implemented by a large number of partners working in remote areas. For reasons outside of the control of PRIMEX, this evaluation was delayed and encountered numerous challenges. USAID, however, is including the below statement of difference in the final report, because a number of the evaluation questions were not adequately addressed despite several rounds of comments. Primex submitted two MTE draft reports to USAID before submission of a third and final report. In this final report, USAID found the following remaining issues. A. Evaluation Question 1 Under what conditions are the regulations and/or tools being introduced by USAID SEA to relevant authorities leading to the adoption of these policies, regulations, and tools? On page 11, paragraph 2 and page 30, paragraph 1, Primex made the following statement: “Thus, policies, regulations, and tools enacted at the provincial level have not been supported by appropriate national decrees.” USAID questioned Primex whether a "national decree" is actually needed for coordination. Primex replied to USAID that “...a more efficient national coordination mechanism is needed to help facilitate adoption of appropriate national decrees. We are not proposing a national decree for coordination.” However, Primex did not modify the text of the report to clarify this point. As such, Primex’s statement in the report still implies that a “national decree” is needed to improve coordination. This lack of clarity in the report introduces confusion into the statement. On page 11, paragraph 3 and Page 30, paragraph 2, Primex stated that “For EQ1, the major conditions that facilitated the development and adoption of policies, regulations, and tools introduced by the project include the following: (i) National Law No. 23/2014;...”. On page 27, Primex stated: “The triangulated data identified the conditions that facilitated the adoption of RZWP-3-K PerDa at the provincial level and of regulations at the village and district levels. At the provincial level, the conditions include: (i) National Law No. 23/2014, which requires the partner provincial governments to develop institutional capacity in planning and managing the country’s marine and fisheries and produce RZWP-3-K as a requirement under the same law;” USAID noted that the list, and the emphasis on National Law 23/2014, did not appear to logically answer the question. Primex replied that “The conclusions are the MTPE team’s assessment of key conditions from the findings described previously at national, provincial and local/site level generically applicable across 48 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV institutional levels. The logic is as follows: Item (i) National Law 23/2014 is drawn from the national and provincial list on p.9; item (ii) is drawn from (ii) and (iii) of the provincial list on p.9; and item (iii) is drawn from item (v) of local/site enhancing conditions on p.10. Our assessment is that these are the most generically applicable enhancing conditions across institutional levels (national, provincial and local/site level). The others have institutional level specificity.” Primex did not answer the question in the way that it was intended. The question states "Under what condition...", which is broader than 'under which enabling laws/policies/regulations'. Law 23/2014 applies to all subnational governments across Indonesia, not only those where USAID’s SEA project is implementing interventions. Primex did not investigate or identify the specific conditions in the three provinces, and delivered a finding that has such wide-ranging applicability that its usefulness at the project level is questionable. Primex did not evaluate other possible conditions, including local champions, commitment of local and national government decision makers, capacity of civil society to advocate, etc. B. Evaluation Question 2 Under what conditions do people trained to manage marine and fisheries resources and/or advocate for sustainable management of these resources, as a result of USAID SEA interventions, apply their skills in marine and fisheries resources management (MFRM) or advocacy? Primex provides almost three pages of information pertaining to the number of trainings delivered, before providing a list that begins to answer the question on page 34: “The KII and FGD outputs revealed that other conditions facilitated the application of knowledge and skills by the trainees.” USAID commented that most of this information seemed ancillary to answering the question and that description of the training can easily be accessed in project reports. In response, the MESP Quality Assurance specialist stated that the “outline of the learning process is useful as it’s relevant to enabling factors for application of skills” and asked Primex to elaborate further. Primex stated that the “...background on the learning process is essential, as not all projects conduct capacity-building using such a process. Expanding further on (i)-(v) not pursued as such already succinctly make the point about other enhancing conditions given text length limitations (30 pp.).” In revising the report, Primex added a paragraph on page 32, with the beginning sentence: “The most compelling evidence of the application of trainees’ acquired knowledge/skills is the very good progress that the project has made in its various SAs, TAs, and activities.” Despite this slight revision, it seems that Evaluation Question 2 was never sufficiently answered. The focus of the question was on application of training, and under what conditions the skills were applied. Primex chose to focus on evaluation efforts on delivery of training, not on application of skills from training. If Primex had provided less information and material that is easily accessible in project reports, the evaluation team would have had more space within the report page limits to directly answer the question. C. Evaluation Question 3 To what extent do non-fishing livelihood activities introduced or strengthened by USAID SEA change behavior and reduce threats and improve marine conservation (e.g., livelihood program in Galo-galo, Morotai)? Under what conditions are these activities effective? Primex provided a conclusion that describes the process of behavior change implemented by USAID SEA, in great detail. However, Primex does not answer the question “Under what conditions are these activities effective?” As such, Primex did not fully answer the question. Primex utilized village level compliance with local waste management regulation as an example to demonstrate “to what extent”. However Primex did 49 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV not evaluate other possible contributing factors and USAID SEA interventions to answer the question, such as campaign activities, champion assessment tools, etc. As such, Primex’s findings may overlook these important aspects of the project. D. Evaluation Question 4 To what extent do USAID-supported fishermen with more secure marine tenure to fisheries resources apply sustainable fishing practices and reduce destructive fishing practices (e.g., Territorial Use Rights for Fishing [TURF] in Raja Ampat and Local Marine Management Area [LMMA] in Maluku)? Under what conditions are these activities effective? Primex sufficiently addressed USAID’s comments. E. Evaluation Question 5 Given that USAID SEA addresses key marine threats and drivers through a complex project comprised of multiple components/SAs implemented by a large number of partners, how well are all the components/SAs working together? What aspects of overall management are working well, and what can be improved? To what extent are there capacity gaps among the partners that might be hindering effective implementation? Primex primarily answered this question in terms of coordination and capacity building for the Government of Indonesia. USAID intended the question to evaluate overall project management and find where improvements could be made. In review of a draft of the evaluation report, USAID commented that the last portion of the evaluation question (capacity gaps) was intended for the project’s implementing partners, and not to be focused on the Government of Indonesia. Primex replied to this comment by stating that key informant interviews and focus group discussions conducted during the evaluation did not have data pointing to capacity gaps of SEA’s NGO partners. Primex further replied by stating that the report implies “potential NGO capacity gaps from other elements of the FCR for the EQ (e.g., reimbursement scheme, reporting requirements) which are largely administrative in nature pointing to financial capacity, report writing facility, etc.” In answering the last portion of the evaluation question (capacity gaps), Primex focused entirely on capacity gaps of government partners, and not implementing partners. The evaluation question did not specify a focus on government or implementing partners, and as such, the evaluation team missed an important opportunity for gathering information and conducting analysis on all of the project’s partners. This missed opportunity is supported by one of Primex’s own overall recommendations to USAID (page 64, paragraph 2), where Primex noted the following: “Because of the technical and complex nature of NRM projects, all the project implementers, including local communities, need to be fully oriented and adequately trained on project concepts and practices and provided with continuing technical advice, guidance, and support during project planning and implementation.” Primex had the opportunity to expand on this recommendation under Evaluation Question 5, but chose not to. In answering the portion of Evaluation Question 5 concerning “overall management”, PRIMEX assessed and provided examples from Strategic Approach 4 and did not explicitly assess “overall management” which addressed other Strategic Approaches. USAID finds that this focus narrows the utility of this portion of the assessment, especially when Primex concluded that “In general, the outputs/outcomes of each component/Strategic Approach could not be achieved without the support of other Strategic Approaches, including Strategic Approach 1, since the components/Strategic Approaches/Technical Approaches at the 50 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV project start were all new to the target local government personnel and coastal villagers” (Page 32, paragraph 1). Specifically concerning capacity gaps of government partners, Primex introduced uncertainty into the evaluation’s conclusions. In the conclusions for Evaluation Question 5, Primex stated that “It is necessary to provide specialized training for target groups at MMAF and DKPs in the remaining project period to facilitate the capacity building of concerned staff and the adoption of draft or recommended policies and regulations and sustain the implementation of project activities at the ground level” (page 56, paragraph 1). This appears to contradict the summary of findings for Evaluation Question 2, where Primex stated that “Current evidence indicates that a large number of trained people have applied their learnings and skills (e.g., in community advocacy, community surveillance, sustainable fishing practices, MPA management, and non-fishing livelihood activities)” (Executive Summary, page viii, paragraph 2). F. Evaluation Question 6 Based on the answers above about the conditions under which the USAID SEA project strategies are or are not effective, what are the recommended areas (thematic and/or geographic) for expansion/contraction for USAID? Primex’s main conclusion for this question was the following: "The project components/SAs/TAs were all found effective in improving fisheries management and conserving marine biodiversity in the context of FMA 715. The combined implementation of these SAs/TAs has produced concrete proofs of the desired results of working well together. It has led to improved fisheries, enhanced marine biodiversity, and early benefits for fishing villages in the project sites" (Page 36, paragraph 1). The recommendations flowing from this overall conclusion do not fully answer the Evaluation Question. Primex states seven “... areas are recommended for expansion in the thematic areas”, where at least two of the seven “areas” do not have actionable recommendations. For example, in the thematic area of “Policy development process”, Primex states a brief description of the process without stating any recommendations for expansion/contraction as stated in the question. As another example, in the thematic area of “Incentives for marine stewardship”, Primex recommends to “Give due recognition to villages that have been successful in their non-fishing livelihood activities and/or in securing marine tenure to serve as models for other villages to emulate.” However, Primex does not clarify or suggest what “due recognition” means in terms of expansion/contraction as stated in the question. Primex also stated that “monitoring and evaluation (M&E) of components/SAs should also cover socio-economic development” which appears to contradict other aspects of this evaluation. For instance, the project implemented a robust system for monitoring socio￾economic development which Primex used for answering Evaluation Question 3. In Evaluation Question 6, Primex provided recommendations to improve gender equality. This question asked to consider recommendations based on findings and conclusions questions 1 to 5. In these findings and conclusions, Primex did not provide any discussion regarding gender which could provide a basis for the recommendation in Evaluation Question 6. September 9, 2020 USAID INDONESIA 51 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV ANNEX B. THEORY OF CHANGE OF USAID SEA PROJECT 52 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV ANNEX C. USAID SEA PROJECT AREAS 53 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV ANNEX D. THE EVALUATION STATEMENT OF WORK (SOW) 54 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 55 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 56 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 57 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 58 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 59 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 60 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 61 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 62 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV ANNEX E. TEAM ORGANIZATION, COMPOSITION, AND ASSIGNED TASKS At the institutional level, PRIMEX is contractually responsible to USAID/Indonesia for the delivery of the required services for the midterm performance evaluation of the USAID SEA Project. In line with PRIMEX’s commitment to quality and excellence and to facilitate the attainment of the evaluation objectives, Ms. Elvira C. Ablaza, PRIMEX President and CEO, serves as the Project Director, with oversight responsibility for the satisfactory and timely execution of the services. She is directly involved in the preparation of the major reports and participate in major meetings of the Team with USAID/Indonesia, if necessary. At the team level, the Team Leader (TL): (i) provides overall leadership and direction for the Team’s activities to ensure the effective and efficient conduct of the evaluation and its timely completion; (ii) supervise the activities of the Project Evaluation Team members and coordinate with the USAID/Indonesia Contracting Officer’s Representative (COR) and Alternate COR in the planning and scheduling of evaluation activities, as needed; and (iii) primarily responsible for preparing the required evaluation reports with inputs from the individual specialists. The individual specialists will report to the TL in the performance of their tasks and will interact, cooperate, collaborate, and support each other in a cohesive, integrated manner in the timely delivery of high-quality outputs. Their assigned tasks are shown in the table below. Key Personnel and Assigned Tasks Name and Position Tasks/Assignments Geronimo T. Silvestre Team Leader/Marine Conservation and Sustainable Fisheries Specialist • Provide technical leadership and overall direction for the project’s activities including evaluation of marine conservation activities of the Project measured through MPA Effectiveness Index scores or other approved national or international standards; • Prepare and manage the detailed Work/Implementation Plan and ensure the effective and efficient conduct of the MTPE in close coordination with USAID/Indonesia and Tetra Tech ARD; • Supervise the MTPE field teams and conduct field data gathering (i.e., KII and FGD) in selected Project areas; • Lead the preparation of the required reports and the conduct of briefings for USAID/Indonesia and other key stakeholders; • Assess the performance of SEA Project in terms of its success in achieving its goals of improving fisheries management as a result of USG assistance, measured through the MMAF EAFM or other approved national or international standards; and • Assess the success of SEA Project in delivering results in terms of improvements in sustainable fisheries management created, strengthened, promulgated, and/or enforced at all levels. Luis P. Eleazar Senior Evaluation Specialist • With the TL and Senior Survey Specialist, develop the data gathering (i.e., KII and FGD) instruments, and lead the analysis of gathered data (i.e., computation of changes in certain indicators over time, content analysis, triangulation analysis, external validity tools, etc.), and draft the report on the data analysis findings and conclusions; and • Assist the TL in the development of the required MTPE reports/ deliverables, particularly the Evaluation Approach and Implementation Plan, First Draft of Evaluation Report and Final Evaluation Report including the preparation of PowerPoint materials for briefings/debriefings. 63 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Name and Position Tasks/Assignments Imelda G. Pagtolun-an, Ph. D. Senior Survey Specialist / Statistician • With the TL and Senior Evaluation Specialist, develop the data gathering (i.e., KII and FGD) instruments, and lead the analysis of gathered data (i.e., computation of changes in certain indicators over time, content analysis, triangulation analysis, external validity tools, etc.), and draft the report on the data analysis findings and conclusions; • Orient/train KII and FGD field teams (facilitators, documenters, and data encoders) with the Statistician; • With the TL, conduct KIIs at the national level; • With the other Team members, supervise primary data encoding and data verification with KII/FGD teams; and • Assist the TL in the development of the required MTPE reports/ deliverables, particularly the Evaluation Approach and Implementation Plan, First Draft of Evaluation Report, and Final Evaluation Report including the preparation of PowerPoint materials for briefings/debriefings; and • Assist the TL in the conduct of debriefings with USAID/Indonesia and learning event with other key stakeholders. Gusti Ayu Anggara Kasih National Evaluation Specialist • Assist the international MTPE Team members in gathering data from the field, including quality assurance of data gathered; • Assist in identifying, organizing, and training the KII/FGD field teams; • Assist in supervising field teams; and • Assist in the organization and conduct of debriefing workshops for USAID/Indonesia and other key stakeholders Purwito Martosubroto, Ph. D. National Fisheries Expert • Assist the TL in assessing the performance of SEA Project in terms of its success in achieving its goals of improving fisheries management as a result of USG assistance, measured through the MMAF EAFM or other approved national or international standards; • Assist in the assessment of the success of SEA Project in delivering results in terms of improvements in sustainable fisheries management created, strengthened, promulgated, and/or enforced at all levels; • Assist in the supervision of field teams and conduct textual analysis and direct observation in the assigned area; and • Prepare reports on the findings from the above assessments for incorporation in the Evaluation Team’s reports to USAID/INO. These experts are all highly qualified professionals with: (i) extensive experience in the planning, design, implementation, and evaluation of donor-supported fisheries and NRM projects in Indonesia and other countries in Southeast Asia; (ii) an excellent familiarity with the TOC approach and results-based M&E; and (iii) strong technical, analytical, writing, and interpersonal skills. Mr. GERONIMO T. SILVESTRE Team Leader/Marine Conservation and Sustainable Fisheries Specialist Mr. Geronimo T. Silvestre is a recognized fisheries and coastal resources research, policy, and management expert in Southeast Asia, particularly the Philippines. He has over 40 years of work experience in his field of expertise. He was a member of the Faculty of the College of Fisheries of the University of the Philippines (UP) from 1978–1988, a Research Scientist at ICLARM (now the WorldFish Center) from 1988- 2002, and managed various USAID-funded fisheries research and management projects under Tetra Tech (2002-2018) in the Philippines and the region. He earned his BS in Marine Fisheries from UP in 1978 and his MS in Marine Science (major in Biology and Living Resources) from the Rosenstiel School of Marine and 64 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Atmospheric Science of the University of Miami in the USA in 1984. He has published and/or edited over 90 scientific articles, volumes, and books in the course of his career. Mr. LUIS P. ELEAZAR Senior Evaluation Specialist Mr. Luis P. Eleazar is a senior evaluation specialist with 30 years of experience in natural resources and environmental management and assessment; institutional development and capacity building; and monitoring and evaluation (M&E). He has worked with major multilateral agencies, such as ADB, WB, and UNDP, as well as with bilateral donors including USAID. Since the 1980s, he has worked, in various capacities, with communities and local governments in their rural/community development and natural resource management projects. He has also been involved in the development and implementation of results-based M&E systems and in the conduct of impact evaluation of community-based development projects sponsored by government and civil society organizations (CSOs). Examples of his experience as Evaluation Specialist/Team Leader include the following projects: (i) Final Performance Evaluation of USAID/P’s Strengthening the Capacity of Civil Society Organizations (CSOs) in the Philippines; (ii) USAID/P’s Alliance for Mindanao Off-Grid Renewable Energy (AMORE) 3 Program; (iii) USAID/P’s Environmental Governance Project; (iv) ADB’s Local Government Finance and Fiscal Decentralization Reform Program (TA 8390-PHI); and (v) JICA’s Environmental and Social Baseline Survey for the Comprehensive Capacity Development Project for the Bangsamoro. Mr. Eleazar holds a Post-Graduate Diploma in Integrated Regional Development Planning (MA degree) from Rehovot Development Centre, Israel, an MS in Resource Economics from the University of Alberta in Edmonton, Canada, and a BS in Forestry from the University of the Philippines at Los Baños (UPLB). Dr. IMELDA G. PAGTOLUN-AN Senior Survey Specialist / Statistician Dr. Imelda G. Pagtolun-an is a seasoned statistician/social survey specialist with over 40 years of experience in research and monitoring and evaluation (M&E) of development projects, including numerous methodologies covering quantitative and qualitative field designs, such as household surveys (HHS), FGDs, KIIs, participant observation and studies, action research, and participatory researches. She is an expert in “before and after” and “with and without” intervention analysis using triangulation (mixed method) and attribution analysis for quantitative and qualitative data with various stakeholders as sources of data. Dr. Pagtolun-an was Principal Investigator in more than 35 impact assessments and evaluations of development and ameliorative projects in agriculture and natural resources in the Philippines and the Asia￾Pacific region. Recent examples of her relevant experience include the following projects: (i) Mid-term Performance Evaluation of the Ecosystems Improved for Sustainable Fisheries (ECOFISH) Project, Philippines, May-December 2015; (ii) Third Party Assessment (TPA) of Integrated Coastal Management in the Philippines (ICM PH), Philippines, 2017-2018; and (iii) Development of the Implementation Completion Results Report (ICR) for the World Bank-PEMSEA Knowledge Management Project, China, Indonesia, Philippines, and Viet Nam, May-September 2016. Dr. GUSTI AYU ANGGARA KASIH National Evaluation Specialist Dr. Gusti Ayu Anggara Kasih is a senior environment specialist with over two decades of experience in environmental and climate change work in Indonesia. She studied and analyzed the relationship of climate change to the marine pelagic zone and in coastal areas as a Research Scientist at the Toyohashi University of 65 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Technology in Japan, where she obtained her degree in Doctor of Engineering in Environmental and Life Engineering specializing in Ecological Modelling. Since the completion of her postgraduate studies in Japan, she has been involved in GTZ-assisted climate change projects in Indonesia, focusing on vulnerability assessment and climate change adaptation. Dr. Kasih was the Climate Change Specialist for the Coastal and Marine Resource Management in the Coral Triangle South East Asia (Philippine, Indonesia and Malaysia) from 2012-2017 where she handled climate change adaptation assessment and development. She also coordinated in-country activities in developing adaptation to potential impacts of climate change based on vulnerability assessments and build resiliency of local communities. She also provides consultancy services, including leading researchers, statisticians, and surveyors in the conduct of baseline, midterm, and end-of￾project assessments. Dr. PURWITO MARTOSUBROTO National Fisheries Expert Dr. Purwito Martosubroto is a resource assessment, fisheries management, integrated coastal management (ICM) and project evaluation specialist with more than 35 years of experience in fisheries management. He acted as the Senior Fisheries Management Specialist of the Indonesia Marine and Climate Support (IMACS) Projectassisting Marine Affairs and Fisheries (MMAF) in the development of Strategic Planning for 2015-2019. In February 2012, he became part of the Technical Assistance for the EU-Indonesian Trade Support Programme II (TSP II) where he acted as the Legal Consultant. From 1985 to 1990, he acted as Director on Fishery Resources Management, Directorate General of Fisheries and Ministry of Agriculture in Jakarta, Indonesia where he conducted regular assessment of resources potentials in marine and inland water bodies and regularly reviewed status and trends of capture fisheries. Dr. Martosubroto earned his PhD in Fisheries Biology from Dalhousie University, Halifax, Nova Scotia, Canada and his MS in Marine Science from University of Miami, Florida, USA. 66 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV ANNEX F. LIST OF DOCUMENTS REVIEWED 1. Activity Reports a. CTC • Activity Report: Meeting of KKNI Writer Team and MMAF Law and Organization Department on KKNI Draft Clarification, 23 January 2019 • Resource Use Monitoring and Spawning Aggregation Sites Survey in Lease Islands MPA, Maluku Province, 5-8 March 2019 b. Marine Change (MC) • 2018  An Overview of Tourism and Reef & Demersal Fisheries Sector in Indonesia and a Rapid Assessment of the Tourism & Fisheries Sectors in Morotai, North Maluku, Indonesia, January 2018 • 2017  Activity Report: Review of Marine Ecosystem Services and Valuation Methodologies for the USAID SEA Project, 2 June 2017 c. MDPI • Activity Report: Update DKP & Submit 22 Vessel Document Requirement of the Fishermen in UPP Moratai, 4 December 2018 • Activity Report: World Fisheries Day (WFD) Celebration: Together we can make the World without Plastic Waste, 30 November 2018 • Activity Report: World Fisheries Day (WFD) Celebration: Building Love Ad Pride for the Fishery Resources in North Maluku, 26-30 November 2018 • Activity Report: Capacity Building Field Staff in Bali Celebration: Happy People Many Fish, 12-15 November 2018 • Regular Meeting III: Data Management Committee, North Maluku Province, 8 November 2018 • Focus Group Discussion Fisheries Certification, 7 November 2018 • Activity Report: Arrangement of Small Fishing Vessel Document in Jailolo Island and Ternate City, North Maluku, 29 October – 2 November 2018 • Activity Report: Spot Trace Technology, Introduction for the Student of Khairun University, Ternate, North Maluku, 23 October 2018 • Activity Report: Monthly Update Program to DKP Ternate and Submit the Document Requirement of BPKP, 18 October 2018 • Pre-Our Ocean Conference (OOC). Panel Discuss “ Commitment towards Sustainable Fisheries”, 15 October 2018 • Activity Report: Arrangement of Small Fishing Vessel Document in Waya Village & Completion of the Statement Letter of BPKP Issuance at Bahu and Lele Village, South Halmahera-North Maluku, 13-18 October 2018 • Activity Report: Capacity Building and Data Review of Fisher Champion, 10-11 October 2018 • Activity Report: Writing the Name of “Selar mark” on the Hull of Fishing Vessel to the Small Pas Issue, Base on the Policy of KSOP Ternate, 1-4 October 2018 • Report: Basic Training of Port Sampling Data Collection on Small Scale Tuna Fisheries, April 2018 67 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV d. RARE • Workshop Report: Fisheries Assessment I – Teluk Mayalibit e. SEA EAFM • Consultative Meeting on Dissemination Insights and Findings of Logbook Fieldwork, 9 April 2019 • The Participation of the USAID-SEA Project in the 12th Asian Fisheries and Aquaculture Forum, Iloilo City, Philippines, 8-12 April 2019 • Stakeholder Consultation Workshop (SCW) on Formulating Operational Objectives for Developing Harvest Strategy in the Management of Small Pelagic Fishery in FMA 715, 25-29 March 2019 • Technical Report: Lessons Learned of the Three year Sustainable Fisheries Activities Implementation, 15 March 2019 • Workshop on Initiating Harvest Strategy Development for Small Pelagic and Reef Fish at FMA 715, 12 February 2019 • Verification Workshop for Fishery Data Collected from FMA 715: Initial Processing and Analyses, 18-20 December 2018 • Socialization and Consultation Meeting on the Initiative to Pilot Fishing Logbook for Small-Scale Fishers, 21 November 2018 • Activity Report: Initial gap analysis of socio-economic research needs to support harvest strategies for red-snapper, grouper and snapper and fishery management implementation, 23 January 2017 f. SEA MPA • Activity Report: PPP & Small-Scale Tourism in MPAs Workshop, 5-9 November 2018 • Activity Report: Technical Review of SEA MPAs Zoning and Management Plans, 17-20 September 2018 g. SEA MSP • Preparation Meeting for National MSP Workshop on the Integration of MSP Geoportal System to Accelerate the Implementation of MSP in Indonesia, 28 January 2019 • Seanode Technical Coordination Meeting and Preparation of MSP Implementation Symposium, 10-11 December 2018 h. TNC • Workshop Report: Integrating Fisheries and Social-Economic-Cultural Aspects for Refining the Guidelines of Marine Protected Area Networks in Indonesia, May 2018 i. WCS • Activity Report: Coordination and Kickoff Meeting of MPA Working Group of North Maluku, 13-14 August 2018 • Activity Report: Community Meeting and Focused Group Discussion for Morotai MPA Establishment, TA2-18C, July 2018 • Training Report: Open Standards for the Practice of Conservation and MPA Management Plan Document Preparation, 19-24 February 2018 • Report: Fisheries Scoping Survey in North Maluku Province, October 2017 • Activity Report: Training Report in Fish Catch Data Collecting, August 2017 • Activity Report: Initiation of MPA Development in Morotai and Dissemination of Baseline Survey Results, TA2-18C • Assessment Report: Endangered Threaten and Protected (ETP) Species, Wildlife Trade, and Destructive Fishing in North Maluku 68 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV • Report: Training Needs Assessment for Law Enforcement North Maluku • Report: Training Needs Assessment for Fisheries Management in the North Maluku Province j. WWF • 2019  Activity Report: Socialization of Community Based Marine and Coastal Resources Surveillance System in Serutbar MPA (SA 2 TA4 – 19), January – February 2019  Activity Report: Focus Group Discussion (FGD) on Marine Tourism Governance Identification to Support Serutbar MPA Management (SA 3-13), 16 January 2019  Activity Report: Support the Provincial West Papua Governments on Small Scale (≤10 GT) Fishing Vessels Registration in Sorong Selatan, 11 November 2018 – 18 February 2019  Activity Report: Marine Tourism Carrying Capacity Analysis at Koon MPA and its Surrounding (SA3-13), September 2018 – January 2019 • 2018  Training Report: Training on Fisheries Product Processing – Batch 2 (SA5-48), 2-7 December 2018  Activity Report: West Papua RZWP-3-K: Final Document Review Meetings with Ministry of Marine Affairs and Fisheries (SA2-TA35), 26 November-14 December 2018  Training Report: Basic Marine Conservation Training for Local Communities in North Seram and North-West Seram Sub-District (SA5-44), 25-26 October and 6-7 December 2018  Activity Report: Daily Data Collection Fishery North Maluku, November 2018  Activity Report: Fisheries Daily Data Collection in West Papua, November 2018  Activity Report: Periodic Resource Use Monitoring (RUM) at North Seram and Northwest Seram Sub District, 29-30 November 2018  Activity Report: Fisheries Onboard Observer SKKNI: Training Curriculum and Module Trial (SA5-2), 5-11 November 2018  Activity Report: Support the Maluku Provincial Government on Small Scale (≤10 GT) Fishing Vessel Registration in Labuan Village, 8 October-9 November 2018  Activity Report: Daily Fisheries Data Collection in Maluku, October 2018  Activity Report: Fisheries Daily Data Collection in North Maluku, October 2018  Activity Report: Fisheries Daily Data Collection in West Papua, October 2018  Activity Report: Periodic Resource Use Monitoring (RUM) at Sawai Bay Area (TA2-33), 30- 31 October 2018  Activity Report: West Papua RZWP-3-K Technical Assistance and Technical Consultations of Intermediate Document (TA 3-5), 15-26 October 2018  Training Report: Training on Fisheries Product Processing (SA5-48), 30 September – 2 October 2018  Activity Report: Fisheries Daily Data Collection in North Maluku, September 2018  Workshop Report: Koon MPA Payment for Environmental Services (PES) Review 2016- 2018, 12 September 2018  Activity Report: Grading Phase of PT. Kurabesi Nusantara, Signing Blue Candidate in West Papua, 1-11 September 2018  Activity Report: Fisheries Improvement Program Introduction and Seafood Savers Engagement to PT. Irian Marine Product Development ( SA3-25), August to October 2018  Activity Report: Daily Fisheries Data Collection in Maluku, August 2018  Activity Report: Fisheries Daily Data Collection in North Maluku, August 2018 69 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV  Activity Report: Fisheries Daily Data Collection in West Papua, August 2018  Activity Report: Signing Blue Monthly Progress in Maluku and West Papua until August 2018  Activity Report: The Adat Declaration to Support Marine Protected Area Development in South Sorong, 14-15 August 2018  Activity Report: Support the Provincial West Papua Governments on Small Scale (≤10 GT) Fishing Vessels Registration in Bintuni Bay, 30 July-30 September 2018  Activity Report: RZWP-3-K: Preparation Towards Technical Consultation of Intermediate Document, 24 July-11 October 2018  Activity Report: Periodic Spawning Aggregation (SPAG) Monitoring at Koon MPA, July – August 2018  Activity Report: Daily Fisheries Data Collection in Maluku, July 2018  Activity Report: Periodic Resource Use Monitoring (RUM) at Sawai Bay Area, 19-20 July 2018  Activity Report: Daily Fisheries Data Collection in Maluku, June 2018  Activity Report: Daily Fisheries Data Collection in West Papua, June 2018  Activity Report: Signing Blue Promotion to Engage and Recruit Marine Tourism Private Sectors in West Papua to Conduct Marine Tourism Improvement Program (SA3), 4-10 June 2018  Activity Report: Daily Fisheries Data Collection in Maluku, May 2018  Activity Report: Data Collection on Fisheries Activities in West Papua, May 2018  Activity Report: Resource Use Monitoring, 29 May-1 June 2018  Workshop Report: Dissemination of Destructive Fishing Study and Initial Development of Action Plan on Combatting Destructive Fishing in West Papua (SA2 TA4), 8 May 2018  Training Report: First Responder Initiation in Maluku, 7-9 May 2018  Activity Report: The Pre-Convention Meeting of Indonesian National Work Competency Standard (SKKNI) for Fisheries Onboard Observer Program, 7 May 2018  Activity Report, Spawning Aggregation Monitoring at Koon MPA, May 2018 (SA2 TA2), 2-6 May 2018 (Full Moon)  Activity Report: Daily Fisheries Data Collection in Maluku, April 2018  Activity Report: Fisheries Daily Data Collection in North Maluku, April 2018  Activity Report: Daily Fisheries Data Collection in West Papua, April 2018  Workshop Report: Dissemination of Destructive Fishing Study and Initial Development of Provincial Action Plan on Combatting Destructive Fishing in Maluku (SA2 TA4), 19 April 2018  Activity Report: Marine Tourism’s Best Environmental Equitable Practices (BEEP) Socialization in Sawai Bay (SA 3), 17-19 April 2018  Activity Report: Public Consultations of West Papua’s RZWP-3-K Initial Document (TA3- 10), 17-19 April 2018  Activity Report: Resource Use Monitoring at Koon-Neiden MPA, 17 April 2018  Activity Report: Rouse Use Monitoring, 11-12 April 2018  Activity Report: The 2nd Preparation Meeting for Indonesian National Work Competency Standard (SKKNI) for Fisheries Onboard Observer Program Development (SA5-11), 11-12 April 2018  Activity Report: Reef Health Monitoring at Koon MPA, 9-22 April 2018 70 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV  Activity Report: Public Consultation for New MPA Initiation in South Sorong, West Papua (TA2-33A), 9-14 April 2018  Training Report: SCUBA for Masyarakat Adat Kataloka, 3-5 April 2018  Activity Report: Spawning Aggregation Monitoring at Koon MPA in April 2018, 1-4 April 2018 (Full Moon); 13-17 April 2018 (New Moon)  Activity Report: Fisheries Daily Data Collection in Maluku, March 2018  Activity Report: Fisheries Daily Data Collection in North Maluku, March 2018  Activity Report: Fisheries Daily Data Collection, March 2018  Workshop Report: Roadmap to Initiate Marine Protected Area (MPA) in Bintuni and South Sorong Districts, 27 March 2018  Activity Report: Tourism BEEP Training, 22-27 March 2018  Activity Report: Resource Use Monitoring, 22 March 2018  Activity Report: Resource Use Monitoring (RUM) at Sawai Bay, as Part of Sawai Bay MPA Initiation, 20-22 March 2018  Activity Report: West Papua RZWP-3-K Task Force Coordination Meeting on Initial Document Review and Public Consultation Preparation, 15 March 2018  Activity Report: The Preparation Meeting for Indonesian National Work Competency Standard (SKKNI) for Fisheries Onboard Observer Program Development, 12 and 22-23 March 2018  Activity Report: Spawning Aggregation Monitoring at Koon MPA, 4-5 March 2018 (Full Moon); 17-21 March 2018 (New Moon)  Activity Report: Fisheries Daily Data Collection in Maluku, February 2018  Activity Report: Fisheries Daily Data Collection in North Maluku, February 2018  Activity Report: Fisheries Daily Data Collection in West Papua, February 2018  Training Report: First Responder Initiation Workshop, 26-28 February 2018  Activity Report: Resource Use Monitoring, 21 February 2018  Activity Report: Ecosystem Approach to Fisheries Management (EAFM) Action Plan Development in West Papua Province (Kota Sorong. Sorong Selatan and Teluk Bintuni), 20 February 2018  Activity Report: Spawning Aggregation Monitoring at Koon MPA, 15-19 February 2018  Activity Report: First Public Consultation at Sawai Coastal Villages and Government Level, as Part of New MPA Initiation in Sawai Bay Area, 7-15 February 2018  Activity Report: West Papua RZWP-3-K Technical Consultations of Base Maps – Thematic Maps and Initial Document, 5-8 February 2018  Activity Report: Focus Group Discussion on Presidential Regulation Draft of the Subsidize / Incentive Procedures for Fishers, Fish Farmers and Salt Farmers, 25-26 January 2018  Activity Report: Fisheries Daily Data Collection in Maluku, January 2018  Activity Report: Daily Fisheries Data Collection in North Maluku, January 2018  Activity Report: Daily Fisheries Data Collection in West Papua, January 2018  Activity Report: West Papua RZWP3K Technical Consultation Preparation Meeting-II, 19 January 2018  Activity Report: West Papua RZWP3K Technical Consultation Preparation Meeting-I, 12 January 2018  Activity Report: Spawning Aggregation Monitoring at Koon MPA 71 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 2. Baseline Reports a. Initial PMI (Performance Management Indicators), Baseline Report, August 2018 (revised November 2018) b. Initial PMI (Performance Management Indicators), Baseline Report (Year 1), March 2017 3. Deliverable Reports a. AP2HI • Software for Vessel Registration, September 2017 • Desktop Data for Vessel Registration, September 2017 • Field Assessment Bitung, 10-14 July 2017 • Conference Report: The 6th International Coastal Tuna Business Forum, 1 June 2018 • FAD Registration Software, May 2018 • Technical Report: First Quarterly Report on Public-Private Engagement Activities • Technical Report: Second Quarterly Report on Public-Private Engagement Activities • Technical Report: Third Quarterly Report on Public-Private Engagement Activities • Technical Report: Fourth Quarterly Report on Public-Private Engagement Activities • Technical Report: Fifth Quarterly Report on Public-Private Engagement Activities • Technical Report: Seventh Quarterly Report on Public-Private Engagement Activities • Technical Report: First Quarter Report on Leveraging Investment for Sustainable Fisheries • Technical Report: Second Quarter Report on Leveraging Investment for Sustainable Fisheries • Technical Report: Third Quarter Report on Leveraging Investment for Sustainable Fisheries • Technical Report: Sixth Quarter Report on Leveraging Investment for Sustainable Fisheries • Technical Report: Seventh Quarter Report on Leveraging Investment for Sustainable Fisheries • Technical Report: Trainer Selection to Train Small-Scale Fisher to Adopt Sustainable Practices • Technical Report: Enhancing and Updating Sustainable Pole and Line and Handline Tuna Fisheries Training Curriculum • Evaluating and Monitoring the Social Economic Impact of Eco Label Certification in Pole & Line and Handline Tuna Fisheries • Activity Report: Vessel Marking Activity • Technical Report: Catch and Effort, Vessel Tracking System • Technical Report: Report on the Promotional Strategy and Material b. Conservation International (CI) • Deliverable Report #17: Standard Operation Procedure (SOP) for the Fakfak MPA Management Unit, May 2019 • Deliverable Report #13: Document Proving the Distribution of Communication Tools, March￾May 2019 • Deliverable Report #12: Government Endorsement and Legalization for the Community Surveillance and Patrol System of the Fakfak MPAs, December 2018-Janaury 2019 • Deliverable Report #11: Workshop for the Surveillance Community Group Patrol • Detailed Year 1 Work Plan, Activities YR2, and Organizational Chart for Project Staff (Report on Deliverable 1), 10 November 2018 • Deliverable Report #10: Legal Endorsement Process of the Charter and Operational Structure of the Fakfak Management Body, October-November 2018 72 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV • Activity Report: Fakfak MPAs Zoning and Management Plan Stakeholder Consultation Process and Submission for West Papua Governor Enactment, September 2018 • Activity Report: June 2018 • Activity Report: Public Consultation and Agreement on MPA Management and Zonation Plan, May 2018 • Activity Report: Natural Resource and Socio-economic Assessment to Provide Information on MPA Management Plan and Zoning Development in Fakfak, April 2018 • Activity Report: Report on the Technical Assistance Provided to the Fakfak Regency and West Papua Provincial Governments for the Legal Gazettement of the Fakfak MPAs, January 2018 • Activity Report: Introducing MPA and Conservation Approaches through Various Community Awareness Campaign Events c. ILMMA • IMMA Deliverable Report 6, April 2019 • ILLMA Deliverable Report 5A, April 2019 • ILLMA Deliverable Report 5B, April 2018- March 2019 d. MDPI • Input to Annual Report, Reporting Period: 2017-2018 • Progress Report, March 2018 • Progress Report, September 2017 e. RARE • Technical Report: Supply Chain Assessment for Target Fisheries in Batanta & Salawati Island (Dampier Strait), January 2019 • Campaign Report: Final Social Marketing Campaign Strategies for Mayalibit Bay and Dampier Strait, January 2019 • Technical Report: District Policy Document Drafted for Dampier Strait, September 2018 • Technical Report: Supply Chain Assessment for Target Fisheries in Mayalibit Bay, July 2018 • Technical Report: Business Interaction Improvement with New Technologies – Fishers Registration in Raja Ampat, July 2018 • Technical Report: District Document Drafted for Teluk Mayalibit, July 2018 • Technical Report: Regulatory Pathways and Strategy for Legal Recognition by Adat Councils and Local Government at Teluk Mayalibit and Dampier Strait, July 2018 • Technical Report: Policy Recommendations to West Papua Provincial Marine Spatial Plan, July 2018 • Activity Report: Training on Village Regulation Preparation, April 2018 • Activity Report: Fisheries Management Plan and Standard Operating Procedures for Dampier Strait, May 2018 • Technical Report: TURF Location, Boundary & Fishery Management Regulation in Salawati and Batanta Island, May 2018 • Technical Report: Behavior Change Practices to be Enacted in Mayalibit bay and Dampier Strait, February 2018 • Activity Report: TURF Social Marketing Campaign Strategies, February 2018 • Activity Report: Communication and Outreach Training Workshop, February 2018 • Activity Report: Fishery Management Unit Formally Established at Dampier Strait, February 2018 • Activity Report: Fishery Assessment and Dampier Strait, Raja Ampat, West Papua, November 2017 73 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV • Activity Report: Baseline KAP and Socioeconomic Surveys, September 2017 • Activity Report: Fishery Assessment of Mayalibit Bay, Raja Ampat, West Papua, September 2017 • Management Plan and Standard Operating Procedures for the Customary Fisheries Area of Mayalibit Bay, Raja Ampat, West Papua, September 2017 • Activity Report: Institutional Structures and Capacities in Dampier Strait, Raja Ampat, West Papua, August 2017 • Workshop Report: Fisheries Assessment I – Teluk Mayalibit, 26 July 2017 f. RCI • Product Branding and Marketing, Product Sales and benefit Sharing System Report for Community Products in Maregam, Galogalo, Pam, and Telma, 2018-2019 • Training, Production Workshop, and Capacity Building Report: Soap, Pottery, and Salted Fish, March 2018-2019 • Community Business Plan ver. 1, Technical Document, October 2018 • Training Report: Business Management, Hospitality, Guide and Interpretation and Tourism Best Practices, March-May 2018 • Technical report: Feasibility Study Feed for the Future Through Community based Business Development in Maluku Utara and Papua Barat Province, 29 December 2017 • Activity Report: Site Visits in Maluku Utara Province, 20-30 September and 21 October-2 November 2017 g. TNC • Designing a Network of Marine Protected Areas for West Papua Province, Indonesia, 1 March 2019 • Workshop Report: Technical and Mentoring Review Workshop of SEA MPAs Zoning Design, 17-20 September 2018 • Technical Report: Guidelines of Indonesia Marine Protected Area Networks, September 2018 • Workshop Report: Maluku and North Maluku Marine Protected Area Networks Design Workshop, 22-24 May 2018 • Workshop Report: Social, Economic, Cultural and Governance Performance Indicators for marine Protected Area Networks in Indonesia, 22 February 2018 • Workshop Report: Biophysical Objectives, Criteria and Performance Indicators for Marine Protected Area Networks in Indonesia, 1-2 February 2018 • TNC Contract Deliverable 2. Capacity Building: (Output 1.a Part 1): Outline and Initial Materials for Inclusion in Guidebook for Designing MPAs and MPA Networks • Biophysical Criteria for Designing Networks of Marine Protected Areas to Benefit People and Nature in Indonesia: A Practitioner Guide h. UKIP • Report on Support Provided to Government of West Papua for a Development of Policies for Anchovy Management, 1 March 2019 • Technical Report: Training to Government Officials and Stakeholders on Sustainable Fisheries and Potential Harvest Strategy and Rules, 30 November 2018 • Technical Report: Document with Harvest Strategy and Rules Based on Scientific Data and Existing Tenurial Rights, 27 September 2018 • Technical Report: Report on Data Collection, 29 June 2018 • Technical Report: Report on the Incorporation of the Database into the Government Data, 29 June 2018 74 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV • Technical Report: Progress on Data Collection and Fishery Database Development (January to March 2018), 5 April 2018 • Technical Report: Progress Report on Data Collection and Fishery Database Development, 11 December 2017 • Technical Report: Workshop on Capacity Building for Community Toward Anchovy Fisheries, 17 October 2017 • Technical Report: Rapid assessment on harvest variations of target fishes (anchovy and groupers) in Kabui and Misool to determine the best timing and sampling areas for annual data collection of the target fishes particularly of anchovy. Document establishing the data collection protocol, 11-19 August 2017 i. UNIPA • Activity Report: Survey on the Types, Population, Egg Production, and CPUE Level of Flying Fish in Fakfak and Surrounding Waters, 28 August – 7 September 2017 • Technical Report: Inventory Survey on Fishing Vessels and Fishing Geer of Flying Fish Fishermen as well as other Fisheries in Fakfak Waters • Technical Report: Survey of Business Income, Revenues and Markets of Flying Fish Eggs in Fakfak • Technical Report: Survey on the Level of Local Community Participation and Women in Flying Fish Management in Fakfak • Habitat Characteristics and Distribution of Flyingfish in Fakfak and Surrounding Waters • Technical Report: Document with the Revision of the Management of Flyingfish Resources Continuously Through Various Approaches in Accordance with the Conditions of Local and Migrant Fishermen • Technical Report: Business and the Economic Sustainability of Flying Fish Eggs in Fakfak District, West Papua Province 4. Final Reports a. 2019 • Activity Report: Data Verification & Preparation for Harvest Strategy Development for Indonesia’s FMA 715, 29-30 November 2018 • Coordination and Consultation Workshop for Initiation Harvest Strategies Development for Small-Pelagic, Demersal (Red-Snapper and Grouper), Flying Fish, Anchovy and Turf in 715 FMA, 16 November 2018 • Activity Report: Buano Islands Rapid Baseline Behavior Assessment Pre-Test, 22-28 May 2018 • Activity Report: Internal USAID SEA Meeting to Identify Topics & Create Content for Maluku Fisheries Forum, 15 March 2019 b. 2018 • Activity Report: Stakeholders Meeting to Socialize & Consult Piloting of Small-Scale Fisheries Logbook in Indonesia, 21 November 2018 • Training Report: Utilization of Marine Spatial plan (MSP) Spatial Data to Develop the Maluku Online Geoportal, 13-16 November 2018 • Data Meeting: Identifying Work/Partnership Mechanism in Data Use and Analysis of the Fish Landing Data, 6 November 2018 75 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV • Activity Report: National Commission Consultation & Review of Stock & Risk Assessment of Selected Fisheries in Indonesia, 15, & 18-19 October 2018 • Development of an Economic Model to Optimize Investment Decisions in Support of Marine Protected Areas, 30 September 2018 • Rapid Assessment of Tourism and Fisheries in Lease Islands, Central Maluku, 30 September 2018 • Tourism and Fisheries Rapid Assessment of Sawai Bay MPA, Central Maluku, 29 September 2018 • Rapid Assessment: Tourism and Fisheries in Mare Island, March 2018, Submitted: 29 September 2019 • Workshops on Preparation on Fisheries Frame Survey and Fish Stock and Fisheries Status in some Fishery Management Areas (FMAs), 16-18 July 2018 • Behavior Change Communication and Champion Training: “Influencing Through Awareness Raising”, 14-18 May 2017 • Activity Report: Fishery Implementation Coordination Workshop with West Papua Stakeholders, 1-2 March 2018 • Activity Report: The Workshop on Fishing Logbook Improvement and Development for Small￾Scale Fishers, 22-23 February 2018 • Advanced Training of PSM Inspector to Implement PSM Agreement, 16-25 January 2018 • An Overview of Tourism and Reef & Demersal Fisheries Sectors in Indonesia and a Rapid Assessment of the Tourism & Fisheries Sectors in Morotai, North Maluku, Indonesia, January 2018 • Technical Report TA2-16A: A Rapid Ecological Assessment (REA) for Oceanic Cetaceans in the Banda and Ceram Seas, Maluku, Indonesia(FAM 714 and 715), a Critical Habitat for Pygmy blue Whales (Balaenoptera musculus brevicauda), 15-24 October 2017 • Technical Report: Study Tour to Western Pacific Fishery Management Council, Hawaii, 17-29 September 2017 c. 2017 • Activity Report: Baseline Survey of Fisheries and Mangrove Resources at Teluk Bentuni to Support Marine Conservation Area Initiation, 22-29 September 2017 • Activity Report: Indicator Data Collection to Support Ecosystem Approach to Fisheries Management in Sawai Bay, 12-25 September 2017 • Final Report: Perception Survey of marine Conservation and Sustainable Fisheries Management in Maluku, North Maluku and West Papua for Communications, July 2017 • Workshop Report: Putting the USAID SEA Integrated Communication and Coordination Plan into Practice, 19-20 June 2017 • Activity Report: Scoping Trip to Bintuni Bay and South Sorong, 12-22 June 2017 • Technical Report: Review of Marine Ecosystems Services and Valuation Methodologies for the USAID SEA Project, 2 June 2017 • Activity Report: Rapid Assessment Survey of MPA, Initiation and Fisheries Data Collection at Bintuni Bay, 8-23 May 2017 • Activity Report: Base Line Report, Data Survey (Ecology, Fisheries, and Tourism) at Sawai Bay 2-11 May 2017 • Baseline Report, Teluk Sawai, Maluku, Province: Ecology, Fisheries, Social Status, 2-11 May 2017 • Activity Report: Base Line Data Survey (Ecology, Fisheries and Tourism) at Sawai Bay, Seram Utara, Maluku, 2-10 May 2017 76 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV • Activity Report: Field Orientation to Develop Research Design of New MPA Initiation at Sawai Bay (Preparing Base Line Data Survey), 12-15 April 2017 • WWF Site ID Field Orientation MPA Initiation Sawai Bay 12-15 April17 • Training Report: MPA 101 Training and Competency Examination in North Maluku, 10-17 April 2017 • Workshop Report: Stock Assessment Workshop and Methodology, 28 February – 1 March 2017 • Activity Report: Scoping and Rapid Assessment in Morotai Island, 20-26 February 2017 • Activity Report: Scoping Study at Morotai Islands in North Maluku, 19-26 February 2017 • Activity Report: Scoping Study, Buano Island, 7-12 February 2017 • Activity Report: Coordination Among Related Government Agencies to Accelerate SIMKADA Implementation, 30-31 January 2017 • Activity Report: Costal and Marine Resources Management Capacity Need Assessment in Maluku and North Maluku Province, 13-26 February 2017 • Activity Report: Stock Assessment Focus Group Discussion, 28 December 2016 • Technical Report: A Rapid Ecological Assessment (REA) for Marine Mammals in the Banda and Ceram Seas, 5-15 November 2016 • Technical Report: Fishery Scoping Trip, 31 August to 7 September 2016 5. Outreach Communication Plan a. Integrated Communication and Coordination Plan, March 2016 – March 2017 b. From Awareness Raising to Behavior Change: Considerations in support of Strategic Approach #1 6. Progress Reports a. Annual Reports • Annual Report, April 2016 – September 2017 • Annual Report, October 2017 – September 2018 • Annual Report, October 2018 – September 2019 b. Quarterly Reports • Quarterly Report, Start-Up and Mobilization Period, April-June 2016 • Quarterly Report, October – December 2016 (Bahasa version) • Quarterly Report, Second Quarter of FY 2017, January – March 2017 • Quarterly Report, Third Quarter of FY 2017, April – June 2017 • Quarterly Report, First Quarter of FY 2018, October – December 2017 • Quarterly Report, Second Quarter of FY 2018, January – March 2018 • Quarterly Report, Third Quarter of FY 2018, April – June 2018 • Quarterly Report, First Quarter of FY 2019, October – December 2018 • Quarterly Report, Second Quarter of FY 2019, January – March 2019 7. Work Plan a. FY20/21 18-Month Work Plan, 1 October 2019 to 22 March 2021 77 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV b. Third Annual Work Plan, 1 October 2018 to 30 September 2019 c. Second Annual Work Plan, 1 October 2017 to 30 September 2018 d. First Annual Work Plan, March 2016 to September 2017 8. Indicator Reporting Manual, November 2018 9. Monitoring, Evaluation and Learning (ME&L) Plan, Fifth Revision: September 28, 2018 (Updated) 10. Monitoring, Evaluation and Learning (ME&L) Plan, Third Revision: 21 December 2017 11. Decree of the Director General of Marine, Coasts and Small Islands Number KEP.44/KP3K/2012 Concerning Technical Guidelines for Evaluating the Management Effectiveness of Aquatic, Coasts and Small Islands Conservation Areas (E-KKP3K) 12. Progress Report: Behavior Change Practices at Mayalibit Bay and Dampier Strait, January 2019 13. Final Report: Improving Food Security and Livelihoods through Rights-based Management of Coastal Fisheries in West Papua, August 2019 14. Final Technical Report: Feed for the Future through Community Based Business Development in North Maluku and West Papua Province, 30 June 2019 15. Activity Report: Rapid Baseline Behavior Assessment of Villages in the South Sorong Marine Protected Area, 11-18 March 2019 78 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV ANNEX G. KII GUIDE QUESTIONS SUSTAINABLE ECOSYSTEMS ADVANCED (SEA) PROJECT Midterm Performances Evaluation (MTPE) KII Guide Questions KII Tool 1.a. USAID/COR/SEA Project-COP/DCOP and ME&L Specialist KII Guide Questions for: a) USAID Indonesia - USAID/COR b) SEA Project - COP/DCOP and ME&L Specialist, and other technical staff at the national level. Main Focus: Overall project management, coordination, communication, and ME&L in the following aspects: SA1 - Create demand thru awareness raising and advocacy; SA2 - Improve ecosystem management of FMA-715 and MPAs; SA2-TA1- Sustainable Fisheries thru ecosystem approach to fishery management (EAFM); SA2-TA2 - MPA Management; SA2-TA3 - MSP (Marine Spatial Planning); SA2-TA4 - Law Enforcement; SA3 - Increase incentives for marine stewardship thru non-fishing livelihood activities, and more secure tenure for fisheries resources; SA4 - Advance policy and regulations; and SA5 - Institutionalize training and capacity building. 1. Introduction and Background to the SEA Project MTPE Introduction of KII Interviewer, and Documenter cum interpreter Good day! My name is ___________, KII Interviewer, and my companion is _____________, KII Documenter, of the Pacific Rim Innovation and Management Exponents, Inc. (PRIMEX). We are part of the MTEP Evaluation Team, commissioned by USAID/Indonesia to conduct an evaluation of the USAID SEA Project. Our team in particular is conducting key informant interview (KII) with officials and technical staff of SEA Project (Tetra Tech ARD), MMAF, and representatives of selected Project partners/subcontractors who are involved and are knowledgeable about SEA Project activities. The KII is important to achieve the MTPE objectives. And we would like to seek your understanding for this interview, which may take about one and a half (1.5) hours to two (2) hours. At this early stage, we wish to express our sincerest appreciation to you for participating in this KII session. Protocol on KII session ✓ It is important that the key informant is provided with enough time to express/expound his/her answers. But be sure to focus on the subject of interview. 79 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV ✓ Confidentiality: Assure the respondent that all the data and information to be provided will be treated with utmost confidentiality and will be used mainly to undertake an objective assessment of the SEA Project’s performance. The name of the respondent will not, in anyway, be associated with any specific answers/comments. All the data and information will be collated and analyzed, together or in combination with other data and information collected from other KII respondents. ✓ There will be NO right or wrong answer. Encourage the respondent to feel free and be totally honest and objective in the answers or comments/opinion to be provided. ✓ The FGD session will be recorded; ask permission from the respondents if he/she will allow recording of her/his responses for documentation purposed. ✓ Before the KII begin, ask the respondent for any questions, concerns, or clarifications. 2. Basic Information about the KII Respondent Name of Respondent: Position: Agency/Office and Address Contact Information: Telephone No. Mobile No. E-mail Ad Note: Make sure to get the contact addresses (email and contact number of key informant for possible follow-up communications/calls for clarification on his/her responses). 3. Main Body of Guide Questions Questions Responses/Evidences Key Informant #1 Key Informant #2 EQ1: Under what conditions are the regulations and/or tools being introduced by USAID SEA to relevant authorities leading to the adoption of these policies, regulations, and tools? 1.1 How did the SEA Project create demand through awareness raising and advocacy of numerous marine and fisheries policies it developed? (Possible probe question: How did the SEA Project inform the public and partners at the national, provincial, and community relative to these approaches, policies, regulations, and tools?). Please mention the specific policies, regulations, approaches and tools. 1.2 Are these policies, regulations, and tools that have been authorized effectively implemented at the national, provincial, and community levels? If Yes, in what ways, 80 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences Key Informant #1 Key Informant #2 and what have been the results? If No, what have been the reasons? 1.3 What support or assistance has the SEA Project provided to facilitate the adoption of these policies and regulations? How effective has this support been in accomplishing and facilitating adoption? Why has the SEA Project’s support for adoption of policies and regulations been effective/or not effective? 1.4 What are the conditions that facilitate the adoption (authorization) of these policies, regulations and tools? Please provide answers for each policy, approach, regulation, and tool. 1.5 What are the conditions or factors that hinder or limit the progression of these policies, regulations and tools to reach the final stage of authorization/ adoption? Please provide answers for each policy, approach, regulation, and tool. 1.6 Are these regulations and/ or tools introduced by USAID SEA Project and adopted by relevant authorities lead to changing the following aspects? 1.6.1 Increased the number of people demonstrating behaviors that contribute to biodiversity conservation, as a result of implementing the strategy of “creating demand through awareness raising advocacy? 1.6.2 Increased the number of people exposed to USAID SEA Project messages through communication and media outreach? 1.6.3 Increased the number of champions organized and enabled to deliver messages? 1.6.4 Increased the number of people exhibiting behavior change towards biodiversity conservation and marine sustainable practices? 1.6.5 Increased the number of forums or official groups established and functional for awareness raising and advocacy. 81 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences Key Informant #1 Key Informant #2 EQ 2: Under what conditions do people trained to manage marine and fisheries resources and/or advocate for sustainable management of these resources, as a result of USAID SEA interventions, apply their skills in marine and fisheries resources management or advocacy? 2.1 How did the USAID SEA Project intervene to train and inform stakeholders and partners and apply their skills learned to manage marine and fisheries resources and advocacy? 2.2 How effective is the USAID SEA Project support in facilitating the application of knowledge and skills acquired? 2.3 In what ways do people and partners apply their learnings in marine fisheries management or advocacy? 2.4 What are the factors or conditions that facilitate/assist people and partners apply their learnings? 2.5 Are there people and partners trained/informed that did not apply their learnings and knowledge from these trainings? If Yes, what have been the reasons? 2.6 What are the factors or conditions that make it difficult for trained people to apply their acquired knowledge and skills? EQ3: To what extent do non‐fishing livelihood activities introduced or strengthened by USAID SEA change behavior and reduce threats and improve marine conservation (e.g., Livelihood program in Mare, Ternate, Galo‐ Galo, and Morotai)? Under what conditions are these activities effective? 3.1 How was the strategy “SA3 - Increase Incentives through non-fishing livelihood activities implemented? 3.2 What types of non-fishing livelihood activities were introduced or supported by SEA Project? Have they been effective in increasing the following? If Yes, in what ways and what have been the results? If No, what have been the reasons? 3.2.1 Increase in the number of public￾private partnerships with focused on sustainable fisheries and marine conservation with leveraged resources quantified? 3.2.2 Increase in the number of people receiving social, ecological, and economic benefits derived from conservation and sustainable use of marine resources? 82 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences Key Informant #1 Key Informant #2 EQ 4: To what extent do USAID‐supported fishermen with more secure marine tenure in fisheries apply sustainable fishing practices and reduce destructive fishing practices (e.g., Territorial Use Rights for Fishing [TURF] in Raja Ampat and Local Marine Management Area [LMMA] in Maluku)? Under what conditions are these activities effective? 4.1 How did USAID SEA Project supported fishermen with more secure tenure or managed access to fisheries resources (e.g. LMMA, CFA, TURF, and TCBFMA)? 4.2 How effective are the secure tenure or managed access activities? If Yes, in what ways, and what have been the results? If No, what have been the reasons? Which of these tenure activities (LMMA, CFA, TURF, and TCBFMA) are effective in applying sustainable fishing practices and reduction of destructive fishing practices? 4.3 Under what conditions are the secure tenure or managed access to fisheries resources effective or not effective? (Probing question: Can you cite evidences of effectiveness or ineffectiveness of each tenure instrument?). EQ 5: Given that USAID SEA addresses key marine threats and drivers through a complex project comprised of multiple components / strategic approaches implemented by a large number of partners, how well are all the components /approaches working together? What aspects of overall management are working well and what can be improved? To what extent are there capacity gaps among the partners, which might be hindering effective implementation? 5.1 What actions has the project taken to maximize synergies? How effectively are all the components and approaches of the SEA Project working together? What additional actions need to be taken to maximize synergies? 5.2 How effective has the SEA Project been in coordinating the implementation of the different project components and approaches? 5.3 Can you cite specific evidences where all these components/ approaches of the SEA Project are working together to achieve marine conservation? Are all these effective when working together? If Yes, in what ways, and what have been the results? If No, what have been the reasons? 5.4 What aspects of overall management are working well and what can be improved? 83 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences Key Informant #1 Key Informant #2 5.5 Are there capacity gaps among partners which might be hindering effective implementation? What are these capacity gaps (per partner)? What did SEA Project do to address these gaps? Are these actions effective in arresting/responding to the gaps. What further recommendations or courses of action to take to address these gaps? 5.6 How effective are relationships /working arrangements between USAID SEA Project and subcontractors, and among subcontractors? What other areas could be improved? 5.7. What recommendation/ suggestion or course of action to take to improve more (or make more successful/ effective) the overall implementation of the SEA Project? EQ 6: Based on the answers above about the conditions under which the USAID SEA project strategies are or are not effective, what are the recommended areas (thematic and/or geographic) of expansion/contraction for USAID? 6.1 Which of the project’s strategies are still relevant and effective at this point in time (mid-term implementation) in achieving the goals of the SEA project? What are the strategies that are still relevant? Why? What are strategies that are not relevant? Why 6.2 Which project strategies (thematic and/or geographic) need changes/improvement? Why? 6.3 What are the areas where the project needs to either expand or contract its work in relation to specific strategies and/or geographic scope? Why? 6.4 What actions are needed to improve the overall implementation of the SEA Project? Why? 4. Closing/Sign-Off Questions Again, assure the respondent that all information obtained shall remain confidential and will not be used for purposes other than what have stated. Finally, ask permission from the Key Informant for any possible follow-up communications with them for clarifications or additional inputs to shed more lights on specific topics discussed during the KII. 84 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Say “Thank You” before you leave. Schedule of KII: Date of Interview: Time Started: Time Ended: Venue of KII: Name of Facilitator: Name of Documenter: 85 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV SUSTAINABLE ECOSYSTEMS ADVANCED (SEA) PROJECT Midterm Performances Evaluation (MTPE) KII Guide Questions KII Tool 1.b: MMAF - Cooperation and Public Relation Bureau (Biro Keriosamadan Humos - Kermas) KII Guide Questions for: a) MMAF – Cooperation and Public Relation Bureau (Biro Keriosamadan Humos - Kermas) Main Focus: SA1 - Create demand through awareness raising and advocacy 1. Introduction and Background to the SEA Project MTPE Introduction of KII Interviewer, and Documenter cum interpreter Good day! My name is ___________, KII Interviewer, and my companion is _____________, KII Documenter, of the Pacific Rim Innovation and Management Exponents, Inc. (PRIMEX). We are part of the MTEP Evaluation Team, commissioned by USAID/Indonesia to conduct an evaluation of the USAID SEA Project. Our team in particular is conducting key informant interview (KII) with officials and technical staff of SEA Project (TETRA TECH), MMAF, and representatives of selected Project partners/subcontractors, who are involved and are knowledgeable about SEA Project activities. The KII is important to achieve the MTPE objectives. And we would like to seek your understanding for this interview, which may take about one and a half (1.5) hours to two (2) hours. At this early stage, we wish to express our sincerest appreciation to you for participating in this KII session. Protocol on KII session ✓ It is important that the key informant is provided with enough time to express/expound his/her answers. But be sure to focus on the subject of interview. ✓ Confidentiality: Assure the respondent that all the data and information to be provided will be treated with utmost confidentiality and will be used mainly to undertake an objective assessment of the SEA Project’s performance. The name of the respondent will not, in anyway, be associated with any specific answers/comments. All the data and information will be collated and analyzed, together or in combination with other data and information collected from other KII respondents. ✓ There will be NO right or wrong answer. Encourage the respondent to feel free and be totally honest and objective in the answers or comments/opinion to be provided. ✓ The FGD session will be recorded; ask permission from the respondents if he/she will allow recording of her/his responses for documentation purposed. ✓ Before the KII begin, ask the respondent for any questions, concerns, or clarifications. 86 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 2. Basic Information about the KII Respondent Name of Respondent: Position: Agency/Office and Address Contact Information: Telephone No. Mobile No. E-mail Ad Note: Make sure to get the contact addresses (email and contact number of key informant for possible follow-up communications/calls for clarification on his/her responses). 3. Understanding the Role of MMAF - Cooperation and Public Relation Bureau (Biro Keriosamadan Humos - Kermas) in USAID SEA Project What is your Bureau’s main goal or primary interest in the marine and fisheries resources at the national, provincial, and community? What were your activities relative to these resources before the USAID SEA Project? What activities are you doing/implementing at the national, provincial, community in relation to USAID SEA Project? National National Provincial Provincial Community Community 4. Main Body of Guide Questions Questions Responses/Evidences At the National Level At the Provincial Level EQ 1: Under what conditions are the regulations and/or tools being introduced by USAID SEA to relevant authorities leading to the adoption of these policies, regulations, and tools? 1.1 What is your office’s involvement in creating demand through awareness raising and advocacy of numerous marine and fisheries policies developed by the SEA Project? (Possible probe question: How did your office inform the public and partners at the national, provincial, and community relative to these SEA project approaches, policies, regulations, and tools?). Please mention the specific policies, regulations, approaches and tools. 1.2 Are these policies, regulations, and tools that have been authorized effectively implemented at the national, provincial, and community levels? If Yes, in 87 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level what ways, and what have been the results? If No, what have been the reasons? 1.3 What are the conditions that facilitate the adoption (authorization) of these policies, regulations and tools? Please provide answers for each policy, approach, regulation, and tool. 1.4 What are the conditions or factors that hinder or limit the progression of these policies, regulations and tools to reach the final stage of authorization/ adoption? Please provide answers for each policy, approach, regulation, and tool. 1.5 Are these regulations and/ or tools introduced by USAID SEA Project and adopted by relevant authorities lead to changing the following aspects? 1.5.1 Increased the number of people demonstrating behaviors that contribute to biodiversity conservation, as a result of implementing the strategy of “creating demand through awareness raising advocacy? If yes, how and under what conditions are these achieved? If no, what have been the reasons? 1.5.2 Increased the number of people exposed to USAID SEA Project messages through communication and media outreach? If yes, how and under what conditions are these achieved? If no, what have been the reasons? 1.5.3 Increased the number of champions organized and enabled to deliver messages? If yes, how and under what conditions are these achieved? If no, what have been the reasons? 1.5.4 Increased the number of people exhibiting behavior change towards biodiversity conservation and marine sustainable practices? If yes, how and under what conditions are these achieved? If no, what have been the reasons? 1.5.5 Increased the number of forums or official groups established and functional for awareness raising and advocacy? If yes, how and under what conditions are these achieved? If no, what have been the reasons? EQ 2: Under what conditions do people trained to manage marine and fisheries resources and/or advocate for sustainable management of these resources, as a result of USAID SEA interventions, apply their skills in marine and fisheries resources management or advocacy? 2.1 How is your office making use of the USAID SEA Project interventions to train and inform people and partners to manage marine and fisheries resources and/or advocate for sustainable management of these resources to enable them to apply their skills 88 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level learned in marine fisheries management or advocacy? 2.2 What are the factors or conditions that facilitate/ assist people and partners apply their learnings? 2.3 What are the factors or conditions that make it difficult for trained people to apply their acquired knowledge and skills? EQ3: To what extent do non‐fishing livelihood activities introduced or strengthened by USAID SEA change behavior and reduce threats and improve marine conservation (e.g., Livelihood program in Ternate, North Maluku, Galo‐Galo, and Morotai)? Under what conditions are these activities effective? 3.1 What are the messages/ information that SEA Project disseminate and support to help change the behavior of people and partners in non-fishing livelihood activities towards reduction of threats and improvement in marine conservation? 3. 2 How effectively do these exposures to SEA Project messages help in changing the behavior of the people and partners in non-fishing livelihood activities towards reduction of threats and improvement in marine conservation? 3.3 What conditions or factors contribute in pursuing your office’s efforts in awareness raising and advocacy of marine conservation and sustainable management in changing behavior towards reduction of threats and improvement in marine conservation in non-fishing livelihood activities areas? EQ 4: To what extent do USAID‐supported fishermen with more secure marine tenure in fisheries apply sustainable fishing practices and reduce destructive fishing practices (e.g., Territorial Use Rights for Fishing [TURF] in Raja Ampat and Local Marine Management Area [LMMA] in Maluku)? Under what conditions are these activities effective? 4.1 What is your office’s involvement in supporting USAID SEA Project to provide secure tenure or managed access to fisheries resources (e.g. LMMA, CFA, TURF, and TCBFMA)? 4.2 How effective are the secure tenure or managed access activities? If Yes, in what ways, and what have been the results? If No, what have been the reasons? Which of these tenure activities (LMMA, CFA, TURF, and TCBFMA) are effective in applying sustainable fishing practices and reduction of destructive fishing practices? 4.3 Under what conditions are the secure tenure or managed access to fisheries resources effective or not effective? 89 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level (Probing question: Can you cite evidences of effectiveness or ineffectiveness of each tenure instrument?). EQ 5: Given that USAID SEA addresses key marine threats and drivers through a complex project comprised of multiple components / strategic approaches implemented by a large number of partners, how well are all the components /approaches working together? What aspects of overall management are working well and what can be improved? To what extent are there capacity gaps among the partners, which might be hindering effective implementation? 5.1 What actions has the project taken to maximize synergies? How effectively are all the components and approaches of the SEA Project working together? What additional actions need to be taken to maximize synergies? 5.2 How effective has the SEA Project been in coordinating the implementation of the different project components and approaches? 5.3 What is your office’s involvement in the USAID SEA Project to ensure that all the components/approaches are effectively working together? How and why? 5.4 What aspects of overall management of the project are working well and what can be improved? 5.5 Are there capacity gaps among partners which might be hindering effective implementation? What are these capacity gaps (per partner)? What did SEA Project do to address these gaps? Are these actions effective in arresting/responding to the gaps. What further recommendations or courses of action to take to address these gaps? 5.6 How effective are relationships / working arrangements between USAID SEA Project and subcontractors, and among subcontractors? What other areas could be improved? 5.7. What recommendation / suggestion or course of action to take to improve more (or make more successful/effective) the overall implementation of the SEA Project? EQ6: Based on the answers above about the conditions under which the USAID SEA project strategies are or are not effective, what are the recommended areas (thematic and/or geographic) of expansion/contraction for USAID? 6.1 Which of the project’s strategies are still relevant and effective at this point in time (mid-term implementation) in achieving the goals of the SEA project? What are the strategies that are still relevant? Why? What are strategies that are not relevant? Why 6.2 Which project strategies (thematic and/or 90 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level geographic) need changes/improvement? Why? 6.3 What are the areas where the project needs to either expand or contract its work in relation to specific strategies and/or geographic scope? Why? 6.4 What actions are needed to improve the overall implementation of the SEA Project? Why? 5. Closing/Sign-Off Questions Again, assure the respondent that all information obtained shall remain confidential and will not be used for purposes other than what have stated. Finally, ask permission from the Key Informant for any possible follow-up communications with them for clarifications or additional inputs to shed more lights on specific topics discussed during the KII. Say “Thank You” before you leave. Schedule of KII: Date of Interview: Time Started: Time Ended: Venue of KII: Name of KII Interviewer: Name of Documenter: 91 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV SUSTAINABLE ECOSYSTEMS ADVANCED (SEA) PROJECT Midterm Performances Evaluation (MTPE) KII Guide Questions KII Tool 1.c: MMAF – Fisheries Resource Management Directorate (PSDI) KII Guide Questions for: a) MMAF – Fisheries Resource Management Directorate (PSDI) Main Focus: SA2-TA1 – Sustainable Fisheries through Ecosystem Approach to Fishery Management (EAFM) 1. Introduction and Background to the SEA Project MTPE Introduction of KII Interviewer, and Documenter cum interpreter Good day! My name is ___________, KII Interviewer, and my companion is _____________, KII Documenter, of the Pacific Rim Innovation and Management Exponents, Inc. (PRIMEX). We are part of the MTEP Evaluation Team, commissioned by USAID/Indonesia to conduct an evaluation of the USAID SEA Project. Our team in particular is conducting key informant interview (KII) with officials and technical staff of SEA Project (TETRA TECH), MMAF, and representatives of selected Project partners/subcontractors, who are involved and are knowledgeable about SEA Project activities. The KII is important to achieve the MTPE objectives. And we would like to seek your understanding for this interview, which may take about one and a half (1.5) hours to two (2) hours. At this early stage, we wish to express our sincerest appreciation to you for participating in this KII session. Protocol on KII session ✓ It is important that the key informant is provided with enough time to express/expound his/her answers. But be sure to focus on the subject of interview. ✓ Confidentiality: Assure the respondent that all the data and information to be provided will be treated with utmost confidentiality and will be used mainly to undertake an objective assessment of the SEA Project’s performance. The name of the respondent will not, in anyway, be associated with any specific answers/comments. All the data and information will be collated and analyzed, together or in combination with other data and information collected from other KII respondents. ✓ There will be NO right or wrong answer. Encourage the respondent to feel free and be totally honest and objective in the answers or comments/opinion to be provided. ✓ The FGD session will be recorded; ask permission from the respondents if he/she will allow recording of her/his responses for documentation purposed. ✓ Before the KII begin, ask the respondent for any questions, concerns, or clarifications. 92 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 2. Basic Information about the KII Respondent Name of Respondent: Position: Agency/Office and Address Contact Information: Telephone No. Mobile No. E-mail Ad Note: Make sure to get the contact addresses (email and contact number of key informant for possible follow-up communications/calls for clarification on his/her responses). 3. Understanding the Role of MMAF – Fisheries Resource Management Directorate (PSDI) What is your Office’s main goal or primary interest in the marine and fisheries resources at the national, provincial, and community? What were your activities relative to these resources before the USAID SEA Project? What activities are you doing/implementing at the national, provincial, community in relation to USAID SEA Project? National National Provincial Provincial Community Community 4. Main Body of Guide Questions Questions Responses/Evidences At the National Level At the Provincial Level EQ 1: Under what conditions are the regulations and/or tools being introduced by USAID SEA to relevant authorities leading to the adoption of these policies, regulations, and tools? 1.1 What are the supports of USAID SEA Project for the development and advancement of policies, regulations and/or tools related to a range of government frameworks on EAFM/sustainable fishing. Did your office support these efforts? If yes, how did your office support these efforts? If no, what have been the reasons? 1.2 Are these policies, regulations and tools that have been authorized are effectively implemented to improve ecosystem management in marine and fisheries resources at the national, provincial, and community? If Yes, in what ways, and what have been the results? If No, what have been the reasons? 93 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level 1.3 Are these regulations and/or tools introduced by USAID SEA Project assistance and adopted by relevant authorities lead to increase in number of hectares of biological significance and/or natural resources under improved natural resource management applying EAFM? If yes/no, why? 1.4 What are the conditions that facilitate the adoption (authorization) of these policies, regulations and tools, applying EAFM? Please provide answers for each policy, approach, regulation, and tool. 1.5 What are the conditions or factors that hinder or limit the progression of these policies, regulations and tools to reach the final stage of authorization/adoption applying EAFM? Please provide answers for each policy, approach, regulation, and tool. EQ 2: Under what conditions do people trained to manage marine and fisheries resources and/or advocate for sustainable management of these resources, as a result of USAID SEA interventions, apply their skills in marine and fisheries resources management or advocacy? 2.1 How effective are the USAID SEA Project support through EAFM application in facilitating/enhancing the skills and learnings on natural resource management of stakeholders and partners? 2.2 How effective are the USAID SEA Project support through EAFM approach in helping stakeholders and partners in applying their learnings on natural resource management or advocacy? In what ways do stakeholders and partners apply their learnings? 2.3 What are the conditions that facilitate the adoption and institutionalization of EAFM? What are the conditions or factors that hinder or limit the adoption and institutionalization of EAFM? 2.4 With the introduction of EAFM framework by the USAID SEA Project, what factors or conditions emerges that facilitate/assist people and partners apply their learnings? 2.5. In an EAFM framework, what are the factors or conditions that make it difficult for trained people to apply their acquired knowledge and skills? 94 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level EQ 3: To what extent do non‐fishing livelihood activities introduced or strengthened by USAID SEA change behavior and reduce threats and improve marine conservation (e.g., Livelihood program in Mare, Ternate, Galo‐Galo, and Morotai)? Under what conditions are these activities effective? 3.1 What is your office’s role in supporting the SEA Project implement EAFM in non-fishing livelihood activities? 3.2 Did USAID SEA applying EAFM strengthened non-fishing livelihood activities in North Maluko, Galo-Galo, and Morotai that lead to change behavior and reduction of threats and improvement of marine conservation? If yes, how and why? If no, why? 3.3 What are the conditions or factors that contribute to the effectivity of non-fishing livelihood activities to change behavior of people and partners for reduction of threats and sustainable marine conservation? 3.4 What are the conditions or factors that diminish the effectivity of these non-fishing livelihood activities to reduce threats and improve marine conservation in the selected project sites? EQ 4: To what extent do USAID‐supported fishermen with more secure marine tenure in fisheries apply sustainable fishing practices and reduce destructive fishing practices (e.g., Territorial Use Rights for Fishing [TURF] in Raja Ampat and Local Marine Management Area [LMMA] in Maluku)? Under what conditions are these activities effective? 4.1What is your office’s involvement in supporting USAID SEA EAFM Project to provide secure tenure or managed access to fisheries resources (e.g. LMMA, CFA, TURF, and TCBFMA)? 4.2 How effective are the secure tenure or managed access activities in applying sustainable fishing practices and reduction of destructive fishing practices? If Yes, in what ways, and what have been the results? If No, what have been the reasons? Which of these tenure activities (LMMA, CFA, TURF, and TCBFMA) are effective in applying sustainable fishing practices and reduction of destructive fishing practices? 4.3 Under what conditions are the secure tenure or managed access to fisheries resources effective or not effective? (Probing question: Can you cite evidences of effectiveness or ineffectiveness of each tenure instrument?). 95 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level EQ 5: Given that USAID SEA addresses key marine threats and drivers through a complex project comprised of multiple components / strategic approaches implemented by a large number of partners, how well are all the components /approaches working together? What aspects of overall management are working well and what can be improved? To what extent are there capacity gaps among the partners, which might be hindering effective implementation? 5.1 What does the USAID SEA EAFM provide to ensure that all the components/approaches of the SEA Project are implemented or working together in the project sites to achieve marine conservation? 5.2 How effective has the SEA Project through EAFM been in coordinating the implementation of the different project components and approaches? 5.3 Can you cite specific evidences where all these components/approaches of the SEA Project are working together to achieve marine conservation? Are all these effective when working together? If Yes, in what ways, and what have been the results? If No, what have been the reasons? 5.4 What is your office’s involvement in the USAID SEA Project to ensure that all the components/approaches are effectively working together? How and why? 5.5What aspects of overall management of the SEA Project are working well and what can be improved? 5.6 Are there capacity gaps among partners which might be hindering effective implementation? What are these capacity gaps (per partner)? What did SEA Project do to address these gaps? Are these actions effective in arresting/responding to these gaps. What further recommendations or courses of action to take to address these gaps? 5.7 How effective are relationships/working arrangements between USAID SEA Project and subcontractors, and among subcontractors? What other areas could be improved? 5.8. What recommendation / suggestion or course of action to take to improve more (or make more successful/effective) the overall implementation of the SEA Project? EQ 6: Based on the answers above about the conditions under which the USAID SEA project strategies are or are not effective, what are the recommended areas (thematic and/or geographic) of expansion/contraction for USAID? 96 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level 6.1 Which of the project’s strategies are still relevant and effective at this point in time (mid-term implementation) in achieving the goals of the SEA project? What are the strategies that are still relevant? Why? What are strategies that are not relevant? Why 6.2 Which project strategies (thematic and/or geographic) need changes/improvement? Why? 6.3 What are the areas where the project needs to either expand or contract its work in relation to specific strategies and/or geographic scope? Why? 6.4 What actions are needed to improve the overall implementation of the SEA Project? Why? 5. Closing/Sign-Off Questions Again, assure the respondent that all information obtained shall remain confidential and will not be used for purposes other than what have stated. Finally, ask permission from the Key Informant for any possible follow-up communications with them for clarifications or additional inputs to shed more lights on specific topics discussed during the KII. Say “Thank You” before you leave. Schedule of KII: Date of Interview: Time Started: Time Ended: Venue of KII: Name of KII Interviewer: Name of Documenter: 97 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV SUSTAINABLE ECOSYSTEMS ADVANCED (SEA) PROJECT Midterm Performances Evaluation (MTPE) KII Guide Questions KII Tool 1.d: MMAF – Marine Protected Area and Biodiversity Directorate (KKHL) KII Guide Questions for: a) MMAF – Marine Protected Area Biodiversity Directorate (KKHL) Main Focus: SA2-TA2 – Sustainable Fisheries through MPA Management 1. Introduction and Background to the SEA Project MTPE Introduction of KII Interviewer, and Documenter cum interpreter Good day! My name is ___________, KII Interviewer, and my companion is _____________, KII Documenter, of the Pacific Rim Innovation and Management Exponents, Inc. (PRIMEX). We are part of the MTEP Evaluation Team, commissioned by USAID/Indonesia to conduct an evaluation of the USAID SEA Project. Our team in particular is conducting key informant interview (KII) with officials and technical staff of SEA Project (TETRA TECH), MMAF, and representatives of selected Project partners/subcontractors, who are involved and are knowledgeable about SEA Project activities. The KII is important to achieve the MTPE objectives. And we would like to seek your understanding for this interview, which may take about one and a half (1.5) hours to two (2) hours. At this early stage, we wish to express our sincerest appreciation to you for participating in this KII session. Protocol on KII session ✓ It is important that the key informant is provided with enough time to express/expound his/her answers. But be sure to focus on the subject of interview. ✓ Confidentiality: Assure the respondent that all the data and information to be provided will be treated with utmost confidentiality and will be used mainly to undertake an objective assessment of the SEA Project’s performance. The name of the respondent will not, in anyway, be associated with any specific answers/comments. All the data and information will be collated and analyzed, together or in combination with other data and information collected from other KII respondents. ✓ There will be NO right or wrong answer. Encourage the respondent to feel free and be totally honest and objective in the answers or comments/opinion to be provided. ✓ The FGD session will be recorded; ask permission from the respondents if he/she will allow recording of her/his responses for documentation purposed. ✓ Before the KII begin, ask the respondent for any questions, concerns, or clarifications. 98 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 2. Basic Information about the KII Respondent Name of Respondent: Position: Agency/Office and Address Contact Information: Telephone No. Mobile No. E-mail Ad Note: Make sure to get the contact addresses (email and contact number of key informant for possible follow-up communications/calls for clarification on his/her responses). 3. Understanding the Role of MMAF – Marine Protected Area Biodiversity Directorate (KKHL) What is your Office’s main goal or primary interest in the marine and fisheries resources at the national, provincial, and community? What were your activities relative to these resources before the USAID SEA Project? What activities are you doing/implementing at the national, provincial, community in relation to USAID SEA Project? National National Provincial Provincial Community Community 4. Main Body of Guide Questions Questions Responses/Evidences At the National Level At the Provincial Level EQ 1: Under what conditions are the regulations and/or tools being introduced by USAID SEA to relevant authorities leading to the adoption of these policies, regulations, and tools? 1.1 What are the supports of USAID SEA Project for the development and advancement of policies, regulations and/or tools related to a range of government frameworks on MPA management/ sustainable fishing. Did your office support these efforts? If yes, how did your office support these efforts? If no, what have been the reasons? 1.2 Are these policies, regulations and tools that have been authorized effectively implemented to improve MPA management in marine and fisheries resources at the national, provincial, and community levels? If Yes, in what ways, and what have been the results? If No, what have been the reasons? 99 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level 1.3 Are these regulations and/or tools introduced by USAID SEA Project assistance and adopted by relevant authorities lead to increased in number of hectares of biological significance and/or natural resources under improved natural resource management applying MPA? If yes/no, why? 1.4 What are the conditions that facilitate the adoption (authorization) of these policies, regulations and tools, applying MPA Management? Please provide answers for each policy, approach, regulation, and tool. 1.5 What are the conditions or factors that hinder or limit the progression of these policies, regulations and tools to reach the final stage of authorization/adoption applying MPA management? Please provide answers for each policy, approach, regulation, and tool. EQ 2: Under what conditions do people trained to manage marine and fisheries resources and/or advocate for sustainable management of these resources, as a result of USAID SEA interventions, apply their skills in marine and fisheries resources management or advocacy? 2.1 How effective are the USAID SEA Project support through MPA management application in facilitating/enhancing the skills and learnings on natural resource management of stakeholders and partners? 2.2 How effective are the USAID SEA Project support through MPA management in helping stakeholders and partners in applying their learnings on natural resource management or advocacy? In what ways do stakeholders and partners apply their learnings? 2.3 What are the conditions that facilitate the adoption and institutionalization of MPA management? What are the conditions or factors that hinder or limit the adoption and institutionalization of MPA management? 2.4 With the introduction of MPA management by the USAID SEA Project, what factors or conditions emerges that facilitate/assist people and partners apply their learnings? 2.5. In an MPA management, what are the factors or conditions that make it difficult for trained people to apply their acquired knowledge and skills? 100 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level EQ 3: To what extent do non‐fishing livelihood activities introduced or strengthened by USAID SEA change behavior and reduce threats and improve marine conservation (e.g., Livelihood program in North Maluko, Galo‐galo, and Morotai)? Under what conditions are these activities effective? 3.1 What is your office’s role in supporting the SEA Project implement non-fishing livelihood activities in Marine Protected Areas (MPA Management)? How and why? 3.2 Did USAID SEA applying MPA management strengthened non-fishing livelihood activities in North Maluko, Galo-Galo, and Morotai that lead to change behavior and reduction of threats and improvement of marine conservation? If yes, how and why? If no, why? 3.3 What are the conditions or factors that contribute to the effectivity of non-fishing livelihood activities to change behavior of people and partners for reduction of threats and sustainable marine conservation? 3.4 What are the conditions or factors that diminish the effectivity of these non-fishing livelihood activities to reduce threats and improve marine conservation in the selected project sites? EQ 4: To what extent do USAID‐supported fishermen with more secure marine tenure in fisheries apply sustainable fishing practices and reduce destructive fishing practices (e.g., Territorial Use Rights for Fishing [TURF] in Raja Ampat and Local Marine Management Area [LMMA] in Maluku)? Under what conditions are these activities effective? 4.1What is your office’s involvement in supporting USAID SEA MPA management to provide secure tenure or managed access to fisheries resources (e.g. LMMA, CFA, TURF, and TCBFMA)? 4.2 How effective are the secure tenure or managed access activities in applying sustainable fishing practices and reduce destructive fishing practices in Marine Protected Areas (MPA)? If Yes, in what ways, and what have been the results? If No, what have been the reasons? Which of these tenure activities (LMMA, CFA, TURF, and TCBFMA) are effective in applying sustainable fishing practices and reduction of destructive fishing practices? 4.3 Under what conditions are the secure tenure or managed access to fisheries resources effective or not effective? 101 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level (Probing question: Can you cite evidences of effectiveness or ineffectiveness of each tenure instrument?). EQ 5: Given that USAID SEA addresses key marine threats and drivers through a complex project comprised of multiple components / strategic approaches implemented by a large number of partners, how well are all the components /approaches working together? What aspects of overall management are working well and what can be improved? To what extent are there capacity gaps among the partners, which might be hindering effective implementation? 5.1 What does the USAID SEA MPA management provide to ensure that all the components/approaches of the SEA Project are implemented or working together in the project sites to achieve marine conservation? 5.2 How effective has the SEA Project through MPA/EAFM been in coordinating the implementation of the different project components and approaches? 5.3 Can you cite specific evidences where all these components/approaches of the SEA Project are working together to achieve marine conservation? Are all these effective when working together? If Yes, in what ways, and what have been the results? If No, what have been the reasons? 5.4 What is your office’s involvement in the USAID SEA Project to ensure that all the components/approaches are effectively working together? How and why? 5.5 What aspects of overall management of the SEA Project are working well and what can be improved? 5.6 Are there capacity gaps among partners which might be hindering effective implementation of MPA? What are these capacity gaps (per partner)? What did SEA Project do to address these gaps? Are these actions effective in arresting/responding to these gaps. What further recommendations or courses of action to take to address these gaps? 102 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level 5.7 How effective are relationships/working arrangements between USAID SEA Project and subcontractors, and among subcontractors? What other areas could be improved? 5.8 What recommendation / suggestion or course of action to take to improve more (or make more successful/effective) the overall implementation of the SEA Project? EQ6: Based on the answers above about the conditions under which the USAID SEA project strategies are or are not effective, what are the recommended areas (thematic and/or geographic) of expansion/contraction for USAID? 6.1 Which of the project’s strategies are still relevant and effective at this point in time (mid-term implementation) in achieving the goals of the SEA project? What are the strategies that are still relevant? Why? What are strategies that are not relevant? Why 6.2 Which project strategies (thematic and/or geographic) need changes/improvement? Why? 6.3 What are the areas where the project needs to either expand or contract its work in relation to specific strategies and/or geographic scope? Why? 6.4 What actions are needed to improve the overall implementation of the SEA Project? Why? 5. Closing/Sign-Off Questions Again, assure the respondent that all information obtained shall remain confidential and will not be used for purposes other than what have stated. Finally, ask permission from the Key Informant for any possible follow-up communications with them for clarifications or additional inputs to shed more lights on specific topics discussed during the KII. Say “Thank You” before you leave. Schedule of KII: Date of Interview: Time Started: Time Ended: Venue of KII: Name of KII Interviewer: Name of Documenter: 103 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV SUSTAINABLE ECOSYSTEMS ADVANCED (SEA) PROJECT Midterm Performances Evaluation (MTPE) KII Guide Questions KII Tool 1.e: MMAF – Marine Spatial Planning Directorate (TRL) KII Guide Questions for: a) MMAF – Marine Spatial Planning Directorate (TRL) Main Focus: SA2-TA3 – Sustainable Fisheries through Marine Spatial Planning Management (MSP) 1. Introduction and Background to the SEA Project MTPE Introduction of KII Interviewer, and Documenter cum interpreter Good day! My name is ___________, KII Interviewer, and my companion is _____________, KII Documenter, of the Pacific Rim Innovation and Management Exponents, Inc. (PRIMEX). We are part of the MTEP Evaluation Team, commissioned by USAID/Indonesia to conduct an evaluation of the USAID SEA Project. Our team in particular is conducting key informant interview (KII) with officials and technical staff of SEA Project (TETRA TECH), MMAF, and representatives of selected Project partners/subcontractors, who are involved and are knowledgeable about SEA Project activities. The KII is important to achieve the MTPE objectives. And we would like to seek your understanding for this interview, which may take about one and a half (1.5) hours to two (2) hours. At this early stage, we wish to express our sincerest appreciation to you for participating in this KII session. Protocol on KII session ✓ It is important that the key informant is provided with enough time to express/expound his/her answers. But be sure to focus on the subject of interview. ✓ Confidentiality: Assure the respondent that all the data and information to be provided will be treated with utmost confidentiality and will be used mainly to undertake an objective assessment of the SEA Project’s performance. The name of the respondent will not, in anyway, be associated with any specific answers/comments. All the data and information will be collated and analyzed, together or in combination with other data and information collected from other KII respondents. ✓ There will be NO right or wrong answer. Encourage the respondent to feel free and be totally honest and objective in the answers or comments/opinion to be provided. ✓ The FGD session will be recorded; ask permission from the respondents if he/she will allow recording of her/his responses for documentation purposed. ✓ Before the KII begin, ask the respondent for any questions, concerns, or clarifications. 104 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 2. Basic Information about the KII Respondent Name of Respondent: Position: Agency/Office and Address Contact Information: Telephone No. Mobile No. E-mail Ad Note: Make sure to get the contact addresses (email and contact number of key informant for possible follow-up communications/calls for clarification on his/her responses). 3. Understanding the Role of MMAF – Marine Spatial Planning Directorate (TRL) What is your Office’s main goal or primary interest in the marine and fisheries resources at the national, provincial, and community? What were your activities relative to these resources before the USAID SEA Project? What activities are you doing/implementing at the national, provincial, community in relation to USAID SEA Project? National National Provincial Provincial Community Community 4. Main Body of Guide Questions Questions Responses/Evidences At the National Level At the Provincial Level EQ 1: Under what conditions are the regulations and/or tools being introduced by USAID SEA to relevant authorities leading to the adoption of these policies, regulations, and tools? 1.1 What are the supports of USAID SEA Project for the development and advancement of policies, regulations and/or tools related to a range of government frameworks on MSP management/ sustainable fishing. Did your office support these efforts? If yes, in what ways, and what have been the results? If no, what have been the reasons? 1.2 Are these policies, regulations and tools that have been authorized are effectively implemented to improve MSP management in marine and fisheries resources at the national, provincial, and community levels? If Yes, in what ways, and what have been the results? If No, what have been the reasons? 105 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level 1.3 Are these regulations and/or tools introduced by USAID SEA Project assistance and adopted by relevant authorities lead to increase in number of hectares of biological significance and/or natural resources under improved natural resource management applying MSP? If yes/no, why? 1.4 What are the conditions that facilitate the adoption (authorization) of these policies, regulations and tools, applying MSP Management? Please provide answers for each policy, approach, regulation, and tool. 1.5 What are the conditions or factors that hinder or limit the progression of these policies, regulations and tools to reach the final stage of authorization/adoption applying MSP management? Please provide answers for each policy, approach, regulation, and tool. EQ 2: Under what conditions do people trained to manage marine and fisheries resources and/or advocate for sustainable management of these resources, as a result of USAID SEA interventions, apply their skills in marine and fisheries resources management or advocacy? 2.1 How effective are the USAID SEA Project support through MSP management application in facilitating/enhancing the skills and learnings on natural resource management of stakeholders and partners? 2.2 How effective are the USAID SEA Project support through MSP management in helping stakeholders and partners in applying their learnings on natural resource management or advocacy? In what ways do stakeholders and partners apply their learnings? 2.3 What are the conditions that facilitate the adoption and institutionalization of MSP management? What are the conditions or factors that hinder or limit the adoption and institutionalization of MSP management? 2.4 With the introduction of MSP management by the USAID SEA Project, what factors or conditions emerges that facilitate/assist people and partners apply their learnings? 106 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level 2.5. In an MSP management, what are the factors or conditions that make it difficult for trained people to apply their acquired knowledge and skills? EQ 3: To what extent do non‐fishing livelihood activities introduced or strengthened by USAID SEA change behavior and reduce threats and improve marine conservation (e.g., Livelihood program in Ternate, North Maluku, Galo‐Galo, and Morotai)? Under what conditions are these activities effective? 3.1 What is your office’s role in supporting the SEA Project implement MSP Management in non-fishing livelihood activities? 3.2 Did USAID SEA applying MSP management strengthened non-fishing livelihood activities in Ternate, North Maluku, Galo￾Galo, and Morotai that lead to change behavior and reduction of threats and improvement of marine conservation? If yes, how and why? If no, why? 3.3 What are the conditions or factors that contribute to the effectivity of non-fishing livelihood activities to change behavior of people and partners for reduction of threats and sustainable marine conservation? 3.4 What are the conditions or factors that diminish the effectivity of these non-fishing livelihood activities to reduce threats and improve marine conservation in the selected project sites? EQ 4: To what extent do USAID‐supported fishermen with more secure marine tenure in fisheries apply sustainable fishing practices and reduce destructive fishing practices (e.g., Territorial Use Rights for Fishing [TURF] in Raja Ampat and Local Marine Management Area [LMMA] in Maluku)? Under what conditions are these activities effective? 4.1What is your office’s involvement in supporting USAID SEA MSP management to provide secure tenure or managed access to fisheries resources (e.g. LMMA, CFA, TURF, and TCBFMA)? 4.2 How effective are the secure tenure or managed access activities in applying sustainable fishing practices and reduce destructive fishing practices? If Yes, in what ways, and what have been the results? If No, what have been the reasons? Which of these tenure activities (LMMA, CFA, TURF, and TCBFMA) are effective in applying sustainable fishing practices and reduction of destructive fishing practices? 107 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level 4.3 Under what conditions are the secure tenure or managed access to fisheries resources effective or not effective? (Probing question: Can you cite evidences of effectiveness or ineffectiveness of each tenure instrument?). EQ 5. Given that USAID SEA addresses key marine threats and drivers through a complex project comprised of multiple components / strategic approaches implemented by a large number of partners, how well are all the components /approaches working together? What aspects of overall management are working well and what can be improved? To what extent are there capacity gaps among the partners, which might be hindering effective implementation? 5.1 What does the USAID SEA MSP management provide to ensure that all the components/approaches of the SEA Project are implemented or working together in the project sites to achieve marine conservation? 5.2 Can you cite specific evidences where all these components/approaches of the SEA Project are working together to achieve marine conservation? Are all these effective when working together? If Yes, in what ways, and what have been the results? If No, what have been the reasons? 5.3 If Yes, what is your office’s involvement in the USAID SEA to ensure that all the components/approaches are effectively working together? 5.4 What aspects of overall management of the SEA Project are working well and what can be improved? 5.5 Are there capacity gaps among partners which might be hindering effective implementation of MSP? What are these capacity gaps (per partner)? What did SEA Project do to address these gaps? Are these actions effective in arresting/responding to these gaps. What further recommendations or courses of action to take to address these gaps? 5.6 How effective are relationships/working arrangements between USAID SEA Project and subcontractors, and among 108 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level subcontractors? What other areas could be improved? EQ 6: Based on the answers above about the conditions under which the USAID SEA project strategies are or are not effective, what are the recommended areas (thematic and/or geographic) of expansion/contraction for USAID? 6.1 Which of the project’s strategies are still relevant and effective at this point in time (mid-term implementation) in achieving the goals of the SEA project? What are the strategies that are still relevant? Why? What are strategies that are not relevant? Why 6.2 Which project strategies (thematic and/or geographic) need changes/improvement? Why? 6.3 What are the areas where the project needs to either expand or contract its work in relation to specific strategies and/or geographic scope? Why? 6.4 What actions are needed to improve the overall implementation of the SEA Project? Why? 5. Closing/Sign-Off Questions Again, assure the respondent that all information obtained shall remain confidential and will not be used for purposes other than what have stated. Finally, ask permission from the Key Informant for any possible follow-up communications with them for clarifications or additional inputs to shed more lights on specific topics discussed during the KII. Say “Thank You” before you leave. Schedule of KII: Date of Interview: Time Started: Time Ended: Venue of KII: Name of KII Interviewer: Name of Documenter: 109 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV SUSTAINABLE ECOSYSTEMS ADVANCED (SEA) PROJECT Midterm Performances Evaluation (MTPE) KII Guide Questions KII Tool 1.f: MMAF – Fisheries Surveillance and Marine Surveillance Directorates KII Guide Questions for: a) MMAF – Fisheries Surveillance and Marine Surveillance Directorates Main Focus: SA2-TA4 – Sustainable Fisheries through Law Enforcement 1. Introduction and Background to the SEA Project MTPE Introduction of KII Interviewer, and Documenter cum interpreter Good day! My name is ___________, KII Interviewer, and my companion is _____________, KII Documenter, of the Pacific Rim Innovation and Management Exponents, Inc. (PRIMEX). We are part of the MTEP Evaluation Team, commissioned by USAID/Indonesia to conduct an evaluation of the USAID SEA Project. Our team in particular is conducting key informant interview (KII) with officials and technical staff of SEA Project (TETRA TECH), MMAF, and representatives of selected Project partners/subcontractors, who are involved and are knowledgeable about SEA Project activities. The KII is important to achieve the MTPE objectives. And we would like to seek your understanding for this interview, which may take about one and a half (1.5) hours to two (2) hours. At this early stage, we wish to express our sincerest appreciation to you for participating in this KII session. Protocol on KII session ✓ It is important that the key informant is provided with enough time to express/expound his/her answers. But be sure to focus on the subject of interview. ✓ Confidentiality: Assure the respondent that all the data and information to be provided will be treated with utmost confidentiality and will be used mainly to undertake an objective assessment of the SEA Project’s performance. The name of the respondent will not, in anyway, be associated with any specific answers/comments. All the data and information will be collated and analyzed, together or in combination with other data and information collected from other KII respondents. ✓ There will be NO right or wrong answer. Encourage the respondent to feel free and be totally honest and objective in the answers or comments/opinion to be provided. ✓ The FGD session will be recorded; ask permission from the respondents if he/she will allow recording of her/his responses for documentation purposed. ✓ Before the KII begin, ask the respondent for any questions, concerns, or clarifications. 110 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 2. Basic Information about the KII Respondent Name of Respondent: Position: Agency/Office and Address Contact Information: Telephone No. Mobile No. E-mail Ad Note: Make sure to get the contact addresses (email and contact number of key informant for possible follow-up communications/calls for clarification on his/her responses). 3. Understanding the Role of MMAF – Fisheries Surveillance and Marine Surveillance Directorates What is your Office’s main goal or primary interest in the marine and fisheries resources at the national, provincial, and community? What were your activities relative to these resources before the USAID SEA Project? What activities are you doing/implementing at the national, provincial, community in relation to USAID SEA Project? National National Provincial Provincial Community Community 4. Main Body of Guide Questions Questions Responses/Evidences At the National Level At the Provincial Level EQ 1: Under what conditions are the regulations and/or tools being introduced by USAID SEA to relevant authorities leading to the adoption of these policies, regulations, and tools? 1.1 What are the supports of USAID SEA Project for the development and advancement of policies, regulations and/or tools and their adoption and enforcement to help in conservation and sustainable use of marine resources in the SEA Project areas? Did your office support these efforts? If yes, how did your office support these efforts? If no, what have been the reasons? 1.2 Are these policies, regulations and tools that have been authorized are effectively implemented to improve law enforcement in marine and fisheries resources at the national, provincial, and community? If Yes, in what ways, and what have been the results? If No, what have been the reasons? 111 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level 1.3 Are these regulations and/or tools introduced by USAID SEA Project and adopted and enforced in MPAs, MSPs, and in non-fishing livelihood activities areas lead to increased in number of hectares of biological significance and/or natural resources under improved natural resource management? If yes/no, why? 1.4 What are the conditions that facilitate the adoption (authorization) and enforcement of these policies, regulations and tools? Please provide answers for each policy, approach, regulation, and tool. 1.5 What are the conditions or factors that hinder or limit the progression of these policies, regulations and tools to reach the final stage of authorization/adoption and enforcement activities? Please provide answers for each policy, approach, regulation, and tool. EQ 2: Under what conditions do people trained to manage marine and fisheries resources and/or advocate for sustainable management of these resources, as a result of USAID SEA interventions, apply their skills in marine and fisheries resources management or advocacy? 2.1. How is your office support the USAID SEA Project interventions in training and applying skills learned in sustaining marine fisheries resources through the implementation of interventions for law enforcement? 2.2 How effective are these Law Enforcement interventions in helping conservation and sustainable use of marine resources in the SEA Project areas? How? And Why? 2.3 What are the conditions that facilitate the adoption and institutionalization of law enforcement approach? What are the conditions or factors that hinder or limit the adoption and institutionalization of law enforcement approach? 2.4 With the introduction of law enforcement approach by the USAID SEA Project, what factors or conditions emerges that facilitate/assist people and partners apply their learnings? 2.5. In Law Enforcement Approach, what are the factors or conditions that make it difficult 112 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level for trained people to apply their acquired knowledge and skills? EQ 3: To what extent do non‐fishing livelihood activities introduced or strengthened by USAID SEA change behavior and reduce threats and improve marine conservation (e.g., Livelihood program in Mare, Ternate, Galo‐ Galo, and Morotai)? Under what conditions are these activities effective? 3.1 What is your office’s role in supporting the SEA Project implement law enforcement intervention in non-fishing livelihood activities? 3.2 Did USAID SEA applying law enforcement interventions approach strengthened non￾fishing livelihood activities in Ternate, North Maluku, Galo-Galo, and Morotai that lead to change behavior and reduction of threats and improvement of marine conservation? If yes, how and why? If no, why? 3.3 What are the conditions or factors that contribute to the effectivity of non-fishing livelihood activities to change behavior of people and partners for reduction of threats and sustainable marine conservation? 3.4 What are the conditions or factors that diminish the affectivity of these non-fishing livelihood activities to reduce threats and improve marine conservation in the selected project sites? EQ 4: To what extent do USAID‐supported fishermen with more secure marine tenure in fisheries apply sustainable fishing practices and reduce destructive fishing practices (e.g., Territorial Use Rights for Fishing [TURF] in Raja Ampat and Local Marine Management Area [LMMA] in Maluku)? Under what conditions are these activities effective? 4.1What is your office’s involvement in supporting USAID SEA law enforcement interventions to provide secure tenure or managed access to fisheries resources (e.g. LMMA, CFA, TURF, and TCBFMA)? 4.2 How effective are the secure tenure or managed access activities in applying sustainable fishing practices and reduce destructive fishing practices? If Yes, in what ways, and what have been the results? If No, what have been the reasons? Which of these tenure activities (LMMA, CFA, TURF, and TCBFMA) are effective in applying sustainable fishing practices and reduction of destructive fishing practices? 113 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level 4.3 Under what conditions are the secure tenure or managed access to fisheries resources effective or not effective? (Probing question: Can you cite evidences of effectiveness or ineffectiveness of each tenure instrument?). Why? EQ 5: Given that USAID SEA addresses key marine threats and drivers through a complex project comprised of multiple components / strategic approaches implemented by a large number of partners, how well are all the components /approaches working together? What aspects of overall management are working well and what can be improved? To what extent are there capacity gaps among the partners, which might be hindering effective implementation? 5.1 What does the USAID SEA law enforcement interventions provide to ensure that all the components/approaches of the SEA Project are implemented or working together in the project sites to achieve marine conservation? 5.2 Can you cite specific evidences where all these components/approaches of the SEA Project are working together to achieve marine conservation? Are all these effective when working together? If Yes, in what ways, and what have been the results? If No, what have been the reasons? 5.3 What is your office’s involvement in the USAID SEA to ensure that all the components/approaches are effectively working together? 5.4 How is the establishment and implementation of POKMASWAS help in conservation and sustainable use of marine resources in SEA Project Areas? How effective? Why? 5.5 How is the establishment and implementation of the community-based surveillance systems (SISWASMAS) help in conservation and sustainable use of marine resources in SEA Project Areas? How effective? Why? 5.6 How effective is the establishment and implementation of POKMASWAS and of the community-based surveillance systems (SISWASMAS) resulted in the strengthening of law enforcement networks in SEA Project Areas? How and why? 114 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level 5.7 What aspects of overall management of the SEA Project are working well and what can be improved? 5.8 Are there capacity gaps among partners which might be hindering effective implementation of law enforcement interventions? What are these capacity gaps (per partner)? What did SEA Project do to address these gaps? Are these actions effective in arresting/responding to these gaps. What further recommendations or courses of action to take to address these gaps? 5.9 How effective are relationships/working arrangements in application of law enforcement interventions between USAID SEA Project and subcontractors, and among subcontractors? What other areas could be improved? 5.10. What recommendations or course of action to take to improve more (or make more successful/effective) the overall implementation of the SEA Project? EQ6: Based on the answers above about the conditions under which the USAID SEA project strategies are or are not effective, what are the recommended areas (thematic and/or geographic) of expansion/contraction for USAID? 6.1 Which of the project’s strategies are still relevant and effective at this point in time (mid-term implementation) in achieving the goals of the SEA project? What are the strategies that are still relevant? Why? What are strategies that are not relevant? Why 6.2 Which project strategies (thematic and/or geographic) need changes/improvement? Why? 6.3 What are the areas where the project needs to either expand or contract its work in relation to specific strategies and/or geographic scope? Why? 6.4 What actions are needed to improve the overall implementation of the SEA Project? Why? 115 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 5. Closing/Sign-Off Questions Again, assure the respondent that all information obtained shall remain confidential and will not be used for purposes other than what have stated. Finally, ask permission from the Key Informant for any possible follow-up communications with them for clarifications or additional inputs to shed more lights on specific topics discussed during the KII. Say “Thank You” before you leave. Schedule of KII: Date of Interview: Time Started: Time Ended: Venue of KII: Name of KII Interviewer: Name of Documenter: 116 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV SUSTAINABLE ECOSYSTEMS ADVANCED (SEA) PROJECT Midterm Performances Evaluation (MTPE) KII Guide Questions KII Tool 1.g: MMAF – Planning Bureau (Biro Perencanaan-Roren) KII Guide Questions for: a) MMAF – Planning Bureau (Biro Perencanaan-Roren) Main Focus: SA3- Increase Incentives thru Non-fishing livelihood activities 1. Introduction and Background to the SEA Project MTPE Introduction of KII Interviewer, and Documenter cum interpreter Good day! My name is ___________, KII Interviewer, and my companion is _____________, KII Documenter, of the Pacific Rim Innovation and Management Exponents, Inc. (PRIMEX). We are part of the MTEP Evaluation Team, commissioned by USAID/Indonesia to conduct an evaluation of the USAID SEA Project. Our team in particular is conducting key informant interview (KII) with officials and technical staff of SEA Project (TETRA TECH), MMAF, and representatives of selected Project partners/subcontractors, who are involved and are knowledgeable about SEA Project activities. The KII is important to achieve the MTPE objectives. And we would like to seek your understanding for this interview, which may take about one and a half (1.5) hours to two (2) hours. At this early stage, we wish to express our sincerest appreciation to you for participating in this KII session. Protocol on KII session ✓ It is important that the key informant is provided with enough time to express/expound his/her answers. But be sure to focus on the subject of interview. ✓ Confidentiality: Assure the respondent that all the data and information to be provided will be treated with utmost confidentiality and will be used mainly to undertake an objective assessment of the SEA Project’s performance. The name of the respondent will not, in anyway, be associated with any specific answers/comments. All the data and information will be collated and analyzed, together or in combination with other data and information collected from other KII respondents. ✓ There will be NO right or wrong answer. Encourage the respondent to feel free and be totally honest and objective in the answers or comments/opinion to be provided. ✓ The FGD session will be recorded; ask permission from the respondents if he/she will allow recording of her/his responses for documentation purposed. ✓ Before the KII begin, ask the respondent for any questions, concerns, or clarifications. 117 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 2. Basic Information about the KII Respondent Name of Respondent: Position: Agency/Office and Address Contact Information: Telephone No. Mobile No. E-mail Ad Note: Make sure to get the contact addresses (email and contact number of key informant for possible follow-up communications/calls for clarification on his/her responses). 3. Understanding the Role of MMAF – Planning Bureau (Biro Perencanaan-Roren) What is your Office’s main goal or primary interest in the marine and fisheries resources at the national, provincial, and community? What were your activities relative to these resources before the USAID SEA Project? What activities are you doing/implementing at the national, provincial, community in relation to USAID SEA Project? National National Provincial Provincial Community Community 4. Main Body of Guide Questions Questions Responses/Evidences At the National Level At the Provincial Level EQ 1: Under what conditions are the regulations and/or tools being introduced by USAID SEA to relevant authorities leading to the adoption of these policies, regulations, and tools? 1.1 What are the supports of USAID SEA Project for the development and advancement of policies, regulations and/or tools and their adoption in the establishment of the intervention to increase incentives thru non-fishing livelihood activities in SEA project areas? Did your office support these efforts? If yes, how and in what ways? If no, what have been the reasons? (Probe question: How did SEA project inform the stakeholders and participants of non-fishing livelihood activities of ways to increase incentives thru the project? How effective are these strategies? Why?) 1.2 Are these policies, regulations and tools that have been authorized are effectively implemented to improve/ increase incentives thru non-fishing livelihood activities in SEA project areas? If Yes, in what ways, and what 118 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level have been the results? If No, what have been the reasons? 1.3 Are these regulations and/or tools introduced by USAID SEA Project and adopted in non￾fishing livelihood activities areas lead to increased in number of hectares of biological significance and/or natural resources under improved natural resource management? If yes/no, why? 1.4 What are the conditions that facilitate the adoption (authorization) and enforcement of these policies, regulations and tools in non￾fishing livelihood activities areas? Please provide answers for each policy, approach, regulation, and tool. 1.5What are the conditions or factors that hinder or limit the progression of these policies, regulations and tools to reach the final stage of authorization/adoption and enforcement activities in non-fishing livelihood activities areas? Please provide answers for each policy, approach, regulation, and tool. EQ 2: Under what conditions do people trained to manage marine and fisheries resources and/or advocate for sustainable management of these resources, as a result of USAID SEA interventions, apply their skills in marine and fisheries resources management or advocacy? 2.1 How effective are the establishments of the intervention to increase incentives thru non￾fishing livelihood activities in helping conservation and sustainable use of marine resources in the SEA Project areas? How? And Why? 2.2 What are the conditions that facilitate the adoption and institutionalization of the intervention to increase incentives thru non￾fishing livelihood activities? What are the conditions or factors that hinder or limit the adoption and institutionalization of the intervention to increase incentives thru non￾fishing livelihood activities? Why? 2.3. In Non-fishing livelihood activities, what are the factors or conditions that make it difficult for trained people to apply their acquired knowledge and skills? EQ 3: To what extent do non‐fishing livelihood activities introduced or strengthened by USAID SEA change behavior and reduce threats and improve marine conservation (e.g., Livelihood program in Ternate North Maluku, Galo‐Galo, and Morotai)? Under what conditions are these activities effective? 119 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level 3.1 What are the non-fishing livelihood activities introduced or strengthened by USAID SEA Project? Which of this/these activities were effective in changing the behavior of the community towards reduction on threats and improvement in marine conservation? In what ways are these activities effective? How and why? 3.2 What is your office’s role in supporting the SEA Project implementation of the ways in increasing incentives in non-fishing livelihood activities? How effective are these interventions in changing the behavior of the community towards reduction on threats and improvement in marine conservation? In what ways are these activities effective? How and why? 3.3 These non-fishing livelihood activities, have they been effective in increasing the following? If Yes, in what ways and what have been the results? If No, what have been the reasons? 3.3.1 Increase in the number of public-private partnerships with focused on sustainable fisheries and marine conservation with leveraged resources quantified? 3.3.2 Increase in the number of people receiving social, ecological, and economic benefits derived from conservation and sustainable use of marine resources? 3.4 What are the conditions or factors that contribute to the effectivity of the intervention that increase incentives thru non-fishing livelihood activities to change behavior of people and partners for reduction of threats and sustainable marine conservation? 3.5 What are the conditions or factors that diminish the effectivity of these interventions thru non-fishing livelihood activities to reduce threats and improve marine conservation in the selected project sites? EQ 4: To what extent do USAID‐supported fishermen with more secure marine tenure in fisheries apply sustainable fishing practices and reduce destructive fishing practices (e.g., Territorial Use Rights for Fishing [TURF] in Raja Ampat and Local Marine Management Area [LMMA] in Maluku)? Under what conditions are these activities effective? 120 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level 4.1What is your office’s involvement in supporting USAID SEA interventions to provide secure tenure or managed access to fisheries resources (e.g. LMMA, CFA, TURF, and TCBFMA)? 4.2 How effective are the secure tenure or managed access activities? If Yes, in what ways, and what have been the results? If No, what have been the reasons? Which of these tenure activities (LMMA, CFA, TURF, and TCBFMA) are effective in applying sustainable fishing practices and reduction of destructive fishing practices? 4.3 Under what conditions are the secure tenure or managed access to fisheries resources effective or not effective? (Probing question: Can you cite evidences of effectiveness or ineffectiveness of each tenure instrument?). Why? EQ 5: Given that USAID SEA addresses key marine threats and drivers through a complex project comprised of multiple components / strategic approaches implemented by a large number of partners, how well are all the components /approaches working together? What aspects of overall management are working well and what can be improved? To what extent are there capacity gaps among the partners, which might be hindering effective implementation? 5.1 What does the USAID SEA interventions (taken together) provide to ensure that all the components/approaches of the SEA Project are implemented or working together in the project sites to achieve marine conservation? 5.2 Can you cite specific evidences where all these components/approaches of the SEA Project are working together to achieve marine conservation? Are all these effective when working together? If Yes, in what ways, and what have been the results? If No, what have been the reasons? 5.3 If Yes, what is your office’s involvement in the USAID SEA to ensure that all the components/approaches are effectively working together? 5.4 What aspects of overall management of the SEA Project are working well and what can be improved? 5.5 Are there capacity gaps among partners which might be hindering effective 121 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level implementation of project interventions in non-fishing activities areas? What are these capacity gaps (per partner)? What did SEA Project do to address these gaps? Are these actions effective in arresting/responding to these gaps. What further recommendations or courses of action to take to address these gaps? 5.6 How effective are relationships/working arrangements in application of interventions that increase incentives thru non-fishing activities between USAID SEA Project and subcontractors, between SEA Project and stakeholders and partners and among subcontractors? What other areas could be improved? 5.7 What recommendations or course of action to take to improve more (or make more successful/effective) the overall implementation of the SEA Project? EQ6: Based on the answers above about the conditions under which the USAID SEA project strategies are or are not effective, what are the recommended areas (thematic and/or geographic) of expansion/ contraction for USAID? 6.1 Which of the project’s strategies are still relevant and effective at this point in time (mid-term implementation) in achieving the goals of the SEA project? What are the strategies that are still relevant? Why? What are strategies that are not relevant? Why 6.2 Which project strategies (thematic and/or geographic) need changes/improvement? Why? 6.3 What are the areas where the project needs to either expand or contract its work in relation to specific strategies and/or geographic scope? Why? 6.4 What actions are needed to improve the overall implementation of the SEA Project? Why? 122 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 5. Closing/Sign-Off Questions Again, assure the respondent that all information obtained shall remain confidential and will not be used for purposes other than what have stated. Finally, ask permission from the Key Informant for any possible follow-up communications with them for clarifications or additional inputs to shed more lights on specific topics discussed during the KII. Say “Thank You” before you leave. Schedule of KII: Date of Interview: Time Started: Time Ended: Venue of KII: Name of KII Interviewer: Name of Documenter: 123 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV SUSTAINABLE ECOSYSTEMS ADVANCED (SEA) PROJECT Midterm Performances Evaluation (MTPE) KII Guide Questions KII Tool 1.h: MMAF – Legal Bureau (Biro Hukum) KII Guide Questions for: a) MMAF – Legal Bureau (Biro Hukum) Main Focus: SA4 – Advance Policy and Regulations 1. Introduction and Background to the SEA Project MTPE Introduction of KII Interviewer, and Documenter cum interpreter Good day! My name is ___________, KII Interviewer, and my companion is _____________, KII Documenter, of the Pacific Rim Innovation and Management Exponents, Inc. (PRIMEX). We are part of the MTEP Evaluation Team, commissioned by USAID/Indonesia to conduct an evaluation of the USAID SEA Project. Our team in particular is conducting key informant interview (KII) with officials and technical staff of SEA Project (TETRA TECH), MMAF, and representatives of selected Project partners/subcontractors, who are involved and are knowledgeable about SEA Project activities. The KII is important to achieve the MTPE objectives. And we would like to seek your understanding for this interview, which may take about one and a half (1.5) hours to two (2) hours. At this early stage, we wish to express our sincerest appreciation to you for participating in this KII session. Protocol on KII session ✓ It is important that the key informant is provided with enough time to express/expound his/her answers. But be sure to focus on the subject of interview. ✓ Confidentiality: Assure the respondent that all the data and information to be provided will be treated with utmost confidentiality and will be used mainly to undertake an objective assessment of the SEA Project’s performance. The name of the respondent will not, in anyway, be associated with any specific answers/comments. All the data and information will be collated and analyzed, together or in combination with other data and information collected from other KII respondents. ✓ There will be NO right or wrong answer. Encourage the respondent to feel free and be totally honest and objective in the answers or comments/opinion to be provided. ✓ The FGD session will be recorded; ask permission from the respondents if he/she will allow recording of her/his responses for documentation purposed. ✓ Before the KII begin, ask the respondent for any questions, concerns, or clarifications. 124 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 2. Basic Information about the KII Respondent Name of Respondent: Position: Agency/Office and Address Contact Information: Telephone No. Mobile No. E-mail Ad Note: Make sure to get the contact addresses (email and contact number of key informant for possible follow-up communications/calls for clarification on his/her responses). 3. Understanding the Role of MMAF – Legal Bureau (Biro Hukum) What is your Office’s main goal or primary interest in the marine and fisheries resources at the national, provincial, and community? What were your activities relative to these resources before the USAID SEA Project? What activities are you doing/implementing at the national, provincial, community in relation to USAID SEA Project? National National Provincial Provincial Community Community 4. Main Body of Guide Questions Questions Responses/Evidences At the National Level At the Provincial Level EQ 1: Under what conditions are the regulations and/or tools being introduced by USAID SEA to relevant authorities leading to the adoption of these policies, regulations, and tools? 1.1 What are the supports of USAID SEA Project for the development, advancement, and facilitating dialog on policies, regulations and/or tools and their adoption and enforcement to help in conservation and sustainable use of marine resources in the SEA Project areas? Did your office support these efforts? If yes, how and in what ways? If no, what have been the reasons? 1.2 Are these policies, and regulations and tools that have been authorized are effectively implemented to improve law enforcement in marine and fisheries resources at the national, provincial, and community levels? If Yes, in what ways, and what have been the results? If No, what have been the reasons? 125 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level 1.3 Are these regulations and/or tools introduced by USAID SEA Project and adopted and enforced in a broad range of FMPs, MPAs, MSPs, and in non-fishing livelihood activities areas, as well as in marine tenure and managed access areas (e.g. TURF, LMMA, CFA, and TCBFMA) lead to increased in number of hectares of biological significance and/or natural resources under improved natural resource management? If yes/no, why? 1.4 What are the conditions that facilitate the adoption (authorization) and enforcement of these policies, regulations and tools? Please provide answers for each policy, approach, regulation, and tool. 1.5What are the conditions or factors that hinder or limit the progression of these policies, regulations and tools to reach the final stage of authorization/adoption and enforcement activities? Please provide answers for each policy, approach, regulation, and tool. EQ 2: Under what conditions do people trained to manage marine and fisheries resources and/or advocate for sustainable management of these resources, as a result of USAID SEA interventions, apply their skills in marine and fisheries resources management or advocacy? 2.1 What are the activities of USAID SEA Project interventions to facilitate dialog on policies and regulations with FMP components? What is your office involvement? How effective are these dialogs in developing, incorporating, and implementing these policies and regulations into marine governance? Why? 2.2. What are the USAID SEA Project interventions under facilitating dialog on policies and regulation in the national, provincial, and community levels? What conditions or factors contributed in pursuing these efforts? 2.3 What are the conditions that facilitate the adoption and institutionalization of these policies and regulation? What are the conditions or factors that hinder or limit the adoption and institutionalization of these policies and regulation? 2.4. In facilitating dialog for policies and regulation, what are the factors or 126 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level conditions that make it difficult for trained people to apply their acquired knowledge and skills? Why? EQ 3: To what extent do non‐fishing livelihood activities introduced or strengthened by USAID SEA change behavior and reduce threats and improve marine conservation (e.g., Livelihood program in Ternate North Maluku, Galo‐Galo, and Morotai)? Under what conditions are these activities effective? 3.1 What is your office’s role in supporting the SEA Project implement facilitating dialog on policies and regulation intervention in non-fishing livelihood activities? 3.2 Did USAID SEA applying the interventions strengthened non-fishing livelihood activities in Ternate, North Maluku, Galo￾Galo, and Morotai that lead to economic benefits and change behavior and reduction of threats and improvement of marine conservation? If yes, how and why? If no, why? 3.3 How effective are non-fishing livelihood activities in changing people’s behavior as shown by the increased in number of people demonstrating behavior that contribute to biodiversity conservation as a result of SEA Project applying law enforcement approach? Why? Did the law enforcement interventions lead to reduction of threats in these areas? If yes, how and why? If no, why? 3.4 What are the conditions or factors that contribute to the effectivity of non-fishing livelihood activities to change behavior of people and partners for reduction of threats and sustainable marine conservation? 3.5 What are the conditions or factors that diminish the effectivity of these non-fishing livelihood activities to reduce threats and improve marine conservation in the selected project sites? EQ 4. To what extent do USAID‐supported fishermen with more secure marine tenure in fisheries apply sustainable fishing practices and reduce destructive fishing practices (e.g., Territorial Use Rights for Fishing [TURF] in Raja Ampat and Local Marine Management Area [LMMA] in Maluku)? Under what conditions are these activities effective? 4.1What is your office’s involvement in supporting USAID SEA interventions in facilitating dialog on policies and regulations to provide secure tenure or 127 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level managed access to fisheries resources (e.g. LMMA, CFA, TURF, and TCBFMA)? 4.2 How effective are the secure tenure or managed access activities? If Yes, in what ways, and what have been the results? If No, what have been the reasons? Which of these tenure activities (LMMA, CFA, TURF, and TCBFMA) are effective in applying sustainable fishing practices and reduction of destructive fishing practices? 4.3 Under what conditions are the secure tenure or managed access to fisheries resources effective or not effective? (Probing question: Can you cite evidences of effectiveness or ineffectiveness of each tenure instrument?). Why? EQ 5: Given that USAID SEA addresses key marine threats and drivers through a complex project comprised of multiple components / strategic approaches implemented by a large number of partners, how well are all the components /approaches working together? What aspects of overall management are working well and what can be improved? To what extent are there capacity gaps among the partners, which might be hindering effective implementation? 5.1 What does the USAID SEA interventions in facilitating dialog on policies and regulations to ensure that all the components/approaches of the SEA Project are implemented or working together in the project sites to achieve marine conservation? How? How effective? Why? 5.2 Can you cite specific evidences where all these components/approaches of the SEA Project are working together to achieve marine conservation? Are all these effective when working together? If Yes, in what ways, and what have been the results? If No, what have been the reasons? 5.3 If Yes, what is your office’s involvement in the USAID SEA to ensure that all the components/approaches are effectively working together? 5.4 How is the establishment and implementation of POKMASWAS help in conservation and sustainable use of marine resources in SEA Project Areas? How effective? Why? 128 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level 5.5 How is the establishment and implementation of the community-based surveillance systems (SISWASMAS) help in conservation and sustainable use of marine resources in SEA Project Areas? How effective? Why? 5.6 How effective is the establishment and implementation of POKMASWAS and of the community-based surveillance systems (SISWASMAS) resulted in the strengthening of dialog on policies and regulation networks in SEA Project Areas? How and why? 5.7 What aspects of overall management of the SEA Project are working well and what can be improved? 5.8 Are there capacity gaps among partners which might be hindering effective implementation of the intervention of facilitating dialog on policies and regulations? What are these capacity gaps (per partner)? What did SEA Project do to address these gaps? Are these actions effective in arresting/responding to these gaps. What further recommendations or courses of action to take to address these gaps? 5.9 How effective are relationships/working arrangements in application of policies and regulations interventions between USAID SEA Project and subcontractors, and among subcontractors? What other areas could be improved? 5.10 What recommendations or course of action to take to improve more (or make more successful/effective) the overall implementation of the SEA Project? EQ6: Based on the answers above about the conditions under which the USAID SEA project strategies are or are not effective, what are the recommended areas (thematic and/or geographic) of expansion/contraction for USAID? 6.1 Which of the project’s strategies are still relevant and effective at this point in time (mid-term implementation) in achieving the goals of the SEA project? What are the strategies that are still relevant? Why? What are strategies that are not relevant? Why? 129 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level 6.2 Which project strategies (thematic and/or geographic) need changes/improvement? Why? 6.3 What are the areas where the project needs to either expand or contract its work in relation to specific strategies and/or geographic scope? Why? 6.4 What actions are needed to improve the overall implementation of the SEA Project? Why? 5. Closing/Sign-Off Questions Again, assure the respondent that all information obtained shall remain confidential and will not be used for purposes other than what have stated. Finally, ask permission from the Key Informant for any possible follow-up communications with them for clarifications or additional inputs to shed more lights on specific topics discussed during the KII. Say “Thank You” before you leave. Schedule of KII: Date of Interview: Time Started: Time Ended: Venue of KII: Name of KII Interviewers Name of Documenter: 130 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV SUSTAINABLE ECOSYSTEMS ADVANCED (SEA) PROJECT Midterm Performances Evaluation (MTPE) KII Guide Questions KII Tool 1.i: MMAF – Center for Training and Extension Services KII Guide Questions for: a) MMAF – Center for Training and Extension Services Main Focus: SA5 – Institutionalize Training and Capacity Building 1. Introduction and Background to the SEA Project MTPE Introduction of KII Interviewer, and Documenter cum interpreter Good day! My name is ___________, KII Interviewer, and my companion is _____________, KII Documenter, of the Pacific Rim Innovation and Management Exponents, Inc. (PRIMEX). We are part of the MTEP Evaluation Team, commissioned by USAID/Indonesia to conduct an evaluation of the USAID SEA Project. Our team in particular is conducting key informant interview (KII) with officials and technical staff of SEA Project (TETRA TECH), MMAF, and representatives of selected Project partners/subcontractors, who are involved and are knowledgeable about SEA Project activities. The KII is important to achieve the MTPE objectives. And we would like to seek your understanding for this interview, which may take about one and a half (1.5) hours to two (2) hours. At this early stage, we wish to express our sincerest appreciation to you for participating in this KII session. Protocol on KII session ✓ It is important that the key informant is provided with enough time to express/expound his/her answers. But be sure to focus on the subject of interview. ✓ Confidentiality: Assure the respondent that all the data and information to be provided will be treated with utmost confidentiality and will be used mainly to undertake an objective assessment of the SEA Project’s performance. The name of the respondent will not, in anyway, be associated with any specific answers/comments. All the data and information will be collated and analyzed, together or in combination with other data and information collected from other KII respondents. ✓ There will be NO right or wrong answer. Encourage the respondent to feel free and be totally honest and objective in the answers or comments/opinion to be provided. ✓ The FGD session will be recorded; ask permission from the respondents if he/she will allow recording of her/his responses for documentation purposed. ✓ Before the KII begin, ask the respondent for any questions, concerns, or clarifications. 131 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 2. Basic Information about the KII Respondent Name of Respondent: Position: Agency/Office and Address Contact Information: Telephone No. Mobile No. E-mail Ad Note: Make sure to get the contact addresses (email and contact number of key informant for possible follow-up communications/calls for clarification on his/her responses). 3. Understanding the Role of MMAF – Center for Training and Extension Services What is your Office’s main goal or primary interest in the marine and fisheries resources at the national, provincial, and community? What were your activities relative to these resources before the USAID SEA Project? What activities are you doing/implementing at the national, provincial, community in relation to USAID SEA Project? National National Provincial Provincial Community Community 4. Main Body of Guide Questions Questions Responses/Evidences At the National Level At the Provincial Level EQ 1: Under what conditions are the regulations and/or tools being introduced by USAID SEA to relevant authorities leading to the adoption of these policies, regulations, and tools? 1.1 What support or assistance has the SEA Project provided to facilitate training (and inform) and capacity building of stakeholders and partners on the adoption of policies, regulations, and tools for sustainable natural resources management and biodiversity conservation? Did your office support these efforts? If yes, how and in what ways? If no, what have been the reasons? 1.2 Are these training and capacity building effectively implemented to improve marine and fisheries resources at the national, provincial, and community levels? If Yes, in what ways, and what have been the results? If No, what have been the reasons? 1.3 Did these training and capacity building lead to the following: 132 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level 1.3.1 Increased the number of people trained in sustainable resources management and/or biodiversity conservation? If yes, what was the result? If no, what have been the reasons? 1.3.2 Increased the number of people that apply and adopt improved conservation law enforcement practices? If yes, what was the result? If no, what have been the reasons? 1.3.3 Increased the number of people demonstrating behaviors change that contribute to sustainable marine resources and biodiversity conservation? If yes, what was the result? If no, what have been the reasons? 1.3.4 Increased the number of innovations that support marine sustainable practices and biodiversity conservation? If yes, what was the result? If no, what have been the reasons? 1.4 What are the conditions that facilitate the effective implementation of these trainings and capacity building for the adoption (authorization) and enforcement of these policies, regulations and tools? Please provide answers for each policy, approach, regulation, and tool. 1.5 What are the conditions or factors that hinder or limit the affectivity of the training and capacity building in order for these policies, regulations and tools to reach the final stage of authorization/adoption and enforcement activities? Please provide answers for each policy, approach, regulation, and tool. EQ 2: Under what conditions do people trained to manage marine and fisheries resources and/or advocate for sustainable management of these resources, as a result of USAID SEA interventions, apply their skills in marine and fisheries resources management or advocacy? 2.1 How is your office making use of the USAID SEA Project interventions to train and inform people and partners to manage marine and fisheries resources and/or advocate for sustainable management of these resources to enable them to apply their skills learned in marine fisheries management or advocacy? 2.2 How effective is the USAID SEA Project support in facilitating the application of knowledge and skills acquired? 133 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level 2.3 In what ways do people and partners apply their learnings in marine fisheries management or advocacy? 2.4 What are the factors or conditions that facilitate/assist people and partners apply their learnings? 2.5 What are the factors or conditions that make it difficult for trained people to apply their acquired knowledge and skills? EQ 3: To what extent do non‐fishing livelihood activities introduced or strengthened by USAID SEA change behavior and reduce threats and improve marine conservation (e.g., Livelihood program in Ternate, North Maluku, Galo‐Galo, and Morotai)? Under what conditions are these activities effective? 3.1 What is your Center’s role in training and capacitating stakeholders and partners in non￾fishing livelihood activities sites? How effective/not effective are these training inputs in changing behavior, reduce threats, and improve marine conservation? Why? 3.2 Did USAID SEA applying the interventions strengthened non-fishing livelihood activities in Ternate, North Maluku, Galo-Galo, and Morotai that lead to economic benefits and change behavior and reduction of threats and improvement of marine conservation? If yes, how and why? If no, why? 3.3 What types of non-fishing livelihood activities were introduced or supported by SEA Project? Have they been effective in increasing the following? If Yes, in what ways and what have been the results? If No, what have been the reasons? 3.3.1 Increase in the number of public-private partnerships with focused on sustainable fisheries and marine conservation with leveraged resources quantified? 3.3.2 Increase in the number of people receiving social, ecological, and economic benefits derived from conservation and sustainable use of marine resources? 3.4 What are the conditions or factors that contribute to the effectivity of non-fishing livelihood activities to change behavior of people and partners for reduction of threats and sustainable marine conservation? 3.5 What are the conditions or factors that diminish the effectivity of these non-fishing 134 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level livelihood activities to reduce threats and improve marine conservation in the selected project sites? EQ 4: To what extent do USAID‐supported fishermen with more secure marine tenure in fisheries apply sustainable fishing practices and reduce destructive fishing practices (e.g., Territorial Use Rights for Fishing [TURF] in Raja Ampat and Local Marine Management Area [LMMA] in Maluku)? Under what conditions are these activities effective? 4.1What is your office’s involvement in supporting USAID SEA interventions in supporting fishermen with more secure tenure or managed access to fisheries resources (e.g. LMMA, CFA, TURF, and TCBFMA)? 4.2 How effective are the secure tenure or managed access activities? If Yes, in what ways, and what have been the results? If No, what have been the reasons? Which of these tenure activities (LMMA, CFA, TURF, and TCBFMA) are effective in applying sustainable fishing practices and reduction of destructive fishing practices? 4.3 Under what conditions are the secure tenure or managed access to fisheries resources effective or not effective? (Probing question: Can you cite evidences of effectiveness or ineffectiveness of each tenure instrument?). Why? EQ 5: Given that USAID SEA addresses key marine threats and drivers through a complex project comprised of multiple components / strategic approaches implemented by a large number of partners, how well are all the components /approaches working together? What aspects of overall management are working well and what can be improved? To what extent are there capacity gaps among the partners, which might be hindering effective implementation? 5.1 What actions has the project taken to maximize synergies? How effectively are all the components and approaches of the SEA Project working together? What additional actions need to be taken to maximize synergies? 5.2 How effective has the SEA Project been in coordinating the implementation of the different project components and approaches? 5.3 Can you cite specific evidences where all these components/approaches of the SEA Project are working together to achieve marine conservation? Are all these effective when working together? If Yes, in what ways, and what have been the results? If No, what have been the reasons? 135 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences At the National Level At the Provincial Level 5.4 What aspects of overall management are working well and what can be improved? 5.5 Are there capacity gaps among partners which might be hindering effective implementation? What are these capacity gaps (per partner)? What did SEA Project do to address these gaps? Are these actions effective in arresting/responding to the gaps. What further recommendations or courses of action to take to address these gaps? 5.6 How effective are relationships / working arrangements between USAID SEA Project and subcontractors, and among subcontractors? What other areas could be improved? 5.7. What recommendation / suggestion or course of action to take to improve more (or make more successful/effective) the overall implementation of the SEA Project? EQ6: Based on the answers above about the conditions under which the USAID SEA project strategies are or are not effective, what are the recommended areas (thematic and/or geographic) of expansion/contraction for USAID? 6.1 Which of the project’s strategies are still relevant and effective at this point in time (mid-term implementation) in achieving the goals of the SEA project? What are the strategies that are still relevant? Why? What are strategies that are not relevant? Why 6.2 Which project strategies (thematic and/or geographic) need changes/improvement? Why? 6.3 What are the areas where the project needs to either expand or contract its work in relation to specific strategies and/or geographic scope? Why? 6.4 What actions are needed to improve the overall implementation of the SEA Project? Why? 136 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 5. Closing/Sign-Off Questions Again, assure the respondent that all information obtained shall remain confidential and will not be used for purposes other than what have stated. Finally, ask permission from the Key Informant for any possible follow-up communications with them for clarifications or additional inputs to shed more lights on specific topics discussed during the KII. Say “Thank You” before you leave. Schedule of KII: Date of Interview: Time Started: Time Ended: Venue of KII: Name of KII Interviewer: Name of Documenter: 137 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV SUSTAINABLE ECOSYSTEMS ADVANCED (SEA) PROJECT Midterm Performances Evaluation (MTPE) KII Guide Questions KII Tool 2.a. Provincial Government Project Coordinator/Heads of the Provincial Marine and Fisheries Agencies/Regional Technical Coordinator KII Guide Questions for: a) Provincial Government Project Coordinator b) Heads of the Provincial Marine and Fisheries Agencies c) Regional Technical Director (Maluku) d) West Papua Coordinator Main Focus: Overall project management, coordination, communication, in the province of Maluku, North Maluku, and West Papua in the following aspects: SA1 - Create demand thru awareness raising and advocacy; SA2 - Improve ecosystem management of FMA-715 and MPAs; SA2-TA1- Sustainable Fisheries thru ecosystem approach to fishery management (EAFM); SA2-TA2 - MPA Management; SA2-TA3 - MSP (Marine Spatial Planning); SA2-TA4 - Law Enforcement; SA3 - Increase incentives for marine stewardship thru non-fishing livelihood activities, and more secure tenure for fisheries resources; SA4 - Advance policy and regulations; and SA5 - Institutionalize training and capacity building. 1. Introduction and Background to the SEA Project MTPE Introduction of KII Interviewer, and Documenter cum interpreter Good day! My name is ___________, KII Interviewer, and my companion is _____________, KII Documenter, of the Pacific Rim Innovation and Management Exponents, Inc. (PRIMEX). We are part of the MTEP Evaluation Team, commissioned by USAID/Indonesiato conduct an evaluation of the USAID SEA Project. Our team in particular is conducting key informant interview (KII) with officials and technical staff of SEA Project (Tetra Tech ARD), MMAF, and representatives of selected Project partners/subcontractors who are involved and are knowledgeable about SEA Project activities. The KII is important to achieve the MTPE objectives. And we would like to seek your understanding for this interview, which may take about one and a half (1.5) hours to two (2) hours. At this early stage, we wish to express our sincerest appreciation to you for participating in this KII session. Protocol on KII session ✓ It is important that the key informant is provided with enough time to express/expound his/her answers. But be sure to focus on the subject of interview. ✓ Confidentiality: Assure the respondent that all the data and information to be provided will be treated with utmost confidentiality and will be used mainly to undertake an objective assessment 138 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV of the SEA Project’s performance. The name of the respondent will not, in anyway, be associated with any specific answers/comments. All the data and information will be collated and analyzed, together or in combination with other data and information collected from other KII respondents. ✓ There will be NO right or wrong answer. Encourage the respondent to feel free and be totally honest and objective in the answers or comments/opinion to be provided. ✓ The FGD session will be recorded; ask permission from the respondents if he/she will allow recording of her/his responses for documentation purposed. ✓ Before the KII begin, ask the respondent for any questions, concerns, or clarifications. 2. Basic Information about the KII Respondent Name of Respondent: Position: Agency/Office and Address Contact Information: Telephone No. Mobile No. E-mail Ad Note: Make sure to get the contact addresses (email and contact number of key informant for possible follow-up communications/calls for clarification on his/her responses). 3. Main Body of Guide Questions Questions Responses/Evidences Key Informant EQ1: Under what conditions are the regulations and/or tools being introduced by USAID SEA to relevant authorities leading to the adoption of these policies, regulations, and tools? 1.1 In your province, how did the SEA Project create demand through awareness raising and advocacy of numerous marine and fisheries policies it developed? (Possible probe question: How did the SEA Project inform the public and partners in your province and communities relative to these approaches, policies, regulations, and tools? In particular, how was the implementation of EAFM, MPA, MSP, law enforcement, non-fishing livelihood activities, marine tenure and managed access to marine resources, and training and capability building?). Please mention the specific policies, regulations, approaches and tools in its application to EAFM, MPA, MSP, law enforcement, non-fishing livelihood activities, marine tenure and managed access to marine resources. 1.2 How did you support the efforts of SEA projects in your province? In what ways and what was the results? If no, what have been the reasons? 1.3 Are these policies, regulations, and tools that have been authorized effectively implemented at the provincial, and 139 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences Key Informant community levels? If Yes, in what ways, and what have been the results? If No, what have been the reasons? 1.4 What support or assistance has the SEA Project provided to your province and communities to facilitate the adoption of these policies and regulations? How effective has this support been in accomplishing and facilitating adoption? Why has the SEA Project’s support for adoption of policies and regulations been effective/or not effective? 1.5 What are the conditions that facilitate the adoption (authorization) of these policies, regulations and tools in your province and communities? Please provide answers for each policy, approach, regulation, and tool. 1.6 What are the conditions or factors that hinder or limit the progression of these policies, regulations and tools to reach the final stage of authorization/adoption in your province and communities? Please provide answers for each policy, approach, regulation, and tool. 1.7 Are these regulations and/ or tools introduced by USAID SEA Project and adopted by relevant authorities in your province and communities lead to changing the following aspects? 1.7.1 Increased the number of people demonstrating behaviors that contribute to biodiversity conservation, as a result of implementing the strategy of “creating demand through awareness raising advocacy? If yes, how and under what conditions are these achieved? If no, what have been the reasons? 1.7.2 Increased the number of people exposed to USAID SEA Project messages through communication and media outreach? If yes, how and under what conditions are these achieved? If no, what have been the reasons? 1.7.3 Increased the number of champions organized and enabled to deliver messages? If yes, how and under what conditions are these achieved? If no, what have been the reasons? 1.7.4 Increased the number of people exhibiting behavior change towards biodiversity conservation and marine sustainable practices? If yes, how and under what conditions are these achieved? If no, what have been the reasons? 1.7.5 Increased the number of forums or official groups established and functional for awareness raising and advocacy? If yes, how and under what conditions are these achieved? If no, what have been the reasons? EQ 2: Under what conditions do people trained to manage marine and fisheries resources and/or advocate for sustainable management of these resources, as a result of USAID SEA interventions, apply their skills in marine and fisheries resources management or advocacy? 2.1 In your province, how did the USAID SEA Project intervene to train and inform stakeholders and partners and apply their skills learned to manage marine and fisheries resources and advocacy? Probe its application in the frameworks of EAFM, MPA, MSP, law 140 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences Key Informant enforcement, non-fishing livelihood activities, marine tenure and managed access to marine resources. 2.2 How effective is the USAID SEA Project support in your province and communities in facilitating the application of knowledge and skills acquired? 2.3 In what ways do people and partners in your province and communities apply their learnings in marine fisheries management or advocacy? 2.4 What are the factors or conditions that facilitate/assist people and partners apply their learnings in your province and communities? 2.5 In your province, what are the factors or conditions that make it difficult for trained people to apply their acquired knowledge and skills? EQ 3: To what extent do non‐fishing livelihood activities introduced or strengthened by USAID SEA change behavior and reduce threats and improve marine conservation (e.g., Livelihood program in Mare, Ternate, Galo‐ Galo, and Morotai)? Under what conditions are these activities effective? 3.1 How was the strategy – “Increase Incentives through non-fishing livelihood activities” implemented in your province and communities? 3.2 What types of non-fishing livelihood activities were introduced or supported by SEA Project in your province and communities? Have they been effective in increasing the following? If Yes, in what ways and what have been the results? If No, what have been the reasons? 3.2.1 Increase in the number of public-private partnerships with focused on sustainable fisheries and marine conservation with leveraged resources quantified in your province and communities? 3.2.2 Increase in the number of people receiving social, ecological, and economic benefits derived from conservation and sustainable use of marine resources in your province and communities? EQ 4: To what extent do USAID‐supported fishermen with more secure marine tenure in fisheries apply sustainable fishing practices and reduce destructive fishing practices (e.g., Territorial Use Rights for Fishing [TURF] in Raja Ampat and Local Marine Management Area [LMMA] in Maluku)? Under what conditions are these activities effective? 4.1 In your province and communities, how did USAID SEA Project supported fishermen with more secure tenure or managed access to fisheries resources (e.g. LMMA, CFA, TURF, and TCBFMA)? 4.2 How effective are the secure tenure or managed access activities in your province and communities? If Yes, in what ways, and what have been the results? If No, what have been the reasons? Which of these tenure activities (LMMA, CFA, TURF, and TCBFMA) are effective in applying sustainable fishing practices 141 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences Key Informant and reduction of destructive fishing practices in your province and communities? 4.3 Under what conditions are the secure tenure or managed access to fisheries resources effective or not effective in your province and communities? (Probing question: Can you cite evidences of effectiveness or ineffectiveness of each tenure instrument?). EQ 5: Given that USAID SEA addresses key marine threats and drivers through a complex project comprised of multiple components / strategic approaches implemented by a large number of partners, how well are all the components /approaches working together? What aspects of overall management are working well and what can be improved? To what extent are there capacity gaps among the partners, which might be hindering effective implementation? 5.1 In your province, what actions have the project taken to maximize synergies? How effectively are all the components and approaches of the SEA Project working together? What additional actions need to be taken to maximize synergies? (Probe: How is this done in relation to application of frameworks of EAFM, MPA, MSP, non-fishing livelihood activities, law enforcement, and establishment of marine tenure and managed access to marine resources?) 5.2 How effective has the SEA Project been in coordinating the implementation of the different project components and approaches in your province and communities? 5.3 Can you cite specific evidences where all these components/approaches of the SEA Project are working together to achieve marine conservation in your province and communities? Are all these effective when working together? If Yes, in what ways, and what have been the results? If No, what have been the reasons? 5.4 What aspects of overall management are working well and what can be improved in your province and communities? 5.5 Are there capacity gaps among partners which might be hindering effective implementation in your province and communities? What are these capacity gaps (per partner)? What did SEA Project do to address these gaps? Are these actions effective in arresting/ responding to the gaps. What further recommendations or courses of action to take to address these gaps? 5.6 In your province, how effective are relationships / working arrangements between USAID SEA Project and subcontractors, and among subcontractors? What other areas could be improved? 5.7. What recommendation / suggestion or course of action to take to improve more (or make more successful/effective) the overall implementation of the SEA Project in your province and communities? 142 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences Key Informant EQ 6: Based on the answers above about the conditions under which the USAID SEA project strategies are or are not effective, what are the recommended areas (thematic and/or geographic) of expansion/ contraction for USAID? 6.1 Which of the project’s strategies are still relevant and effective at this point in time (mid-term implementation) in achieving the goals of the SEA project in your province and communities? What are the strategies that are still relevant? Why? What are strategies that are not relevant? Why 6.2 Which project strategies (thematic and/or geographic) need changes/improvement in your province and communities? Why? 6.3 What are the areas where the project needs to either expand or contract its work in relation to specific strategies and/or geographic scope in your province and communities? Why? 6.4 What actions are needed to improve the overall implementation of the SEA Project in your province and communities? Why? 4. Closing/Sign-Off Questions Again, assure the respondent that all information obtained shall remain confidential and will not be used for purposes other than what have stated. Finally, ask permission from the Key Informant for any possible follow-up communications with them for clarifications or additional inputs to shed more lights on specific topics discussed during the KII. Say “Thank You” before you leave. Schedule of KII: Date of Interview: Time Started: Time Ended: Venue of KII: Name of KII Interviewer: Name of Documenter: 143 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV SUSTAINABLE ECOSYSTEMS ADVANCED (SEA) PROJECT Midterm Performances Evaluation (MTPE) KII Guide Questions KII Tool 2.b. Project Leaders for SEA Project – WWF, CTC, AP2HI, MDPI, ILMMA, WCS, RCFI, RARE, CI, UKIP, UNIPA, and Marine Change KII Guide Questions for: Project Leaders for SEA project (i) WWF for SEA Project (ii) CTC for SEA Project (iii) AP2HI for SEA Project (iv) MDPI for SEA Project (v) ILMMA for SEA Project (vi) WCS for SEA Project (vii) RCFI for SEA project (viii) RARE for SEA Project (ix) CI for SEA Project (x) UKIP for SEA Project (xi) UNIPA for SEA Project (xii) Marine Change Main Focus: Overall project management, coordination, communication, in each SEA Project in the province of Maluku, North Maluku, and West Papua in the following aspects: SA1 - Create demand thru awareness raising and advocacy; SA2 - Improve ecosystem management of FMA-715 and MPAs; SA2-TA1- Sustainable Fisheries thru ecosystem approach to fishery management (EAFM); SA2-TA2 - MPA Management; SA2-TA3 - MSP (Marine Spatial Planning); SA2-TA4 - Law Enforcement; SA3 - Increase incentives for marine stewardship thru non-fishing livelihood activities, and more secure tenure for fisheries resources; SA4 - Advance policy and regulations; and SA5 - Institutionalize training and capacity building. 1. Introduction and Background to the SEA Project MTPE Introduction of KII Interviewer, and Documenter cum interpreter Good day! My name is ___________, KII Interviewer, and my companion is _____________, KII Documenter, of the Pacific Rim Innovation and Management Exponents, Inc. (PRIMEX). We are part of the MTEP Evaluation Team, commissioned by USAID/Indonesiato conduct an evaluation of the USAID SEA Project. Our team in particular is conducting key informant interview (KII) with officials and technical staff of SEA Project (Tetra Tech ARD), MMAF, and representatives of selected Project partners/subcontractors who are involved and are knowledgeable about SEA Project activities. The KII is important to achieve the MTPE objectives. And we would like to seek your understanding for this interview, which may take about one and a half (1.5) hours to two (2) hours. At this early stage, we wish to express our sincerest appreciation to you for participating in this KII session. 144 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Protocol on KII session ✓ It is important that the key informant is provided with enough time to express/expound his/her answers. But be sure to focus on the subject of interview. ✓ Confidentiality: Assure the respondent that all the data and information to be provided will be treated with utmost confidentiality and will be used mainly to undertake an objective assessment of the SEA Project’s performance. The name of the respondent will not, in anyway, be associated with any specific answers/comments. All the data and information will be collated and analyzed, together or in combination with other data and information collected from other KII respondents. ✓ There will be NO right or wrong answer. Encourage the respondent to feel free and be totally honest and objective in the answers or comments/opinion to be provided. ✓ The FGD session will be recorded; ask permission from the respondents if he/she will allow recording of her/his responses for documentation purposed. ✓ Before the KII begin, ask the respondent for any questions, concerns, or clarifications. 2. Basic Information about the KII Respondent Name of Respondent: Position: Agency/Office and Address Contact Information: Telephone No. Mobile No. E-mail Ad Note: Make sure to get the contact addresses (email and contact number of key informant for possible follow-up communications/calls for clarification on his/her responses. 3. Understanding the Role and Focus of the Organization/Institution/Office in the SEA Project What is your Orgnanization’s /Office’s main goal or primary interest in the marine and fisheries resources at the areas where you are (provincial and community sites)? What were your activities relative to these resources before the USAID SEA Project? What activities are you doing/implementing at the areas where you are ( provincial and community sites) in relation to USAID SEA Project? National National Provincial Provincial Community Community 145 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 4. Main Body of Guide Questions Questions Responses/Evidences Key Informant EQ1: Under what conditions are the regulations and/or tools being introduced by USAID SEA to relevant authorities leading to the adoption of these policies, regulations, and tools? 1.1 In your specific project, how did the SEA Project create demand through awareness raising and advocacy of numerous marine and fisheries policies it developed? (Possible probe question: How did the SEA Project informed and facilitated partnership with your organization in raising awareness and advocacy of its numerous policies and regulations and its adoption in areas where your project is located?). Please mention the specific policies, regulations, approaches and tools. 1.2 How did your organization support and assisted the planning, implementation and adoption of these numerous policies, regulations and tools in your areas? How effective/not effective in accomplishing authorization and adoption of these policies, regulations, and tools in your area? How and why? 1.3 What are the conditions that facilitate the adoption (authorization) of these policies, regulations and tools in your areas and communities? Please provide answers for each policy, approach, regulation, and tool. 1.4 What are the conditions or factors that hinder or limit the progression of these policies, regulations and tools to reach the final stage of authorization/ adoption in your areas and communities? Please provide answers for each policy, approach, regulation, and tool. 1.5 Are these regulations and/ or tools introduced by USAID SEA Project and adopted by relevant authorities in your areas and communities lead to changing the following aspects? 1.5.1 Increased the number of people demonstrating behaviors that contribute to biodiversity conservation, as a result of implementing the strategy of “creating demand through awareness raising advocacy? If yes, in what ways and what have been the results? If no, what have been the reasons? 1.5.2 Increased the number of people exposed to USAID SEA Project messages through communication and media outreach? If yes, in what ways and what have been the results? If no, what have been the reasons? 1.5.3 Increased the number of champions organized and enabled to deliver messages? If yes, in what ways and what have been the results? If no, what have been the reasons? 15.4 Increased the number of people exhibiting behavior change towards biodiversity conservation and marine sustainable practices? If yes, in what ways and what have been the results? If no, what have been the reasons? 15.5 Increased the number of forums or official groups established and functional for awareness raising and advocacy? If yes, in what ways and what have been the results? If no, what have been the reasons? 146 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences Key Informant EQ 2: Under what conditions do people trained to manage marine and fisheries resources and/or advocate for sustainable management of these resources, as a result of USAID SEA interventions, apply their skills in marine and fisheries resources management or advocacy? 2.1 In your organization and areas/communities, how did the USAID SEA Project intervene to train and inform stakeholders and partners and apply their skills learned to manage marine and fisheries resources and advocacy? Probe its application in the frameworks of EAFM, MPA, MSP, law enforcement, non-fishing livelihood activities, marine tenure and managed access to marine resources. 2.2 How effective is the USAID SEA Project support in your organization’s areas and communities in facilitating the application of knowledge and skills acquired? 2.3 In what ways do people and partners in your organization’s areas and communities apply their learnings in marine fisheries management or advocacy? 2.4 What are the factors or conditions that facilitate/assist people and partners apply their learnings in your organization’s areas and communities? 2.5 In your organization’s areas and communities, what are the factors or conditions that make it difficult for trained people to apply their acquired knowledge and skills? EQ 3: To what extent do non‐fishing livelihood activities introduced or strengthened by USAID SEA change behavior and reduce threats and improve marine conservation (e.g., Livelihood program in Ternate, North Maluku, Galo‐Galo, and Morotai)? Under what conditions are these activities effective? 3.1 How was the strategy “Increase Incentives through non-fishing livelihood activities implemented in your organization’s areas and communities? How did your organization support USAID SEA project in increasing incentives through non-fishing livelihood activities? How effective/not effective are these support? Why? 3.2 What types of non-fishing livelihood activities were introduced or supported by SEA Project in your organization’s area and communities that you participated? Have they been effective in increasing the following? If Yes, in what ways and what have been the results? If No, what have been the reasons? 3.2.1 Increase in the number of public-private partnerships with focused on sustainable fisheries and marine conservation with leveraged resources quantified in your organization’s area and communities? 3.2.2 Increase in the number of people receiving social, ecological, and economic benefits derived from conservation and sustainable use of marine resources in your organization’s areas and communities? EQ 4: To what extent do USAID‐supported fishermen with more secure marine tenure in fisheries apply sustainable fishing practices and reduce destructive fishing practices (e.g., Territorial Use Rights for Fishing [TURF] in Raja Ampat and Local Marine Management Area [LMMA] in Maluku)? Under what conditions are these activities effective? 147 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences Key Informant 4.1 How did USAID SEA Project through its partnership with your organization supported fishermen with more secure tenure or managed access to fisheries resources (e.g. LMMA, CFA, TURF, and TCBFMA)? 4.2 How effective are the secure tenure or managed access activities in your organization’s areas and communities in applying sustainable fishing practices and reduce destructive fishing practices? If Yes, in what ways, and what have been the results? If No, what have been the reasons? Which of these tenure activities are your organizations implementing in your areas and communities (LMMA, CFA, TURF, and TCBFMA)? How effective are these activities in applying sustainable fishing practices and reduction of destructive fishing practices in your organization’s areas and communities? 4.3 Under what conditions are the secure tenure or managed access to fisheries resources effective or not effective in your organization’s areas and communities? (Probing question: Can you cite evidences of effectiveness or ineffectiveness of each tenure instrument?). EQ 5: Given that USAID SEA addresses key marine threats and drivers through a complex project comprised of multiple components / strategic approaches implemented by a large number of partners, how well are all the components /approaches working together? What aspects of overall management are working well and what can be improved? To what extent are there capacity gaps among the partners, which might be hindering effective implementation? 5.1 What actions has the project through its partnerships with your organization taken to maximize synergies? How effectively are all the components and approaches of the SEA Project working together? What additional actions need to be taken to maximize synergies? (Probe: How is this done in your organization’s areas and communities? How does your organization/operation contribute to achieving these synergies?) 5.2 How effective has the SEA Project through its partnership with your organization been in coordinating the implementation of the different project components and approaches in your area and communities? 5.3 Can you cite specific evidences where all these components/approaches of the SEA Project are working together to achieve marine conservation in your organization’s area and communities? Are all these effective when working together? If Yes, in what ways, and what have been the results? If No, what have been the reasons? 5.4 What aspects of overall management are working well and what can be improved in your organization’s area and communities? 5.5 Are there capacity gaps in your organization/operation and among other partners which might be hindering effective implementation in your organization’s areas and communities? What are these capacity gaps (in your organizations/operation and other partner)? What did 148 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences Key Informant SEA Project and your organization do to address these gaps? Are these actions effective in arresting/responding to the gaps. What further recommendations or courses of action to take to address these gaps? 5.6 How effective are relationships/working arrangements between USAID SEA Project and your organization and other partners and among subcontractors? What other areas could be improved? 5.7. What recommendation/ suggestion or course of action to take to improve more (or make more successful/ effective) the overall implementation of the SEA Project in your organization and areas and communities? (Probe: What recommendation/suggestion or course of action your organization should take to improve more (or make more successful/effective) the overall implementation of the SEA Project in your areas and communities? EQ 6: Based on the answers above about the conditions under which the USAID SEA project strategies are or are not effective, what are the recommended areas (thematic and/or geographic) of expansion/contraction for USAID? 6.1 Which of the project’s strategies are still relevant and effective at this point in time (mid-term implementation) in achieving the goals of the SEA project in your organization’s areas, and communities? What are the strategies that are still relevant? Why? What are strategies that are not relevant? Why 6.2 Which project strategies (thematic and/or geographic) need changes/improvement in your organization’s areas, and communities? Why? 6.3 What are the areas where the project needs to either expand or contract its work in relation to specific strategies and/or geographic scope in your organization’s areas and communities? Why? 6.4 What actions are needed to improve the overall implementation of the SEA Project in your organization’s areas and communities? Why? How can your organization support these actions? Why? 5. Closing/Sign-Off Questions Again, assure the respondent that all information obtained shall remain confidential and will not be used for purposes other than what have stated. Finally, ask permission from the Key Informant for any possible follow-up communications with them for clarifications or additional inputs to shed more lights on specific topics discussed during the KII. Say “Thank You” before you leave. Schedule of KII: Date of Interview: Time Started: Time Ended: Venue of KII: Name of KII Interviewer: Name of Documenter: 149 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV SUSTAINABLE ECOSYSTEMS ADVANCED (SEA) PROJECT Midterm Performances Evaluation (MTPE) KII Guide Questions KII Tool 2.c: Managers of FMA, MPA, and MSP at the Site Levels in Maluku, North Maluku, and West Papua KII Guide Questions for: a) Managers of FMA, MPA, and MSP at the site Levels in Maluku, North Maluku, and West Papua Main Focus: SA2 - Improve ecosystem management of FMA-715 and MPAs; SA2-TA1- Sustainable Fisheries thru ecosystem approach to fishery management (EAFM); SA2-TA2 – Sustainable Fisheries through MPA Management; SA2-TA3 – Sustainable Fisheries through Marine Spatial Planning Management (MSP); SA2-TA4 – Sustainable Fisheries through Law Enforcement; SA3 - Increase incentives for marine stewardship thru non-fishing livelihood activities, and more secure tenure for fisheries resources; SA4 – Advance Policy and Regulations SA5 – Institutionalize Training and Capacity Building 1. Introduction and Background to the SEA Project MTPE Introduction of KII Interviewer, and Documenter cum interpreter Good day! My name is ___________, KII Interviewer, and my companion is _____________, KII Documenter, of the Pacific Rim Innovation and Management Exponents, Inc. (PRIMEX). We are part of the MTEP Evaluation Team, commissioned by USAID/Indonesia to conduct an evaluation of the USAID SEA Project. Our team in particular is conducting key informant interview (KII) with officials and technical staff of SEA Project (TETRA TECH), MMAF, and representatives of selected Project partners/subcontractors, who are involved and are knowledgeable about SEA Project activities. The KII is important to achieve the MTPE objectives. And we would like to seek your understanding for this interview, which may take about one and a half (1.5) hours to two (2) hours. At this early stage, we wish to express our sincerest appreciation to you for participating in this KII session. Protocol on KII session ✓ It is important that the key informant is provided with enough time to express/expound his/her answers. But be sure to focus on the subject of interview. ✓ Confidentiality: Assure the respondent that all the data and information to be provided will be treated with utmost confidentiality and will be used mainly to undertake an objective assessment of the SEA Project’s performance. The name of the respondent will not, in anyway, be associated with any specific answers/comments. All the data and information will be collated and analyzed, together or in combination with other data and information collected from other KII respondents. ✓ There will be NO right or wrong answer. Encourage the respondent to feel free and be totally honest and objective in the answers or comments/opinion to be provided. 150 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV ✓ The FGD session will be recorded; ask permission from the respondents if he/she will allow recording of her/his responses for documentation purposed. ✓ Before the KII begin, ask the respondent for any questions, concerns, or clarifications. 2. Basic Information about the KII Respondent Name of Respondent: Position: Agency/Office and Address Contact Information: Telephone No. Mobile No. E-mail Ad Note: Make sure to get the contact addresses (email and contact number of key informant for possible follow-up communications/calls for clarification on his/her responses. 3. Understanding the Role of MFA, Marine Protected Area (MPA), Marine Spatial Planning (MSP), etc. in the SEA Project What is your Office’s main goal or primary interest in the marine and fisheries resources at the provincial, and community? What were your activities relative to these resources before the USAID SEA Project? What activities are you doing/implementing at the provincial and community in relation to USAID SEA Project? Provincial Provincial Community Community 4. Main Body of Guide Questions Questions Responses/Evidences Key Informant EQ 1: Under what conditions are the regulations and/or tools being introduced by USAID SEA to relevant authorities leading to the adoption of these policies, regulations, and tools? 1.1 What are the supports of USAID SEA Project for the development and advancement of policies, regulations and/or tools related to a range of government frameworks on FMA, MPA, MSP, management/sustainable fishing. How did your office support these efforts? 1.2 Are these policies, regulations and tools that have been authorized effectively implemented to improve FMA, MPA, MSP management in marine and fisheries resources at the national, provincial, and community levels? If Yes, in what ways, and what have been the results? If No, what have been the reasons? 1.3 Are these regulations and/or tools introduced by USAID SEA Project assistance and adopted by relevant authorities lead to increased in number of hectares of biological significance and/or natural 151 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences Key Informant resources under improved natural resource management applying FMA, MPA, MSP ? If yes/no, why? 1.4 What are the conditions that facilitate the adoption (authorization) of these policies, regulations and tools, applying FMA, MPA, MSP Management? Please provide answers for each policy, approach, regulation, and tool. 1.5 What are the conditions or factors that hinder or limit the progression of these policies, regulations and tools to reach the final stage of authorization/adoption applying FMA, MPA, MSP management? Please provide answers for each policy, approach, regulation, and tool. EQ 2. Under what conditions do people trained to manage marine and fisheries resources and/or advocate for sustainable management of these resources, as a result of USAID SEA interventions, apply their skills in marine and fisheries resources management or advocacy? 2.1 How is your (FMA/MPA/MSP) office making use of the USAID SEA Project interventions to train and inform people and partners to manage marine and fisheries resources and/or advocate for sustainable management of these resources to enable them to apply their skills learned in marine fisheries management or advocacy? 2.2 How effective are the USAID SEA Project support through your office (FMA, MPA, MSP) management application in facilitating/enhancing the skills and learnings on natural resource management of stakeholders and partners? 2.3 How effective are the USAID SEA Project support through your office (FMA, MPA, MSP) in helping stakeholders and partners in applying their learnings on natural resource management or advocacy? In what ways do stakeholders and partners apply their learnings? 2.4 What are the conditions that facilitate the adoption and institutionalization of your office (FMA, MPA, MSP)? What are the conditions or factors that hinder or limit the adoption and institutionalization of FMA, MPA, MSP management? 2.5 With the introduction of FMA, MPA, MSP management by the USAID SEA Project through your assistance, what factors or conditions emerges that facilitate/assist people and partners apply their learnings? 2.6. In an FMA, MPA, MSP management, what are the factors or conditions that make it difficult for trained people to apply their acquired knowledge and skills? EQ 3: To what extent do non‐fishing livelihood activities introduced or strengthened by USAID SEA change behavior and reduce threats and improve marine conservation (e.g., Livelihood program in Ternate, North Maluku, Galo‐Galo, and Morotai)? Under what conditions are these activities effective? 3.1 What is your office’s role in supporting the SEA Project implement FMA, MPA, MSP Management in non-fishing livelihood activities? 3.2 Did USAID SEA applying FMA, MPA, MSP management strengthened non-fishing livelihood activities in North Maluku,, Galo-Galo, and 152 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences Key Informant Morotai that lead to change behavior and reduction of threats and improvement of marine conservation? If yes, how and why? If no, why? 3.3 What are the conditions or factors that contribute to the effectivity of non-fishing livelihood activities to change behavior of people and partners for reduction of threats and sustainable marine conservation? 3.5 What are the conditions or factors that diminish the effectivity of these non-fishing livelihood activities to reduce threats and improve marine conservation in the selected project sites? EQ 4: To what extent do USAID‐supported fishermen with more secure marine tenure in fisheries apply sustainable fishing practices and reduce destructive fishing practices (e.g., Territorial Use Rights for Fishing [TURF] in Raja Ampat and Local Marine Management Area [LMMA] in Maluku)? Under what conditions are these activities effective? 4.1 What is your office’s involvement in supporting USAID SEA FMA, MPA, MSP management to provide secure tenure or managed access to fisheries resources (e.g. LMMA, CFA, TURF, and TCBFMA)? 4.2 How effective are the secure tenure or managed access activities in applying sustainable fishing practices and reduce destructive fishing practices? If Yes, in what ways, and what have been the results? If No, what have been the reasons? Which of these tenure activities (LMMA, CFA, TURF, and TCBFMA) are effective in applying sustainable fishing practices and reduction of destructive fishing practices? 4.3 Under what conditions are the secure tenure or managed access to fisheries resources effective or not effective? (Probing question: Can you cite evidences of effectiveness or ineffectiveness of each tenure instrument?). EQ 5: Given that USAID SEA addresses key marine threats and drivers through a complex project comprised of multiple components / strategic approaches implemented by a large number of partners, how well are all the components /approaches working together? What aspects of overall management are working well and what can be improved? To what extent are there capacity gaps among the partners, which might be hindering effective implementation? 5.1 What does the USAID SEA FMA, MPA, MSP management provide to ensure that all the components/approaches of the SEA Project are implemented or working together in the project sites to achieve marine conservation? What is you office’s role in making all the components/approaches work together in your project sites to achieve marine conservation? 5.2 How effective has the SEA Project through EAFM been in coordinating the implementation of the different project components and approaches? 5.3 Can you cite specific evidences where all these components/approaches of the SEA Project are working together to achieve marine conservation? Are all these effective when working 153 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Questions Responses/Evidences Key Informant together? If Yes, in what ways, and what have been the results? If No, what have been the reasons? 5.4 What is your office’s involvement in the USAID SEA Project to ensure that all the components/approaches are effectively working together? How and why? 5.5 What aspects of overall management of the SEA Project are working well and what can be improved? 5.6 Are there capacity gaps in your office/operation and among other partners which might be hindering effective implementation? What are these capacity gaps (in your operation or other partner)? What did SEA Project do to address these gaps? Are these actions effective in arresting/responding to these gaps. What further recommendations or courses of action to take to address these gaps? 5.7 How effective are relationships/working arrangements between USAID SEA Project and your office/operation, and among subcontractors? What other areas could be improved? 5.8 What recommendation / suggestion or course of action to take to improve more (or make more successful/effective) the overall implementation of the SEA Project? EQ 6: Based on the answers above about the conditions under which the USAID SEA project strategies are or are not effective, what are the recommended areas (thematic and/or geographic) of expansion/contraction for USAID? 6.1 Which of the project’s strategies are still relevant and effective at this point in time (mid-term implementation) in achieving the goals of the SEA project? What are the strategies that are still relevant? Why? What are strategies that are not relevant? Why 6.2 Which project strategies (thematic and/or geographic) need changes/improvement? Why? 6.3 What are the areas where the project needs to either expand or contract its work in relation to specific strategies and/or geographic scope? Why? 6.4 What actions are needed to improve the overall implementation of the SEA Project? Why? 154 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 5. Closing/Sign-Off Questions Again, assure the respondent that all information obtained shall remain confidential and will not be used for purposes other than what have stated. Finally, ask permission from the Key Informant for any possible follow-up communications with them for clarifications or additional inputs to shed more lights on specific topics discussed during the KII. Say “Thank You” before you leave. Schedule of KII: Date of Interview: Time Started: Time Ended: Venue of KII: Name of KII Interviewer: Name of Documenter: 155 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV ANNEX H. LIST OF KII INFORMANTS No Names Position and Institution 1 Ms. Celly Catharina Ahmad Hafizh Adyas Marine Manager, USAID Indonesia 2 Alan T. White, Ph. D.; and Tine Gunawan, Ph. D. Chief of Party, USAID SEA Deputy Chief of Party, USAID SEA 3 Dame Siahaan Monitoring and Evaluation Specialist, USAID SEA 4 Annisa Budiayu Program Manager, North Maluku, USAID SEA 5 Arsiel Djunaidi Ii Rosna Irene Sahertian Regional Technical Director, USAID SEA Regional MEL Specialist, USAID SEA Maluku Provincial Coordinator, USAID SEA 6 Evi Nurul Ihsan Muhamad Korebima Conservation Coordinator, USAID SEA Government Relation and Outreach Specialist, USAID SEA 7 Irna Sari Ses Rini Mardiani Muhammad Ichsan Sustainable Fisheries Advisor, USAID SEA Sustainable Fisheries Specialist, USAID SEA Sustainable Fisheries Coordinator, USAID SEA 8 Zulficar Mochtar, S.T, M.Sc. Director General Capture Fisheries, Ministry of Marine and Fisheries Affairs (MMAF) 9 Amer Head of sub-directorate of Kawasan Konservasi Hayati Laut (KKHL) Marine Biodiversity and Conservation Area, MMAF 10 Ir. Ishartini Head of Planning Bureau, Secretariate General Marine and Fisheries Affairs, MMAF 11 Agung Tri Prasetyo, S.Si., M.A. Head of Public Relation and Cooperation, MMAF 12 Husni Mubarok Head of Legal, Bureau of Law, MMAF 13 Syahril A. Raup Head of Monitoring and Fisheries Resources Management (SDI), Capture Directorate General, MMAF 14 Sofyan Hasan, S.Pi, M.Si Head of sub-directorate National Strategic Area under Directorate General of Marine Spatial Management, MMAF 15 Dr. Irwan Muliawan Researcher for Socio-economic Unit under Board of Research and Human Resources, MMAF 16 Dr. Najamuddin Rustam Paembonan Marine Chemistry, Lecturer, University of UNKHAIR GIS Expert, Lecturer, University of UNKHAIR 17 Mr. Steven Community Officer, WCS, North Maluku 18 Mr. Kusnadi Mr. Hamka Karepesina Ms. Dewi Deputy Team Leader, WCS, North Maluku Marine Protected Area MPA Specialist, WCS, North Maluku Sustainable Fisheries Specialist, WCS, North Maluku 19 Mr. Maskur Tamanyira Small Scale Fisheries Project, WWF North Maluku 20 Mr. Buyung Radjiloen Head, North Maluku Provincial Marine and Fisheries Services 21 Mr. Henry Erafat, S.T Head of Sub-directorate Marine and Fisheries, Directorate General Regional Development, Ministry of Home Affairs (MOHA) 156 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV No Names Position and Institution 22 Karel Yerusa; and Sri Jalil Regional Supervisor, MDPI, Supervisor for Fair Trade, MDPI 23 Dr. M. Janib, S.Pi, M.Sc Dean of Marine and Fisheries Faculty, University of Khairun Ternate 24 M. T. Fabanyo Head, Marine and Fisheries Product Competitiveness, North Maluku Marine and Fisheries Services 25 Nur Khalish Head of Economic Division of Tidore Municipality Planning Board 26 Didik S. Suawarsono Kurniawan Ishak Head of Sub-division of Cooperation and Public Relation of Marine and Fisheries Surveillance Directorate and Head of Sub-directorate for Marine Surveillance 27 Arief Husein Kabid Pengawasan Dan SDM, DKP Maluku 28 Sandra Tjan Program Manager, Indonesia Locally-Managed Marine Area (ILMMA) 29 Irwanto Site Coordinator, West Papua, 30 Hari Kushardanto Program Director, RARE 31 Ridwan Sala Fanny Fransina Carolina Simatauw Sevly Tebay Dean, Marine and Fisheries Faculty Head of Marine and Fisheries Department, University of Papua (UNIPA) Head of Mitra Bahari Consorsium of West Papua and Support Dean II 32 Roger Tabalessy Lecturer, Faculty of Marine Science and Fisheries, UKIP 33 Rahel Archy W. Randa S.Pi; Wahyu Ramandey Head of Section for Protection of Structuring and Utilization of Biological Conservation Areas and Marine Services, Marine and Fisheries Service (DKP) Head of Section Coastal Utilization and Marine Spatial Planning, DKP 34 Christovel Reinhard S. Rotinsulu West Papua Coordinator 35 Gayus Raunsay Field Coordinator, RARE West Papua 36 Fatma Rumagia Kepala Seksi Konservasi Dan Keanekaragaman Hayati Laut 37 Miko Budi Raharjo Site Coordinator Seram Seas, WWF 38 Ricardo Siringoringo Moh Nurcahyadi Siwi D. Widodo Pokja Data Dan Informasi, PPN – FMA 715 Pokja Kepatuhan, PPN – FMA 715 Pokja Konservasi, PPN – FMA 715 39 I Made Malehartadena Isnawati Manager Quality Assurance Officer PT Harta Samudra, Harta Samudra Corporate Office, Desa DAEO, Majiko, South Morotai 157 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV ANNEX I. FGD GUIDE QUESTIONS SUSTAINABLE ECOSYSTEMS ADVANCED (SEA) PROJECT Midterm Performances Evaluation (MTPE) FGD Guide Questions FGD Tool 1.a. Local Communities/CBO Involved in Non-Fishing Livelihood Activities Focus Group: Local Community/Community-Based Organization (CBO) members; with two separate gender-based FGD sessions Main Focus: Non-fishing livelihood activities (EQ3) introduced or supported by USAID SEA Project to determine and evaluate to what extent are these activities leading to changes in local people’s behavior, reduction of threats to marine and fisheries resources management, and improvement of marine conservation; and under what conditions are these activities effective in the selected Project sites. Secondary Focus: Contributions of relevant policies, regulations and/or tools introduced by USAID SEA Project (EQ1); trainings provided by the Project (EQ2); more secure marine tenure in fisheries (EQ4); other Strategic Approaches, such as the creation of demand through awareness raising and advocacy (SA1), and the improvement of ecosystem management of FMA-715 and MPAs (or LLMAs) (SA2) through Technical Approaches; and all the Project components/Strategic Approaches when “working together” (EQ5). Target Sites: For confirmation with USAID SEA Project a) Mare, Ternate, North Maluku Province; and b) Galo-Galo, Morotai, North Maluku Province. 1. Introduction and Background to the SEA Project MTPE Introduction of Interviewer, Facilitator cum Interpreter, and Documenter Good day! My name is ___________, FGD Facilitator, and my companion is _____________, FGD Documenter, of the Pacific Rim Innovation and Management Exponents, Inc. (PRIMEX). We are part of the MTEP Evaluation Team, commissioned by USAID/Indonesia to conduct an evaluation of USAID SEA Project, which is implementing non-fishing livelihood activities in your community. Our Team, in particular, is conducting focus group discussion (FGD) with members of your community/CBOs in the selected Project sites (Mare, Terfnate; or Galo-Galo, Morotai) who are directly involved in non￾fishing livelihood activities. This FGD is important to achieve the MTPE objectives, which include: (i) review the Project midterm performance, particularly the non-fishing livelihood activities in your community; (ii) analyze factors and conditions that enhance or diminish the achievements of the target outputs and outcomes of these activities; and (iii) recommend specific actions to further improve implementation effectiveness of these activities. And we would like to seek your support 158 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV for the completion of this FGD as it may take about one and a half (1.5) hours to two (2) hours. At this early stage, we wish to express our sincerest appreciation to your CBO/s and community for participating in this FGD session. Protocols on FGD session ✓ It is important that each of the FGD participants is given enough time to express his/her ideas in order to articulate his/her answers. So, ensure that everybody is heard in the FGD session. ✓ Confidentiality: Assure the participants that all the data/information they will provide will be treated with utmost confidentiality, and will be used mainly to undertake an objective assessment of USAID SEA Project’s performance. The names of participants will not, in anyway, be associated with any specific answers/comments provided. All responses will be collated and analyzed, together or in combination with other data and information collected from other FGD participants. ✓ There will be NO right or wrong answer; only differing points of view. The participants need not agree with each other’s replies, but must listen respectfully as others share their views. Encourage the participants to feel free, and be honest and objective in the answers or comments/opinion to be provided. ✓ The FGD session will be recorded; hence, each participant is encouraged to speak one at a time. The participants will be called on a first name basis. ✓ Request mobile phones to be turned off or switch to silent mode. If an urgent call needs to be answered, please ask the participants to do so as quietly as possible and rejoin the FGD as quickly as possible. ✓ Before the FGD begin, ask the participants for any questions, concerns, or clarifications. 2. Basic Information about the FGD Participants Notes: • This table can be accomplished in a separate sheet of paper. • Request for any print materials (brochures, accomplishment reports, pictures, etc. about the community/CBOs to serve as evidence/support documents that will substantiate the answers provided) Name of FGD Participant Position / Designation or Role in CBO/s Name of Organization CBO’s’ Areas of Operation (province/ district/site) CBO/s’ Number of years in operation Year started as partner/ beneficiary of USAID SEA Project Number of Members Male Female 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 159 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 3. Understanding the Role/s of FGD Participants’ CBO/s in USAID SEA Project FGD Participant’s Number What is your organization’s main goal or primary interest in the marine and fisheries resources in your community? What were your activities relative to these resources before joining USAID SEA Project? What is your organization’s main interest in joining USAID SEA Project? What activities are you doing/implementing in relation to USAID SEA Project? 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 160 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 4. Main Body of Guide Questions Note: The following Tables can be printed and accomplished in a separate sheet of paper to provide enough space for the answers of each of the FGD participants. 1. Create demand through awareness raising and advocacy (SA1) 1.1 At the start of USAID SEA Project implementation in your community, how did the Project inform you or your CBO/s about its objectives, activities and strategies? (Note: meeting, forum, seminar, communication materials, media, etc.) 1.1.1 How did this information help you or your CBO/s to better understand the Project objectives, activities, and strategies? #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 2. Institutionalize training and capacity building (SA5, EQ2) 2.1 What are important knowledge, skills, attitudes, and practices on fishing and non-fishing livelihood activities that you or your CBO/s learned from the training and capacity building programs provided by USAID SEA Project? (Note: Request participants to be more specific on the types of knowledge, skills, attitudes and practices that they have learned for non-fishing livelihood activities.) #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 2.2 Have you or your CBO/s applied or used the knowledge and skills learned from the training and capacity building programs provided by USAID SEA Project? If YES, in what ways and what have been the results? If No, what have been the reasons? #1: #2: #3: #4: 161 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV #5: #6: #7: #8: #9: #10: 3. Increase incentives for marine stewardship (SA3, EQ3) 3.1 What types of non-fishing livelihood activities (i.e., fish processing and/or marketing, fish storage, marine tourism, boats for rent services, tourist guiding services, etc.) introduced or supported by USAID SEA Project do you or your CBO/s implement in your community? (Note: Request participants to be more specific on the types of activities to identify where such activities fall within the value chain of non-fishing livelihood development.) #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 3.2 What are the roles of male and female members in your non-fishing livelihood activities? #1: Role of men: Role of women: #2: Role of men: Role of women: #3: Role of men: Role of women: #4: Role of men: Role of women: #5: Role of men: Role of women: 162 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV #6: Role of men: Role of women: #7: Role of men: Role of women: #8: Role of men: Role of women: #9: Role of men: Role of women: #10: Role of men: Role of women: 3.3 If you have received training on non-fishing livelihood development provided by USAID SEA Project, what do you consider as the most relevant and effective to your current non-fishing livelihood activities? (Note: Request participants to identify the specific title/s of training/s on non-fishing livelihood development provided to them by the Project.) 3.3.1 Please elaborate in what ways was the training relevant and effective? #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 3.4 What effects do you observe from implementation of your non-fishing livelihood activities on the following: (a) Attitudes, behavior or practices of the members involved in these activities? (b) Threats to marine and fisheries resources in your community? (c) Conservation of marine and fisheries resources in your community? #1: #2: #3: #4: #5: #6: #7: #8: 163 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV #9: #10: 3.5 What are the factors or conditions which make your non-fishing livelihood activities more or less effective in (a) improving people’s behavior, (b) reducing threats to marine and fisheries resources, and (c) improving marine conservation in your community? (Note: Request participants to be more specific on their responses.) 3.5.1 Please elaborate in what ways such activities are more or less effective. #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 4. Facilitate Dialogue on Policies and Regulations (SA4, EQ1) 4.1 What policies, regulations and/or tools introduced or supported by USAID SEA Project are relevant and effective to your non-fishing livelihood activities, if any? (Note: Request participants to identify the specific policies, regulations and/or tools that are relevant and effective to their activities. Get copies of such instruments/tools.) 4.1.1 Please elaborate in what ways are these policies. regulations and/or tools relevant and effective in your activities? #1: Policies: Regulations: Tools: #2: Policies: Regulations: Tools: #3: Policies: Regulations: Tools: #4: Policies: Regulations: Tools: #5: Policies: Regulations: Tools: 164 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV #6: Policies: Regulations: Tools: #7: Policies: Regulations: Tools: #8: Policies: Regulations: Tools: #9: Policies: Regulations: Tools: #10: Policies: Regulations: Tools: 2. Improve ecosystems management of FMA-715 and MPAs 5.1 What specific methods or approaches (i.e., EAFM/sustainable fishing, MPA establishment, MSP and law enforcement) to improve ecosystem management in marine and fisheries resources introduced or supported by USAID SEA Project that are relevant and effective in your non￾fishing livelihood activities? (Note: Request participants to identify the specific methods or approaches that are relevant and effective to their activities. Get copies of evidences like pictures or reports to demonstrate the relevance and effectiveness of specific methods or approaches.) 1.5.1Please elaborate in what ways are the specific methods or approaches relevant and effective in your activities? #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 3. Overall management aspects of non-fishing livelihood activities 6.1 What approaches (i.e., policies, regulations, and/or tools; trainings; livelihoods; more secure marine tenure #1: 165 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV in fisheries; and awareness raising or advocacy) introduced by USAID SEA Project are effective to your non-fishing livelihood activities? (Note: Develop a list of approaches identified by participants as effective.) 6.1.1 Ask whether EACH approach is effective by itself or relies on being implemented with one or more of the other approaches? (Note: Get copies or pictures to document evidences.) #2: #3: #4: #5: #6: #7: #8: #9: #10: 6.2 What capacity gaps, if any, do you or your CBO/s encounter in implementing your non-fishing livelihood activities? (Note: Request participants to identify the specific capacity gaps that need improvement.) #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 6.3 What suggestions or course of action to take to improve (or make successful) the implementation of non-fishing livelihood activities in your community? #1: #2: #3: #4: #5: #6: #7: #8: #9: 166 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV #10: 6.4 What suggestions can you make to USAID for future project design in building Indonesia’s capacity and ability to effectively manage your own fisheries and marine resources? #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 5. Closing/Sign-Off Questions Again, assure the participants that all data/information obtained shall remain confidential, and will not be used for purposes other than what have been stated. Finally, ask permission from participants for possible follow-up communications with them for clarifications or additional inputs to shed more lights on specific topics discussed during the FGD. Ensure to get their complete contact numbers. Say “Thank You” before you leave. Schedule of FGD: Date of the FGD Session: Time Started: Time Ended: Venue of FGD: Name of Facilitator: Name of Documenter: 167 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV SUSTAINABLE ECOSYSTEMS ADVANCED (SEA) PROJECT Midterm Performances Evaluation (MTPE) FGD Guide Questions FGD Tool 1.a. Komunitas Lokal/CBO meliputi kegiatan dalam mata pencaharian bukan perikanan Focus Group: Masyarakat lokal/ anggota dari Organisasi Berbasis Masyarakat (CBO); sesi FGD yang dibagi berdasarkan gender Main Focus: Kegiatan mata pencaharian bukan perikanan(EQ3) yang di perkenalkan atau didukung oleh Proyek SEA USAID untuk menentukan dan mengevaluasi sejauh mana kegiatan ini berdampak pada perubahan perilaku masyarakat lokal, berkurangnya ancaman terhadap pengelolaan sumber daya laut dan perikanan, dan peningkatan konservasi laut; dan dalam kondisi seperti apa kegiatan ini efektif di lokasi Proyek terpilih. Secondary Focus: Kontribusi kebijakan, peraturan, dan / atau alat yang relevan yang diperkenalkan oleh Proyek SEA USAID (EQ1); pelatihan yang diberikan oleh Proyek (EQ2); kepemilikan laut perikanan yang lebih aman (EQ4); pendekatan Strategis lainnya, seperti terbentuknya permintaan melalui peningkatan kesadaran dan advokasi (SA1), dan peningkatan pengelolaan ekosistem FMA-715 dan KKL (atau LLMA) (SA2) melalui Pendekatan Teknis; dan semua komponen Proyek / Pendekatan Strategis pada saat"semuanya bersinergi" (EQ5). Target Lokasi : Untuk di konfirmasikan dengan Proyek SEA USAID a) Mare, Ternate, Provinsi Maluku Utara; dan b) Galo-Galo, Morotai, Provinsi Maluku Utara 1. Pengantar dan Latar belakang dari MTPE Proyek SEA Perkenalan dari Pewawancara, Fasilitator sekaligus Peterjemah dan Dokumenter Selamat Siang! Nama saya ___________, Fasilitator FGD, dan teman saya _____________, FGD Documenter, dari the Pacific Rim Innovation and Management Exponents, Inc. (PRIMEX). Kami adalah bagian dari Tim Evaluasi MTEP, yang ditugaskan oleh USAID / Indonesia untuk melakukan evaluasi Proyek SEA USAID, yang mengimplementasikan kegiatan mata pencaharian bukan-perikanan di komunitas Anda. Tim kami, khususnya, sedang melakukan Focus Group Discussion (FGD) dengan anggota komunitas / CBO Anda di lokasi Proyek terpilih (Mare, Ternate; atau Galo-Galo, Morotai) yang terlibat langsung dalam kegiatan mata pencaharian bukan perikanan. FGD ini penting untuk mencapai hal tersebut yang meliputi: (i) meninjau kinerja jangka menengah Proyek, khususnya kegiatan mata pencaharian bukan-perikanan di komunitas Anda; (ii) menganalisis faktor dan kondisi yang meningkatkan atau mengurangi pencapaian target akhir dan hasil dari kegiatan ini; dan (iii) merekomendasikan kegiatan tertentu untuk lebih meningkatkan efektivitas dari implementasi kegiatan-kegiatan tersebut. Untuk itu kami meminta bantuan Anda untuk membantu menyelesaikan FGD ini yang mungkin memerlukan waktu sekitar 168 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV satu setengah (1,5) hingga dua (2) jam. Pada tahap awal ini, kami ingin menyampaikan penghargaan yang tulus kepada CBO / komunitas Anda untuk berpartisipasi dalam sesi FGD ini. Protokol pada sesi FGD ✓ Adalah penting bahwa setiap peserta FGD diberi cukup waktu untuk mengekspresikan idenya untuk mengartikulasikan jawabannya. Jadi, pastikan semua orang didengar dalam sesi FGD. ✓ Kerahasiaan: Yakinkan para peserta bahwa semua data / informasi yang akan mereka berikan adalah rahasia, dan akan digunakan untuk melakukan evaluasi terhadap kinerja dari Proyek SEA USAID. Nama-nama peserta tidak dihubungkan dengan jawaban/komentar spesifik yang diberikan. Semua tanggapan akan disusun dan dianalisis, bersama-sama atau akan dikombinasikan dengan data dan informasi lain yang dikumpulkan dari peserta FGD lainnya. ✓ TIDAK ada jawaban benar atau salah; tapi hanya sudut pandang yang berbeda. Para peserta tidak perlu setuju dengan jawaban dari masing-masing peserta, namun perlu mendengarkan dengan respek pada saat peserta lainnya memberikan pandangan mereka. Dorong peserta untuk merasa bebas, dan jujur serta objektif dalam jawaban atau komentar / pendapat yang akan diberikan. ✓ Sesi FGD akan direkam; karenanya, setiap peserta diharapkan untuk berbicara satu per satu. Para peserta akan dipanggil berdasarkan nama depan. ✓ Minta peserta untuk mematikan ponselnya atau mengubah ke silent mode. Jika peserta mendapat panggilan darurat, minta peserta untuk melakukannya setenang mungkin dan bergabung kembali dengan FGD secepat mungkin. ✓ Sebelum FGD dimulai, tanyakan pada peserta jika ada pertanyaan, masalah, atau memerlukan klarifikasi. 2. Informasi Dasar mengenai Peserta FGD Catatan: • Tabel ini dapat di isi pada lembar terpisah. • Minta materi cetak (brosur, laporan pencapaian, gambar, dll. Mengenai komunitas/CBOs sebagai barang bukti/ dokumen pendukung yang akan mendukung jawaban yang diberikan) Nama Peserta FGD Posisi / Jabatan atau Peran dalam CBO Nama Organisasi Lokasi operasi CBO (provinsi/ wilayah/lokasi) Tahun beroperasi CBO Tahun di mulai sebagai mitra/penerima dari Proyek SEA USAID Jumlah Anggota Pria Wanita 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 169 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 3. Memahami peran dari peserta FGD CBO didalam Proyek SEA USAID Nomor Peserta FGD Apa tujuan/minat utama dari organisasi anda dalam sumber daya kelautan dan perikanan di dalam komunitas anda? Apa kegiatan anda terhadap sumber ini sebelum anda bergabung dengan Proyek SEA USAID? Apa minat utama dari organisasi anda untuk bergabung dengan Proyek SEA USAID? Kegiatan apa yang anda lakukan/implementasikan yang berhubungan dengan Proyek SEA USAID? 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 4. Bagian Utama dari Pertanyaan Catatan: Tabel berikut dapat dicetak dan di jawab dalam lembar jawaban terpisah untuk memberikan ruang yang cukup jawaban dari masing-masing peserta FGD. 1. Menciptakan permintaan melalui peningkatan kesadaran (awareness) dan avokasi (SA1) 1.1 Pada awal pelaksanaan Proyek SEA USAID di implementasikan di dalam komunitas anda, bagaimana Proyek tersebut memberitahukan anda atau CBO (komunitas berbasis masyarakat) anda mengenai tujuan, aktivitas dan strategi? (Catatan: pertemuan, forum, seminar, alat komunikasi, media, dll) 1.1.1 Bagaimana informasi tersebut membantu anda atau CBO anda (komunitas berbasis masyarakat) untuk lebih memahami tujuan, aktivitas dan strategi Proyek tersebut? #1: #2: #3: #4: 170 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV #5: #6: #7: #8: #9: #10: 2. Kelembagaan pelatihan dan peningkatan kapasitas (SA5, EQ2) 2.1Pengetahuan, keterampilan, sikap dan kegiatan mata pencaharian perikanan dan non-fishing (bukan perikanan) penting apa yang anda atau CBO (komunitas berbasis masyarakat) anda pelajari dari program pelatihan dan peningkatan kapasitas yang di lakukan oleh Proyek SEA USAID? (Catatan: Minta peserta untuk lebih spesifik mengenai jenis pengetahuan, keterampilan, sikap dan kegiatan yang telah mereka pelajari untuk mata pencaharian bukan perikanan.) #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 2.2 Sudahkah anda atau CBO (Komunitas Berbasis Masyarakat) anda menerapkan atau menggunakan pengetahuan dan keterampilan yang telah di pelajari dari program pelatihan dan peningkatan kapasitas yang di laksanakan oleh Proyek SEA USAID? Jika YA, dalam bentuk apa dan apa hasilnya? Jika TIDAK, apa alasannya #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 3. Peningkatan insetif untuk pengelolaan laut (SA3, EQ3) 171 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 3.1Kegiatan mata pencaharian bukan perikanan apa (contoh, pemrosesan dan/atau pemasaran, penyimpanan ikan, wisata laut, layanan sewa kapal, layanan pemandu wisata, dll) yang di perkenalkan dan didukung oleh Proyek SEA USAID yang anda atau CBO (komunitas berbasis masyarakat) anda terapkan di dalam komunitas anda? (Catatan: Minta peserta untuk lebih spesifik menjelaskan kegiatan mana yang berada dalam rantai pengembangan mata pencaharian bukan perikanan.) #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 3.2 Apa peran dari anggota pria dan wanita di dalam kegiatan mata pencaharian bukan perikanan anda? #1: Peran pria: Peran wanita: #2: Peran pria: Peran wanita: #3: Peran pria: Peran wanita: #4: Peran pria: Peran wanita: #5: Peran pria: Peran wanita: #6: Peran pria: Peran wanita: #7: Peran pria: Peran wanita: #8: Peran pria: Peran wanita: #9: Peran pria: Peran wanita: 172 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV #10: Peran pria: Peran wanita: 3.3 Jika anda telah menerima pelatihan pada kegiatan mata pencaharian bukan perikanan yang di laksanakan oleh Proyek SEA USAID, apa yang anda anggap paling relevan dan efektif untuk kegiatan mata pencaharian bukan perikanan anda pada saat ini? (Catatan: Minta peserta untuk mengindentifikasi judul tertentu dari pelatihan tentang pengembangan mata pencaharian bukan perikanan yang di berikan oleh Proyek.) 3.3.1 Tolong jelaskan dengan cara apa pelatihan tersebut relevan dan efektif? #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 3.4 Effek apa yang ada amati dari implementasi kegiatan mata pencaharian bukan perikanan anda pada hal hal berikut ini: (a) Sikap, perilaku atau praktik dari anggota yang terlibat di dalam kegiatan ini? (b) Ancaman terhadap sumber kelautan dan perikanan di dalam komunitas anda (c) Konservasi sumber kelautan dan perikanan di dalam komunitas anda #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 3.5 Faktor atau kondisi apa yang membuat mata pencaharian bukan perikanan lebih atau kurang efektif didalam (a) meningkatkan perilaku masyarakat, (b) mengurangi ancaman terhadap sumber daya kelautan dan perikanan, dan (c) meningkatkan konservasi laut di dalam komunitas anda? (Catatan: Minta peserta untuk lebih spesifik pada jawaban mereka.) 3.5.1 Tolong jelaskan dengan cara apa kegiatan tersebut lebih atau kurang efektif #1: #2: #3: #4: #5: 173 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV #6: #7: #8: #9: #10: 4. Fasilitasi Percakapan mengenai Kebijakan dan Regulasi (SA4, EQ1) 4.1 Kebijakan, regulasi dan/atau alat apa yang diperkenalkan atau didukung oleh Proyek SEA USAID yang relevan dan efektif pada kegiatan mata pencaharian bukan perikanan anda, jika ada? (Catatan: Minta peserta untuk mengidentifikasi kebijakan, peraturan dan/atau alat yang relevan dan efektif untuk kegiatan anda. Dapatkan salinan dari instrument/alat tersebut.) 4.1.1 Tolong jelaskan dengan cara apa kebijakan, peraturan dan/atau alat tersebut relevan dan efektif di dalam kegiatan anda? #1: Kebijakan: Regulasi: Alat: #2: Kebijakan: Regulasi: Alat: #3: Kebijakan: Regulasi: Alat: #4: Kebijakan: Regulasi: Alat: #5: Kebijakan: Regulasi: Alat: #6: Kebijakan: Regulasi: Alat: #7: Kebijakan: Regulasi: Alat: #8: Kebijakan: Regulasi: 174 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Alat: #9: Kebijakan: Regulasi: Alat: #10: Kebijakan: Regulasi: Alat: 2. Meningkatkan pengelolan ekosistem FMA-715 dan MPA 5.1 Metode atau pendekatan specifik apa (Contoh: EAFM/Perikanan yang berkelanjutan, Pembentukan MPA, MSP dan penegakan hukum) untuk meningkatkan pengelolaan ekosistem dalam sumber daya kelautan dan perikanan yang diperkenalkan atau di dukung oleh Proyek SEA USAID yang relevan dan efektif dalam kegiatan mata pencaharian non-fishing (bukan perikanan) anda? (Catatan: Minta peserta untuk mengidentifikasi metode atau pendekatan spesifik yang relevan dan efektif untuk kegiatan mereka. Minta bukti seperti foto atau laporan yang menunjukan relevansi dan efektifitas metode atau pendekatan yang di maksud) 1.5.1Tolong jelaskan dalam bentuk apa metode atau pendekatan spesifik yang relevan dan efektif dalam kegiatan anda? #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 3. Aspek manajemen secara keseluruhan dari kegiatan mata pencaharian bukan perikanan 6.1 Pendekatan apa (contoh: kebijakan, peraturan, dan/atau alat; pelatihan; mata pencaharian; penguasaan perikanan laut yang lebih aman; dan peningkatan kesadaran atau avokasi) yang di perkenalkan oleh Proyek SEA USAID yang effektif pada kegiatan mata pencaharian non-fishing (bukan perikanan) anda? (Catatan: Kembangkan daftar pendekatan yang efektif yang telah di identifikasi oleh peserta) 6.1.1 Tanyakan apakah MASING-MASING pendekatan efektif dengan sendirinya atau bergantung pada satu atau lebih dari pendekatan yang telah di implementasi? (Catatan: Dapatkan bukti Salinan atau foto untuk bukti dokumentasi) #1: #2: #3: #4: #5: #6: #7: #8: #9: 175 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV #10: 6.2 Jika ada, kesenjangan kapasitas apa, yang anda atau CBO (Organisasi Berbasis Masyarakat) anda temui didalam pelaksanaan kegiatan mata pencaharian non￾fishing (bukan perikanan) anda? (Catatan: Minta peserta untuk menjelaskan secara specific kesenjangan kapasitas yang perlu di tingkatkan tersebut) #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 6.3 Saran atau tindakan apa yang harus diambil untuk meningkatkan (atau mensukseskan) pelaksanaan untuk kegiatan mata pencaharian non-fishing (bukan kegiatan perikanan) di komunitas anda? #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 6.4 Saran apa yang dapat anda sampaikan kepada USAID untuk desain proyek berikutnya di dalam membangun kapasitas dan kemampuan Indonesia untuk mengelola perikanan dan sumber daya lautnya secara efektif? #1: #2: #3: #4: #5: #6: 176 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV #7: #8: #9: #10: 5. Penutup/Pertanyaan penutup Sekali lagi, yakinkan para peserta bahwa semua data/informasi yang di peroleh akan tetap rahasia, dan tidak akan digunakan untuk tujuan lain selain apa yang telah dinyatakan. Penutup, meminta izin dari peserta untuk menindak lanjuti komunikasi jika ada hal yang perlu diklarifikasikan atau diperlukannya informasi tambahan untuk memperjelas spesifik topik yang didiskusikan selama FGD. Pastikan untuk mendapatkan nomor kontak mereka. Katakan “Terima Kasih” sebelum anda pergi. Jadwal FGD: Tanggal FGD Sesi: Waktu Mulai: Waktu Selesai: Lokasi FGD: Nama Facilitator: Nama Dokumenter: 177 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV SUSTAINABLE ECOSYSTEMS ADVANCED (SEA) PROJECT Midterm Performances Evaluation (MTPE) FGD Guide Questions FGD Tool 1.b. Local Communities/CBO Involved in More Secure Tenure or Managed Access to Fisheries Resources Focus Groups: Local Community/Community-Based Organization (CBO) members, with two separate gender-based FGD sessions Main Focus: More secure marine tenure or managed access to fisheries resources (EQ4) supported by USAID SEA Project to determine and evaluate to what extent are these tenure rights lead to the application of sustainable fishing practices and reduction of destructive fishing activities by local fishermen; and under what conditions are these tenure rights effective in the selected Project sites. Secondary Focus: Contributions of relevant policies, regulations and/or tools (EQ1); trainings (EQ2); and other Strategic Approaches, such as the creation of demand through awareness raising and advocacy (SA1), and the improvement of ecosystem management of FMA￾715 and MPAs (or LLMAs) (SA2) through Technical Approaches introduced or supported by the Project; and how well or not well are the Project components/Strategic Approaches when “working together” (EQ5). Target Sites: (For confirmation with USAID SEA Project) a) Territorial Use Rights for Fishing (TURF) in Raja Ampat, West Papua; and b) Locally Marine Management Area (LMMA) in Maluku 1. Introduction and Background to the SEA Project MTPE Introduction of Interviewer, Facilitator cum Interpreter, and Documenter Good day! My name is ___________, FGD Facilitator, and my companion is _____________, FGD Documenter, of the Pacific Rim Innovation and Management Exponents, Inc. (PRIMEX). We are part of the MTEP Evaluation Team, commissioned by USAID/Indonesia to conduct an evaluation of USAID SEA Project, which is implementing more secure tenure or managed access to fisheries resources in local communities. Our Team, in particular, is conducting focus group discussion (FGD) with members of your community/CBOs in the selected Project sites (TURF in Raja Ampat, West Papua; and LMMA in Maluku) who are directly involved in these activities. This FGD is important to achieve the MTPE objectives, which include: (i) review the Project midterm performance, particularly in terms of implementation of the more secure tenure or managed access to fisheries resources in your community; (ii) analyze factors and conditions that enhance or diminish the achievements of the target outputs and outcomes of these activities; and (iii) recommend specific actions to further improve implementation effectiveness of these activities. And we would like to seek your support for the completion of this FGD as it may take about one and a half (1.5) hours to two (2) hours. At this early stage, we wish to express our sincerest appreciation to your CBO/s and community for participating in this FGD session. 178 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Protocols on FGD session ✓ It is important that each of the FGD participants is given enough time to express his/her ideas in order to articulate his/her answers. So, ensure that everybody is heard in the FGD session. ✓ Confidentiality: Assure the participants that all the data/information they will provide will be treated with utmost confidentiality, and will be used mainly to undertake an objective assessment of USAID SEA Project’s performance. The names of participants will not, in anyway, be associated with any specific answers/comments provided. All responses will be collated and analyzed, together or in combination with other data and information collected from other FGD participants. ✓ There will be NO right or wrong answer; only differing points of view. The participants need not agree with each other’s replies, but must listen respectfully as others share their views. Encourage the participants to feel free, and be honest and objective in the answers or comments/opinion to be provided. ✓ The FGD session will be recorded; hence, each participant is encouraged to speak one at a time. The participants will be called on a first name basis. ✓ Request mobile phones to be turned off or switch to silent mode. If an urgent call needs to be answered, please ask the participants to do so as quietly as possible and rejoin the FGD as quickly as possible. ✓ Before the FGD begin, ask the participants for any questions, concerns, or clarifications. 2. Basic Information about the FGD Participants Notes: • This table can be accomplished in a separate sheet of paper. • Request for any print materials (brochures, accomplishment reports, pictures, etc. about the community/CBOs to serve as evidence/support documents that will substantiate the answers provided) Name of FGD Participant Position / Designation or Role in CBO/s Name of Organization CBO’s’ Areas of Operation (province/ district/site) CBO/s’ Number of years in operation Year started as partner/ beneficiary of USAID SEA Project Number of Members Male Female 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 179 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 3. Understanding the Role/s of FGD Participants in USAID SEA Project FGD Participant’s Number What is your organization’s main goal or primary interest in the marine and fisheries resources in your community? What were your activities relative to these resources before joining USAID SEA Project? What is your organization’s main interest in joining USAID SEA Project? What activities are you doing/implementing in relation to USAID SEA Project? #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 180 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 4. Main Body of Guide Questions Note: The following Tables can be printed and accomplished in a separate sheet of paper to provide enough space for the answers of each of the FGD participants. 1. Create demand through awareness raising and advocacy (SA1) 1.1 At the start of USAID SEA Project implementation in your community, how did the Project inform you or your CBO/s about its objectives, activities and strategies? (Note: meeting, forum, seminar, communication materials, media, etc.) 1.1.1 How did this information help you or your CBO/s to better understand the Project objectives, activities, and strategies? 1.1.2 How did you learn about the more secure tenure or managed access to fisheries resources in your community from the Project? #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 2. Institutionalize training and capacity building (SA5, EQ2) 2.1 What are important knowledge, skills, attitudes, and practices on more secure tenure or managed access to fisheries resources that you or your CBO/s learned from the training and capacity building programs provided by USAID SEA Project? (Note: Request participants to be more specific on the types of knowledge, skills, attitudes and practices that they have learned for non-fishing livelihood activities.) #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 2.2 Have you or your CBO/s applied or used the knowledge and skills on more secure tenure or managed access to fisheries resources learned from the training and capacity building programs provided by USAID SEA Project? If YES, in what ways and what have been the results? If No, what have been the reasons? #1: #2: #3: #4: #5: 181 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV #6: #7: #8: #9: #10: 3. Increase incentives for marine stewardship (SA3, EQ4) 3.1 What benefits (i.e., large sizes of fish catch, more stable fish catch, keep fishing area more accessible, etc.) do fishermen gain from more secure tenure or managed access to fisheries resources, which are not available or possible in the past (i.e., before the introduction of marine tenure rights in your community by USAID SEA Project)? (Note: Request participants to be more specific on the types of benefits.) #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 3.2 Do you observe changes in behavior of fishermen (i.e., applying sustainable fishing practices, reducing destructive fishing practices, etc.) as a result of these more secure tenure or managed access to fisheries resources? If YES, in what ways and what have been the results? If No, what have been the reasons? 3.2.1 What changes do you observe in the roles of males and females in your CBO, community, family in fishing, if any, as a result of more secure tenure or managed access to fisheries resources? #1: Overall: Role of men: Role of women: #2: Overall: Role of men: Role of women: #3: Overall Role of men: Role of women: #4: Overall Role of men: 182 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Role of women: #5: Overall Role of men: Role of women: #6: Overall: Role of men: Role of women: #7: Overall: Role of men: Role of women: #8: Overall: Role of men: Role of women: #9: Overall: Role of men: Role of women: #10: Overall: Role of men: Role of women: 3.3 With more secure tenure or managed access to fisheries resources, do fishermen apply sustainable fishing practices, and reduce destructive fishing practices? If YES, in what ways, and to what extent do these fishermen apply sustainable fishing practices and reduce destructive fishing practices? If No, what have been the reasons? #1: #2: #3: #4: #5: #6: #7: #8: 183 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV #9: #10: 3.4 Why do you think that more secure tenure or managed access to fisheries resources have resulted in sustainable fisheries practices and a reduction in destructive fishing practices (or have not resulted in improved fishing practices)? 3.4.1 Has secure tenure and managed access been enough to result in improved fisheries practices, or have other things/interventions also been needed to bring about positive changes? (Note: If participants identify other interventions that are also needed to bring about positive changes, ask the next question below.) 3.4.2 Have those other interventions happened? 3.4.3 What additional interventions are needed? (Note: Get copies or pictures as proof of evidence to the effectiveness of secure tenure or managed access.) #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 3.5 Did you receive trainings on the policies, regulations and/or tools for carrying out more secure tenure or managed access to fisheries resources from USAID SEA Project? If Yes, in what ways did these trainings help you or your CBO/s to improve fishing practices to ensure stable fish catches and reduce destructive fishing practices? What have been the results? If No, what have been the reasons? (Note: Request participants to identify the specific title/s of training/s on more secure marine tenure or managed access provided to them by the Project.) #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 3.6 What effects do you observe from getting more secure tenure or managed access to fisheries resources in your community on the following: (a) Practicing sustainable fishing methods; and (b) Reducing destructive fishing practices? #1: On practicing sustainable fishing: On practicing destructive fishing: #2: On practicing sustainable fishing: On practicing destructive fishing: #3: On practicing sustainable fishing: On practicing destructive fishing: 184 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV #4: On practicing sustainable fishing: On practicing destructive fishing: #5: On practicing sustainable fishing: On practicing destructive fishing: #6: On practicing sustainable fishing: On practicing destructive fishing: #7: On practicing sustainable fishing: On practicing destructive fishing: #8: On practicing sustainable fishing: On practicing destructive fishing: #9: On practicing sustainable fishing: On practicing destructive fishing: #10: On practicing sustainable fishing: On practicing destructive fishing: 4. Facilitate Dialogue on Policies and Regulations (SA4, EQ1) 4.1 What policies, regulations and/or tools introduced or supported by USAID SEA Project are relevant and effective to carry out more secure tenure or managed access to fisheries resources in your community, if any? (Note: Request participants to identify the specific policies, regulations and/or tools that are relevant and effective to their activities. Get copies of such policies, regulations and/or tools.) 4.1.1 Please elaborate in what ways are these policies, regulations and/or tools relevant and effective in your activities? #1: Policies: Regulations: Tools: #2: Policies: Regulations: Tools: #3: Policies: Regulations: Tools: #5: Policies: Regulations: Tools: #6: Policies: 185 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Regulations: Tools: #6: Policies: Regulations: Tools: #7: Policies: Regulations: Tools: #8: Policies: Regulations: Tools: #9: Policies: Regulations: Tools: #10: Policies: Regulations: Tools: 5. Improve ecosystems management of FMA-715 and MPAs 5.1What approaches (i.e., EAFM, MPA establishment, MSP and law enforcement) introduced by USAID SEA Project are effective to securing marine tenure or managed access to fisheries resources in your community? (Note: Develop a list of approaches identified by participants as effective.) 5.1.1 Ask whether EACH approach is effective by itself or relies on being implemented with one or more of the other approaches (Note: Get copies of evidences like pictures or reports to demonstrate the relevance and effectiveness of each method or approach, or a combination of methods or approaches.) #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 186 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 1. Overall management aspects of implementing non-fishing livelihood activities 6.1 What capacity gaps, if any, do you or your CBO/s encounter in securing your marine tenure or managed access to fisheries resources in your community? Note: Request participants to identify the specific capacity gaps that need improvement. #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 6.2 What suggestions or courses of action to take to improve (or make successful) the implementation of more secure tenure or managed access to fisheries resources in your community? #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 6.3 What suggestions can you make to USAID for future project design in building Indonesia’s capacity and ability to effectively manage your own fisheries and marine resources? #1: #2: #3: #4: #5: #6: #7: 187 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV #8: #9: #10: 5. Closing/Sign-Off Questions Again, assure the participants that all data/information obtained shall remain confidential, and will not be used for purposes other than what have been stated. Finally, ask permission from participants for possible follow-up communications with them for clarifications or additional inputs to shed more lights on specific topics discussed during the FGD. Ensure to get their complete contact numbers. Say “Thank You” before you leave. Schedule of FGD: Date of the FGD Session: Time Started: Time Ended: Venue of FGD: Name of Facilitator: Name of Documenter: 188 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV SUSTAINABLE ECOSYSTEMS ADVANCED (SEA) PROJECT Midterm Performances Evaluation (MTPE) Panduan Pertanyaan FGD Alat FGD 1.b. Masyarakat Lokal/Kelompok Masyarakat yang Terlibat dalam Wilayah Tangkapan Aman atau Akses Terkelola ke Sumberdaya Perikanan Kelompok-Kelompok Peserta: Masyarakat Lokal/anggota organisasi berbasis masyarakat/FGD dengan dua kelompok gender terpisah. Fokus Utama: Wilayah tangkapan yang lebih aman atau akses terkelola untuk sumberdaya perikanan (EQ4) yang didukung oleh Proyek USAID SEA untuk menentukan dan mengevaluasi pada sisi mana hak-hak pada wilayah penangkapan ini mengarah pada penerapan praktik pengelolaan perikanan berkelanjutan dan pengurangan aktivitas perikanan merusak (destructive) oleh masyarakat lokal dan pada kondisi apa hak-hak wilayah pengelolaan menjadi efektif pada lokasi proyek. Fokus kedua: Kontribusi kebijakan-kebijakan relevan, peraturan dan atau alat-alat (EQ1); pelatihan￾pelatihan, (EQ2); dan pendekatan-pendekatan strategis seperti pengembangan kebutuhan melalui peningkatan kesadaran dan advokasi (SA1), dan perbaikan ekosistem dari WPP￾715 dan MPAs (atau LLMAs) (SA2) berdasarkan pendekatan-pendekatan teknis yang telah diperkenalkan oleh proyek dan seberapa baik atau tidak baik komponen-komponen atau pendekatan-pendekatan strategis ketika “kerja bersama” (EQ5). Lokasi-lokasi Target (untuk dikonfirmasikan dengan USAID SEA Project) a) Territorial Use Rights for Fishing (TURF) di Raja Ampat, Papua Barat; dan b) Locally Marine Management Area (LMMA) atau Wilayah Pengelolaan Laut Berbasis Lokal 1. Perkenalan dan latarbelakang MTPE SEA Project Perkenalan Pewawancara, Facslitator berikut Interpreter, dan Documenter Selamat siang! Nama saya ___________,sebagai fasilitator FGD, dan nama mitra saya _____________, sebagai pendokumentasi FGD, dari Pacific Rim Innovation and Management Exponents, Inc. (PRIMEX). Kami bagian dari Tim Evaluasi MTEP yang ditugaskan USAID/Indonesia untuk melaksanakan evaluasi Proyek SEA USAID, yang mana sedang melaksanakan pengelolaan wilayah tangkapan yang lebih aman atau akses terkelola untuk sumberdaya perikanan di tingkat masyarakat lokal. Tim kami, saat ini sedang melaksanakan Focus Group Discussion (FGD) dengan anggota-anggota kelompok masyarakat/CBOs dalam lokasi terpilih proyek TURF di Raja Ampat, Papua Barat dan LMMA di Maluku yang secara langsung terlibat dalam kegiatan-kegiatan disebutkan di atas. FGD ini penting untuk mencapai tujuan MTPE, yang meliputii: (i) review kinerja paruh tengah Proyek, utamanya dalam hal implementasi dari wilayah tangkapan yang lebih aman atau akses terkelola untuk sumber daya perikanan dalam masyarakat; (ii) menganalisis faktor-faktor atau kondisi yang dapat meningkatkan atau menghilangkan capaian-capaian dari sasaran luaran atau hasil-hasil kegiatan; dan (iii) merekomendasikan langkah-langkah spesifik untuk memperbaiki efektivitas implementasi dari masing-masing kegiatan. Dan, kami berharap dapat 189 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV memperoleh dukungan untuk menyelesaikan FGD ini yang akan membutuhkan waktu kurang lebih 1,5 jam atau hingga 2 jam. Pada bagian awal ini, kami menyampaikan apresiasi mendalam kepada organsiasi￾organisasi dan perwakilan masyaakat yang telah ikut serta dalam sesi FGD ini. Panduan sesi FGD ✓ Adalah penting untuk setiap peserta FGD diberikan waktu cukup mengekspresikan ide-idenya dengan maksud menyampaikan jawaban-jawabannya. Jadi pastikan bahwa semua yang terlibat dalam sesi FGD ini didengarkan pendapatnya. ✓ Kerahasiaan: Yakinkan peserta bahwa semua data atau informasi yang mereka siapkan akan dijaga kerahasiaannya dan hanya akan digunakan untuk kepentingan penjajakan kinerja Proyek USAID SEA. Nama-nama peserta, tidak, dalam hal apapun tidak akan dikaitkan dengan jawaban-jawaban spesifik yang disampaikan. Semua tanggapan akan digabungkan dan dianalisis, bersama atau dikombinasikan dengan data dan informasi lain dari peserta FGD lainnya. ✓ Tidak akan ada jawaban benar atau salah; hanya perbedaan cara pandang. Peserta tidak harus setuju jawaban satu sama lain, tetapi harus mendengarkan penuh perhatian saat yang lain menyampaikan pendapat. Dorong para peserta untuk bebas berpendapat, jujur dan obyekif dalam menjawab atau memberikan tanggapan yang disiapkan. ✓ Sesi FGD akan direkam, dan dengan demikian setiap peserta diminta untuk bicara satu demi satu dan akan dipanggil dengan namanya. ✓ Meminta agar telepon genggam dinonantifkan atau mode diam. Jika ada panggilan penting perlu dijawab dan minta izin ke peserta yang lain. Tanya peserta lain untuk melakukan hal sama dan kembali ke sesi FGD sesegara mungkin kalau urusan selesai. ✓ Sebelum FGD dimulai tanya peserta jika ada pertanyaan, perhatian, atau karifikasi. 2. Informasi dasar tentang peserta-peserta FGD. Catatan-catatan:: • Tabel ini dapat diselesaikan dalam lembar kertas terpisah. • Minta untuk setiap lembaran tercetak (brosur, laporan-laporan penyelesaian, gambar, dll tentang masyarakat atau CBOs untuk dijaikan sebagai bukti/dokumen pendukung yang akan dikaitkan dengan jawaban-jawaban yang diberikan. Nama Peserta Posisi/Tanggung Jawab atau Peran dalam Kelompok Nama organisasi Area Operasi Kelompok CBO (provinsi/ kabupaten/lokasi) Jumlah Tahun Keberadaan/O perasi Kelompok CBO Tahun bermula kerjasama dengan atau penetima manfaat dari Proyek USAID SEA Jumlah anggota Pria Perempuan 1. 2. 3. 4. 190 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Nama Peserta Posisi/Tanggung Jawab atau Peran dalam Kelompok Nama organisasi Area Operasi Kelompok CBO (provinsi/ kabupaten/lokasi) Jumlah Tahun Keberadaan/O perasi Kelompok CBO Tahun bermula kerjasama dengan atau penetima manfaat dari Proyek USAID SEA Jumlah anggota Pria Perempuan 5. 6. 7. 8. 9. 10. 3. Memahami peran dari peserta FGD dalam Proyek SEA USAID Nomor Peseta FGD Apa tujuan utama organisasi atau minat/perhatian utama dalam sumber daya kelautan dan perikanan di tengah masyarakat? Apa-apa saja kegiatan kelompok yang berkaitan dengan sumber daya itu sebelum bergabung dengan Proyek SEA USAID? Apa minat utama organisasi anda dalam bergabung dengan Proyek USAID SEA? Apa-apa saja kegiatan yang anda jalankan berkaitan Proyek SEA USAID? #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 191 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV 4. Pokok bahasan utama dari pertanyaan-pertanyaan panduan Catatan: Tabel berikut dapat dicetak dan diselesaikan dalam lembaran terpisah untuk menyiapkan ruang cukup untuk setiap jawaban dari Peserta FGD. 1. Menciptakan peluang melalui peningkatan kesadaran dan advokasi (SA1) 1.1 Pada awal implementasi Proyek USAID SEA di tengah masyatakat anda, bagaimana Proyek menginformasikan anda atau organisasi anda tentang tujuan, kegiatan dan strategi-strateginya? (Catatan: pertemuan, forum, seminar, materi komunikasi, media, dlll.) 1.1.1 Bagaimana infomasi ini membantu anda atau organisasi anda untuk mengerti lebih mudah tujuan proyek, kegiatan-kegiatan dan strategi-straeginya? 1.1.2 Bagaimana anda belajar tentang wilayah tangkapan yang lebih aman atau akses terkelola ke sumber daya perikanan in tengah masyarakat dari proyek ini? #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 2. Melembagakan pelatihan dan pengembangan kapasitas (SA5, EQ2) 2.1 Apa-apa saja pengetahuan penting, keterampilan, sikap dan praktik dalam wilayah tangkapan yang lebih aman atau akses terkelola terhadap sumber daya perikanan yang memungkinkan anda atau organisasi telah belajar dari pelatihan dan pengembangan kapasitas yang disiapkan oleh Proyek USAID SEA? (Catatan: Minta peserta untuk lebih spesifik, pada jenis￾jenis pengetahuan, keterampilan, sikap dan praktik-praktik yang mereka telah pelajari berkaitan kegiatan non￾perikanan.) #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 2.2 Apakah anda atau organisasi anda telah mengaplikasikan pengetahuan dan keterampilan berkaitan wilalayah tangkapan atau akses terkelola sumber daya perikanan yang dipelajari dari training dan program pengembangan kapasitas yang disiapkan oleh Proyek USAID SEA? Jika Ya, dalam bentuk apa dan apa saja hasilnya? Jika No, apa yang jadi alasannya? #1: #2: #3: #4: 192 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV #5: #6: #7: #8: #9: #10: 3. Meningkatkan insentif untuk ‘pelindung’ laut (SA3, EQ4) 3.1 Apa saja menfaat (contohnya., ukuran besar ikan tangkapan, tangkapan ikan yang stabil, tetap menangkap ikan di area yang mudah di akses, dll.) Apakah nelayan￾nelayan memperoleh keuntungan/manfaat yang lebih dari wilayah tangkapan yang aman atau akses terkelola ke sumber daya perikanan, yang mana tidak tersedia atau tidak memungkinakn di masa lalu (misalnya, sebelum pengenalan hak-hak pengelolaan wilayah tangkapan dalam proyek USAID SEA)? (Catatan: minta para peserta untuk lebih spesifik dalam menyebutkan jenis-jenis manfaat) #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 3.2 Apakah anda mengamati perubahan-perubahan perlaku nelayan (contohnya, dalam mengaplikasikan praktik perikanan, mengurangi kegiatan perikanan yang merusak) sebagai hasil dari wilayah tangkapan yang lebih aman atau akses tekelola ke sumberdaya perikanan? Jika YA, dalam bentuk apa dan apa yang telah menjadi hasilnya? Jika Tidak, apa yang menjadi alasan? 3.2.1 Apa perubahan yang anda amati terkait peran perempuan dan pria dalam organisasi, masyarakat atau keluarga dalam usaha perikanan, Jika ada, sebagai hasil dari wilayah tangkapan yang lebih aman atau akses terkelola ke sumberdaya perikanan? #1: Keseluruhan: Peran pria: Peran perempuan: #2: Keseluruhan: Peran pria: Peran perempuan: #3: Keseluruhan: Peran pria: Peran perempuan: #4: Keseluruhan: 193 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Peran pria: Peran perempuan: #5: Keseluruhan: Peran pria: Peran perempuan: #6: Keseluruhan: Peran pria: Peran perempuan: #7: Keseluruhan: Peran pria: Peran perempuan: #8: Keseluruhan: Peran pria: Peran perempuan: #9: Keseluruhan: Peran pria: Peran perempuan: #10: Keseluruhan: Peran pria: Peran perempuan: 3.3 Dengan wilayah tangkapan yang lebih aman atau akses terkelola ke sumber daya perikanan, apakah para nelayan menerapkan praktik-praktik perikanan berkelanjutan dan mengurangi praktik perikanan merusak? Jika YA, dalam bentuk apa, dan pada hal apa para nelayan mengaplikasikan perikanan berkelanjutan dan mengurangi praktik perikanan merusak? Jika Tidak, apa yang menjadi alasannya? #1: #2: #3: #4: #5: #6: #7: 194 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV #8: #9: #10: 3.4 Mengapa anda berpikir bahwa wilayah tangkapan yang lebih aman atau akses terkelola ke sumberdaya perikanan telah menghasilkan dalam praktik perikanan berkelanjutan dan mengurangi praktik perikanan merusak (atau belum menghasilkan dalam perbaikan praktik-praktik perikanan)? 3.4.1 Sudahkah wilayah tangkapan atau akses terkelola cukup menghasilkan dalam memperbaiki praktik perikanan atau sudah hal-hal lain atau intervensi lain yang juga dibutuhkan membawa perubahan positif? (Catatan: Jika para peserta mengidentifikasi ada intevensi lain yang juga dibutuhkan dalam mengarah ke perubahan positif minta mereka untuk melihat pertanyaan berikut) 3.4.2 Apakah telah terjadi bentuk intervensi lain itu? 3.4.3 Apa intervensi lain itu yang dibutuhkan? (Note: Ambil kopian atau gambar bukti dati efektivitas wilayah tangkapan atau akses terkelola itu) #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 3.5 Apakah anda menerima pelaihan terkait kebijakan, regulasi dan atau alat-alat/perangkat untuk melaksanakan wilayah tangkapan aman dan akses terkelola sumber daya perikanan dari USAID SEA Project? Jika ya, dalam bentuk apa pelatihan-pelatihan ini membantu anda atau organisasi anda memperbaiki praktik perikanan untuk memastikan stabilnya ikan-ikan tangkapan dan mengurangi praktik-praktik perikanan merusak? Apa yang yang menjadi hasilnya? Jika Tidak, apa yang menjadi alasannya? (Catatan, minta peserta untuk mengidentifikasi judul pelatihan-pelatihan terkait wilayah tangkapan yang lebih aman dan akses terkelola yang diberikan ke mereka oleh Proyek) #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 3.6 Apa-apa saja akibat yang anda amati dari wilayah tangkapan yang lebih aman atau akses terkelola ke sumberdaya perikanan dalam masyaakat anda dalam hal: (a) Praktik penerapan metode perikanan berkelanjutan dan (b) Pengurangan praktik-praktik perikanan merusak? #1: Terhadap praktik perikanan berkelanjutan Terhadap praktik perikanan merusak; #2: Terhadap praktik perikanan berkelanjutan Terhadap praktik perikanan merusak; 195 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV #3: Terhadap praktik perikanan berkelanjutan Terhadap praktik perikanan merusak; #4: Terhadap praktik perikanan berkelanjutan Terhadap praktik perikanan merusak; #5: Terhadap praktik perikanan berkelanjutan Terhadap praktik perikanan merusak; #6: Terhadap praktik perikanan berkelanjutan Terhadap praktik perikanan merusak; #7: Terhadap praktik perikanan berkelanjutan Terhadap praktik perikanan merusak; #8: Terhadap praktik perikanan berkelanjutan Terhadap praktik perikanan merusak; #9: Terhadap praktik perikanan berkelanjutan Terhadap praktik perikanan merusak; # Terhadap praktik perikanan berkelanjutan Terhadap praktik perikanan merusak; 4. Memfasilitasi dialog terkait kebijakan-kebijakan dan regulasi (SA4, EQ1) 4.1 Kebijakan-kebijakan apa, regulasi dan/atau alat/pedoman yang diperkenalkan atau didukung oleh USAID SEA Project yang relevan dan efektif melaksanakan lokasi tangkapan yang aman atau akses terkelola dalam masyarakat anda, jika ada? (Catatan: minta peserta mengidentifikasi kebijakan spesifik, regulasi, dan atau alat-alat yang relevan dan efektif untuk kegiatan mereka. Dapatkan bukti, kopian, atas kebijakan atau regulasi yang dimaksud) 4.1.1 Jabarkan dalam bentuk apa kebijakan-kebijakan ini, atau regulasi, dan atau alat-alat yang relevan dengan kegiatan anda? #1: Kebijakan: Regulasi: Alat/pedoman: #2: Kebijakan: Regulasi: Alat/pedoman: #3: Kebijakan: Regulasi: Alat/pedoman: #5: Kebijakan: Regulasi: 196 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Alat/pedoman: #6: Kebijakan: Regulasi: Alat/pedoman: #6: Kebijakan: Regulasi: Alat/pedoman: #7: Kebijakan: Regulasi: Alat/pedoman: #8: Kebijakan: Regulasi: Alat/pedoman: #9: Kebijakan: Regulasi: Alat/pedoman: #10: Kebijakan: Regulasi: Alat/pedoman: 5. Memperbaiki pengelolaan ekosistem dari WPP 715 serta MPAs 5.1Pendekatan-pendekatan apa (misalnya, EAFM, pendirian MPA, MSP dan penegakan hukum) yang telah diperkenalkan oleh Proyek USAID SEA yang efektif dalam mengamankan wilayah tangkapan atau akses terkelola ke sumberdaya perikanan fisheries di tengah masyarakat anda? (Catatan: Kembangkan daftar dari pendekatan-pendekatan yang diidentifikasi dari para peserta yang dianggap efektif) 5.1.1 Tanya, manakala SETIAP pendekatan efektif dengan sendirinya atau terhubung dengan satu sama lain atau lebih dari pendekatan-pendekatan lain ini (Catatan: Dapatkan kopian sebagai bukti atau laporan yang menunjukkan relevansi dari efektivitas atau kombinasi dari metode atau pendekatan-pendeatan itu) #1: #2: #3: #4: #5: #6: #7: #8: 197 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV #9: #10: 2. Aspek-aspek manajemen keseluruhan dari pelaksanaan kegiatan-kegiatan mata pencaharian non-perikanan 6.1 Apa gap kapasitas, jika ada, yang anda atau organisasi anda temukan dalam upaya mengamankan wilayah tangkapan atau akses terkelola ke sumberdaya perikanan dalam masyarakat anda? Catatan: Minta peserta mengidentifkasi gap kapasitas spesifik yang butuh perbaikan #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 6.2 Saran-saran apa atau tindakan-tindakan untuk memperbaiki (atau membuat lebih sukses) dalam implementasi wilayah tangkapn yang lebih aman atau akses terkelola ke sumberdaya perikanan dalam masyarakat anda? #1: #2: #3: #4: #5: #6: #7: #8: #9: #10: 6.3 Saran-saran apa yang yang dapat anda berikan ke USAID untuk desain proyek masa depan dalam membangun kapasitas Indonesia dan kemampuannya untuk mengelola sumber daya kelautan dan perikanan secara efektif? #1: #2: #3: #4: 198 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV #5: #6: #7: #8: #9: #10: 5. Penutupan/Pengakhiran Pertanyaan Lagi, pastikan ke peserta bahawa semua data dan informasi yang diperoleh akan dijaga kerahasiaannya dan tidak akan digunakan untuk tujuan yang tidak sesuai dengan apa yang telah disebutkan sebelumnya. Akhirnya, minta perkenan ke peserta dengan memberi kesempatan atau peluang informasi atau komunikasi tambahan untuk klarifikasi atau input tambahan berikutnya untuk kejelasan pada topik-topik spesifik selama FGD. Pastikan untuk memperoleh alamat kontak mereka. Sampaikan ‘terima kasih’ sebelum anda pamit. Skedul FGD: Tanggal sesi FGD: Waktu mulai: Waktu selesai: Lokasi FGD: Nama Fasilitator: Nama Pendokumentasi: 199 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV ANNEX J. LIST OF FGD PARTICIPANTS Female Participants Name of FGD Participant Position / Designation or Role in CBO/s Name of Organization CBO’s’ Areas of Operation (Province/District/Site) 1 Popi Laka Village Staff Village Government Maluku/Taniwel/Pasinalo 2 Min Latupatula Community PKK Maluku/Taniwel/Pasinalo 3 Lina Mustamu Community PKK Maluku/Taniwel/Pasinalo 4 Ola Warahuwena Community PKK Maluku/Taniwel/Pasinalo 5 Ruth Izach PKK PKK Maluku/Taniwel/Pasinalo 6 Mada Warahuwena Community PKK Maluku/Taniwel/Pasinalo 7 Ilon Latue Community PKK Maluku/Taniwel/Pasinalo 8 Tin Maitale Community PKK Maluku/Taniwel/Pasinalo 9 Oca Seluholo Community PKK Maluku/Taniwel/Pasinalo 10 Misye Elake Community PKK Maluku/Taniwel/Pasinalo 11 Dejde Watimuri Community PKK Maluku/Taniwel/Pasinalo 12 Ati Latupasinalo Community PKK Maluku/Taniwel/Pasinalo 13 Yubelina Adadikam Chairwoman Kelompok PKK Waisai – Raja Ampat 14 Dorthea Dimara Member Kelompok PKK Waisai – Raja Ampat 15 Ester Rumbiak Member Kelompok PKK Waisai – Raja Ampat 16 Magdalena Rejaum Member Kelompok PKK Waisai – Raja Ampat 17 Monika Mambraku Member Kelompok PKK Waisai – Raja Ampat 18 Yanti Wanma Member Kelompok PKK Waisai – Raja Ampat 19 Yustina Rejauw Member Kelompok PKK Waisai – Raja Ampat 20 Nahra Kacoa Member POKDARWIS Galo-Galo 21 Dian Kodo Member POKDARWIS Galo-Galo 22 Rusami Ngongano Member POKDARWIS Galo-Galo 23 Endang Ngongano Member POKDARWIS Galo-Galo 24 Hajdria Moloku Member POKDARWIS Galo-Galo 25 Nurnida Tapitapi Member POKDARWIS Galo-Galo 26 Nujula H. Sudin Member POKDARWIS Galo-Galo 27 Nurlela Kacoa Member POKDARWIS Galo-Galo 28 Jamalia Kacoa Member POKDARWIS Galo-Galo 29 Surianti Lopu Member POKDARWIS Galo-Galo 200 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Male Participants Name of FGD Participant Position / Designation or Role in CBO/s Name of Organization CBO’s’ Areas of Operation (Province/District/Site) 1 N. Narupuye Secretary of Village Government Village Government Maluku/Taniwel/Pasinalo 2 Joni Tolau Member of CBO CBO - Conservation Maluku/Taniwel/Pasinalo 3 D. Latupasinayabo Leader of Conservation Conservation Maluku/Taniwel/Pasinalo 4 R Pasinau Village Figure￾Traditional Leader Village Government and Conservation Maluku/Taniwel/Pasinalo 5 Dominggus Maitale Leader of BPD Village Government Maluku/Taniwel/Pasinalo 6 Izak Laka Member of Society Conservation Maluku/Taniwel/Pasinalo 7 M. Warahuwena BPD Village Government Maluku/Taniwel/Pasinalo 8 Nalo Seluholo Representative of Youth Youth Organization Maluku/Taniwel/Pasinalo 9 Rudolf Mustamu BPD Village Government Maluku/Taniwel/Pasinalo 10 Derek Latupasinolo BPD Village Government Maluku/Taniwel/Pasinalo 11 Frans Maitale Religion Figure Religious Community Maluku/Taniwel/Pasinalo 12 Aser Seriholo community Conservation Maluku/Taniwel/Pasinalo 13 Bram Sehulo community Conservation Maluku/Taniwel/Pasinalo 14 A. Latupasinalu community Conservation Maluku/Taniwel/Pasinalo 15 Marthen Burdam Member Fisherman groups Yensnser Waisai – Raja Ampat 16 Marhen Mambrasar Member Fisherman groups Yensnser Waisai – Raja Ampat 17 Didimus Sonoi Member Fisherman groups Yensnser Waisai – Raja Ampat 18 Keliopas Burdam Member Fisherman groups Yensnser Waisai – Raja Ampat 19 Orgenes Burdam Member Fisherman groups Yensnser Waisai – Raja Ampat 20 Paulus Mambraku Customary Leader Fisherman groups Yensnser Waisai – Raja Ampat 21 Robert Andarek Member Fisherman groups Yensnser Waisai – Raja Ampat 22 Yefta Mirino Member Fisherman groups Yensnser Waisai – Raja Ampat 23 Abd Anu Lohor Member POKDARWIS Galo-Galo 24 Iksan Kompami Secretary POKDARWIS Galo-Galo 25 Mahmud Laite Member POKDARWIS Galo-Galo 26 Musnik Ngongano Member POKDARWIS Galo-Galo 27 Muris Kudo Member POKDARWIS Galo-Galo 28 Bahrudin Saratu Member POKDARWIS Galo-Galo 29 Sahril Kacoa Member POKDARWIS Galo-Galo 30 Rasib Adam Member POKDARWIS Galo-Galo 201 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV ANNEX K. FGD PHOTO DOCUMENTATION FGD at Pasinalo, Taniwel 202 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV FGD at Waisai, Raja Ampat, West Papua 203 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV FGD in Galo-galo 204 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV ANNEX L. SUMMARY OF DATA TRIANGULATION MATRIX USAID Sustainable Ecosystem Advanced (SEA) Project Mid-Term Performance Evaluation (MTPE) TRIANGULATION MATRIX Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments 1. Under what conditions are the regulations and/or tools being introduced by USAID SEA to relevant authorities leading to the adoption of these policies, regulations, and tools? • Through TETRA TECH and the Technical Assistants assigned in each province. The whole time Tetra Tech was referring to the Theory of Change (TOC) • In the beginning, there was stakeholders workshop and also situation analysis of the project also some specific discussion to collect inputs from the participants. Through this, threats were identified during the life cycle of the project. • All respondents are one in saying that the implementation of EA project is through NGO partners contracted to cover a particular thematic and geographic area. • All respondents are in agreement that their approach is to create demand through awareness raising and advocacy for a better fisheries and marine management, and biodiversity conservation. These approach and strategies are mostly implemented through their NGO partners contracted to • Socialization (discussions, meetings, education, trainings, workshops, mouth-to-mouth information sharing of purpose, guidelines and activities) • Strong support from provincial governments, particularly in making decree/policy and guidelines • Strong partnership between traditional and religious leaders, government, and villages • Identification and training of champions to influence and share with • Socialization, standards and processes (expert inputs, consultations, trainings, coordination, coaching and mentoring) • Hindering factors: Time required in the process to produce the adoption instrument; availability of funding support • Leadership commitment to change, including provision of funding support • Community commitment to participate in implementation • Cooperation between Provincial Governments, MAFF and SEA Project, and presence of local NGO partners • Establishment and training of POKMASWAS (8-15 members) to carry out marine surveillance • Identification and training of champions to inspire other local people to support village regulations • Male and female participants in FGD sessions from Galo-Galo, Morotai, North Maluku, who are involved in non￾fishing livelihood activities cited SEA Project’s socialization with community, provision of training, and introduction of non-fishing livelihood activities are important conditions to adoption of and compliance with village policies and regulations. Males also added the organization of community￾based POKDARWIS as another key • * Factors enabling/hindering project-introduced policies/regulations/ tools: > Desk review validation of MEL indicators and targets indicate the project has largely exceeded targets, and well on the way to achieving LOP targets where such has not been achieved (compare and contrast, for example, the figures reported in documents [5] and [9], as supported by activity reports in documents [4], [7] and [8], versus targets reported in documents [1] to [3], inclusive, and dicument [6] which outlined ‘catch-up’ elements for the rest of the project • 205 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments • Tetra Tech provided the annual work plan that identifies key issues and work with MMAF. The project at the start also gathered lessons from IMACS (USAID Project) before making decisions to cover the 3 provinces: Maluku. North Maluku, and Papua Barat. The decision to cover FMA 715 was because of the threat to marine resource and fishing activities • There was a different team, in the beginning, MMAF proposes Natuna and Java Sea but they don’t have similar interest. I mean cover a thematic and geographic area. • SEA Project applied two parallel strategies: behavior change and communication. Thus behavior change communication start with Project SEA messages aiming for awareness raising and then driving into action. These actions include the establishment of the champion program that cover stock assessment, log book, life cycle of fish, etc. aiming to have a better sustainable fisheries management. Another example of behavior change includes “establishment of “no take zone in villages for MPAs, the law and regulation of others the information about marine conservation and sustainable fishing practices • Organization and training of non￾fishing livelihood groups of women (PKK) • Sharing of success stories from other villages or other partner institutions (local governments) • Training and guidance of conservation groups (POKMASWAS or law enforcement/sur veillance group, and POKDARWIS or tourism group) to implement policies and regulations, and to include this in village plans Integration of customary laws (“Adat”) and village traditions (“Sasi” as sacred • Mapping/zoning of marine area, and training of villages on this aspect • Socialization introduced by SEA Project • Hindering factor: Reassignment of personnel trained by SEA Project to other posts facilitating condition. • In fishing villages in Pasinalo, Taniwel in Maluku; and Waisai, Raja Ampat in West Papua, which are involved in more secure tenure or managed access to fisheries resources, male and female participants in FGD sessions identified SEA Project’s socialization; coordination with village government, traditional leader and community; and organization and training of conservation group for marine surveillance as the important conditions for adopting policies and regulations at village level. Males also recognized the importance of the integration of lifespan). Enabling factors cited contributing to success include the good technical inputs, standards and processes (from socialization to coaching and mentoring) used by the project. Moreover, consistency of project approaches with existing or enhanced policies/guidelines has been noted as partly contributing to success (see for example document [10] in the case of the MPA work). Progress is evidenced by achievements in SA 1.1b, c and e (progress at 124% - 906% of targets), with progress expecting to reach targets in SA1.1a and d (59% - 92% of target) by the end of the project lifespan (see document [6]). Similar findings and conclusions were 206 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments understanding the issues and objective and as you know that we focus on Maluku, North Maluku and Papua Barat. So, creating demand may be quite delay and related to strategy of their communication strategy. • For adoption of policy, they identify and train champions that communicate in the sites through socialization and sharing what they learned to the community. • The project should focus on the position of BRSDM (Research and human resources, part of MMAF), how we link with MMAF public Marine Spatial Planning (MSP), and formulation and adoption of marine policies and regulations. • Behavior change is more focus on the community rather than the government partners. • Behavior change includes “No take zone for establishment of MPAs, the law and regulation of Marine Spatial Planning (MSP), and the regulations that operates for the community based surveillance. • On the communication, SEA project is coming up with one platform online and another is off￾line-paper based materials. The online, website￾social media is very effective for the establishment grounds) in fisheries resources management and marine conservation customary tenure approach to fisheries management. kargely reached for the other performance indicators and progress status. The key challenge pertains to adoption of legal instruments to be able to proceed to implementation and achievement of impacts. This has been achieved in project work in PSM, MSP and fishing units registration, but requires substantive attention during the rest of the project lifespan (particularly in efforts in MPAs, fisheries management and law enforcement) to attain targets (e.g., in SA 4.1 and 2.1b). 207 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments relation, with their extension officer. • Through planning discussions, to encourage them to perform their roles. • The effectiveness of SEA Project is how this SEA team support local government through consultations • Also by establishing tasks force as part of consultation and a key step to the smooth process. • I think, MPA scheme is a bit delay because of the need of acknowledgme nt from the government. Also, the need to consider why we have to have physical indicator and of the Champions program. • SEA Project involved the national and provincial government in decision making relative to the running of SEA project and the direction that it should take. • At the start, there was a senior level discussion. • The project is very successful in helping the provincial government because they suddenly needed assistance as they are now mandated to manage their marine/fisheries resources and conservation. For more than 20 years, they do not have the authority to plan and manage their marine resources and then with the new government 208 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments the need to have a formal endorsement from provincial and central government. This process is not easy to achieve. • One of the weaknesses for the establishment of MPA is the absence of monitoring in the site. • MPA is a multi￾purpose and need to clarify fishing zone, there are three zones. • There is problem with data requirements and the lack of capacity in this regard is a problem. • It depends on how to apply communication or the level of communication strategy for the need to develop skills/ mandate requiring them to plan and provide budgetary requirements. • They needed badly the help of the SEA project which is one of the key of the Project’s success. The project plan with the national and local government, build their capacity relative to planning and marine management, and working with them in formulating their provincial and local regulations and policies, and in capacity development – training of trainors, spatial planning for MSP, and establishment of MPAs. • The project provided the common understanding of EAFM, MPA, MSP, by starting 209 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments capacity on the aspect of needed data. • In relation to raising awareness, the national government roles in considering like RZWP3K process. They are involved also in the process. • Marine spatial planning also in the good momentum. Just like MSP and permit or regulation for fisheries has become the focus for all. • Based on SEA Project report, there are 500 champions. They just need to maintain the communication process. Community or champions are part of what we call voluntary and maybe we have the initiative of discussion and educate/train the national and local government partners by combining theory and practice. • The mechanism of SEA project, that is, working with partners is very effective because our partners are very good, committed, and work hard to achieve their targets. They work independently in their areas of coverage. • Awareness raising and adoption of policy was accomplished through communication team, awareness raising poster both formal and informal strategies (e.g. through local champions, socialization through field 210 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments to be aware about the meaning of salary to them (what is the reward that they get for volunteering as champions). SEA support by discussing about criteria of champions. SEA also suggests for the champions to become extension officer. Good facilitation with MDPI in order to have Fair Trade for tuna and stakeholders involved fish traders or exporters/ suppliers which in turn relate to small fishermen. coordinators and through passing of information by mouth-to-mouth discussion, etc.), • The five activities of the communication team include: 1.Data collection: 48 landing sites (3 Province); standard methodology, catch composition, how many catch? 2.Genetic study collects samples of species 3.Vessel registration goal robust data base of vessel promoting and advocating with the provincial government and also involve 4 Ministries (transport, MMAF, PTSP (investment and home affair) but the main foundation is in province. 211 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments Barcode by scan to registered vessel Integrated database: Simkada (data base to identify vessel) 10-30 gross ton based on UU23 but at North Maluku below 5 gross ton delegated to district at Maluku have a branch office to take care of below 5 gross ton vessel. 4.TNC database system with DG they don’t have basic information. 5.Partnership with specific agency 6.Fishing log book, harvest strategy (reef fish (North Maluku); Flying fish, anchovy, prawn fish (West Papua). Work together with SDI advocacy by MMAF, legal bureau, 212 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments academic paper, identified management unit need to share with the province, management plan small scale fishing boat, data collection with BRPL The implementation of EAFM, MPA, and MSP are quite successful in the three provinces. For example, the MSP is already completed mainly through the assistance of SEA project to the three provinces. The push for each province to complete their MSP soonest is because of the national policy that put a deadline for each province completion of their MSP. This requirement motivates the 213 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments provinces to seek assistance from SEA Project. After the completion of MSP, then the paper work for MPA was started. Many of the MPAs in the region and the three provinces were already existing in reality but the legalities and paper requirements have just started. The local government buy-in/accept SEA project. One indicator that the province and villages support the project is when they put up funding for their activities. For example for the three provinces, the provincial government of Maluku allocated more 214 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments than US $300,000 million for marine conservation, MSP, establishment of MPA. Law enforcement and marine resource protection and sustainability, the same amount were also allocated by West Papua and North Maluku provinces for the similar activities. Another indicator of support and buy-in are the support of the village people through the champions and volunteers in surveillance group such as volunteers in POKMASWAS. The high participation in non-fishing livelihood activities, 215 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments weaving, soap making and handicrafts, homestay, in local tourism also indicate support and acceptance of SEA project through its partner that implement the project. The factors that hinder or limit the progression of policies/ regulations to reach the final stage of authorization include: 1. The process and the bureaucracy at the national level is so slow because of the many stages and steps the process need to accomplished before it reaches the national level review. 216 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments 2. At the start, there is no common understanding of EAFM, MSP, MPA, marine resource management, and conservation and what is the task at hand. This required more and long discussions, training, SEA projects messages and awareness raising and more closer partnerships in the ground or developing and training of more champions. 3. Planning, budgeting, and funding for the formulation and adoption of these policies at the village level need support and technical assistance 217 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments from SEA Project and NGO partners. Recommendations to reduce the weakness: 1.Initiative driven by project not by government how to encourage them is not easy such as increase the vessel to be registered (work plan together with province) 2.Work as much as possible with partner through awareness raising and , capacity building 3.Involved the private sector 4.Budget problem The stake holder still do not follow the management plan 218 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments 2. Under what conditions do people trained to manage marine and fisheries resources and/or advocate for sustainable management of these resources, as a result of USAID SEA interventions, apply their skills in marine and fisheries resources management or advocacy? • Capacity is one the aspect that need to be strengthened in the SEA project. From this two component, MPA and MSP there was advance training for stakeholders. MPA is a new thing so in term of training this one is more activity to strengthen stakeholders. In the beginning, the project also develop fisheries model. Fisheries issues is very complex and need analysis, strategy to develop a plan, etc. At the ground, fisheries also need transparency. • The respondents reported that they are not implementing the project. It is through the partner - that trained and informed their communities and see to it that these skills learned are applied in marine conservation, reduced threats, and destructive fishing practices. • Through our partner NGOs and the way they capacitate and involve their communities in the work. The partners’ ability to work with their area and community. There should be continued guidance and practicum demonstration. If a new model and new concept, new process, the more that • All key informants provided evidence of the success of the training they conducted by showing the ways in which participants, partners, and villagers translate the training into reality. The following conditions contribute to application of knowledge/ skills gained from SEA Project interventions: (i) sequential training, starting from MPA Managers and trainers, then fish suppliers, champions, traditional/ customary leaders, religious leaders, government leaders, and then village or community leaders who, in turn, train • Most key informants (6 of 9) considered SEA Project as largely successful in training and capacity-building activities for both thematic and technical components within their respective portfolios. The conditions for application of knowledge/skills gained from these activities include: (i) good screening/ selection of trainees, (ii) institutional support (e.g. organizational commitment to retain trainees in positions relevant to their training), and (iii) effective process of SEA Project support (e.g. coaching and mentoring beyond formal training) • Hindering factors: Personal • All key informants provided evidence for the application of knowledge/skills gained by provincial government personnel from training and capacity-building activities, as a result of SEA Project interventions. Identified key conditions for application of such knowledge/skills include: (i) continued socialization with communities conducted by SEA Project and NGO partners; (ii) secure job tenure of trained personnel to carry out the System for Vessel Registration, or Data Input for Boat/Kapal; (iii) continued coordination and communication • In three selected Project sites, majority of male and female participants in FGD sessions attended training and capacity-building activities related to marine conservation and sustainable fishing practices, except the females in Waisai, Raja Ampat in West Papua as explained in the next column. • Both male and female participants cited the observed or perceived improvements on fisheries and marine resources, particularly in “Sasi” area declared as no take zone (i.e., recovery of corals, re/appearance of rare fish • * Capacity-building and Subsequent deployment > The desk review largely validated the successful and effective project progress in capacity-building and subsequent deployment of capacitated personnel (compare for example the targets given in documents [1]- [3] inclusive versus reported project progress in documents [5] and [9], as supported by activity reports in documents [4], [7] and [8]). This is evidenced by the progress made in SA5.1 (105%); although substantive work remains in law enforcement training (SA5.3 - 59%) requiring adoption of legal instruments as mentioned above. Enabling conditions point to effective • 219 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments There is a training for POKMASWA S (communty surveillance) • Already mentioned. They expect output from the training to apply what they get from the training. • The government people in line with the training objective and budget availability. If the trainee move to other place or promoted, the office must have a response to this kind of situation. guidance is provided by accompanying the community in implementing and practicing what they learned. • KII respondents believed that different NGO partners have different approach. They expressed that the following are quite effective: • The champions, works in developing a standard way of working with the community. Working with community leaders, and the community to conduct workshop and training to change behaviour. The process of learning include socialization, bring back tradition and customary way of doing things, community members, villagers, and small-scale fishers; (ii) socialization activities (personal contact, and person-to-person dissemination of information); (iii) participatory coaching, apprenticeship, and walk-through process in case of formulating village policies and regulations, making soap and handicrafts, and developing eco￾tourism package; (iv) deployment of young and idealistic local staff with commitment to marine resource conservation; (v) learning from successful or model villages through actual cross-learning visit to model villages; and (vi) strong approach versus institutional approach which also considers strengthening of mandates, budgets, and technical capacities, thus ignoring institutional hierarchy and time constraints for sharing of information on vessel registration among different concerned units/ministries (i.e., Marine and Fisheries Unit, Sea Transportation Ministry, Tourism Unit) at provincial level; (iv) top leaders’ support: budget and facilities, and no rotation of personnel assignments; (v) training and awareness (socialization) programs for communities; and (vi) zonation process or initiative coming from indigenous peoples, who want conservation of their marine area, with continued government and partners (USAID, NGOs) support • Hindering factors: Reassignment of species, etc.) as a key condition for motivating conservation groups, fisher groups and women groups to apply knowledge/skills gained from training and capacity-building activities provided by SEA Project. • Male participants also recognized the adequacy of training provided: increasing their understanding of the importance and potentials of marine conservation and sustainable fishing methods. • Female participants reiterated the importance of socialization with communities as another condition for screening and institutional commitment/suppo rt, and hindering factors principally point to limitations in the wider institutional mandates, technical capacity and finances of base organizations (see for example document [5], [7] and [8]). 220 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments play games, etc. Their learnings are shared with friends and neighbors in local language. In simple term, “the project begins with propaganda” • Local leaders, gender, with passion to influence others • Experienced with other NGOs that worked with marine resource conservation • Individual commitment, like boat owners or fish suppliers that do not allow crew to use plastic bottle or do not deal with plastic waste. • Training of POKMASWAS and they do enforcement and surveillance in the community • It really depends on the partner NGO how they institutional support SEA Project, NGO partners, and local governments. personnel trained by SEA Project to other posts application of acquired skills from training activities provided by the Project. • Hindering factors: Female participants in Waisai, Raja Ampat in West Papua identified time constraints and other interests of women because of family/househol d related obligations as major constraints to their active participation. • All seven female participants indicated that they have not attended training on more secure tenure or managed access to fisheries, but some of them are aware of the training attended by their husband- 221 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments worked and implement these policies. For example, ILMMA site, the project involve the community at the grass root level, capacitate them to come up with their conceptual framework, developed their village development plan with village development fund. This is a good lessons in sustaining the project because the village/ community take over and maintain the activities because they see the benefit to their communities. • Some difficulties include: • Village leaders in competition with Champions that resulted in conflict that may derail or slow fishers, and the changes in local fishing practices and marine surveillance activities. 222 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments down the progress of the project (e.g. Mare, Ternate) • Movement/ transfer of government staff that were trained create vacuum and resulted to dealing with a new person not knowledgeable of the project. • Sometimes those trained have no way of applying what they learned because of lack of facility or budget, or support, etc. • Lack of budget and support from the community. 3. To what extent do non‐ fishing livelihood activities introduced or strengthened by USAID SEA change behavior and reduce threats and improve marine conservation (e.g., Livelihood program in Mare, Ternate, Galo‐ Galo, and Morotai)? • Galo-Galo with tourism, Rare at Raja Ampat for alternative activity, also ILLMA in Maluku. How they get money from the • According to all key informants, “there are so many example like eco-tourism in Galo-Galo, with POKDARWIS (tourism group) for eco-tourism working group. • Most key informants indicated that they do not have non-fishing livelihood activities, although WWF, WCS, CTC, and MDPI mentioned • All key informants recognized the potential effectiveness of non-fishing livelihoods to reduce negative impacts or displacements in communities where technical •All key informants noted an increase in number of local champions, and members of conservation groups (POKWARDIS and POKMASWAS) as evidence of local • All male and female participants In Galo-Galo, Morotai, North Maluku province cited specific marine conservation and sustainable fishing practices as • * Non-fishing livelihoods Project documents indicate that non￾fishing livelihoods introduced/strengt hened by SEA Project in Mare, Ternate, Galo-Galo and Morotai have resulted in more • 223 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments Under what conditions are these activities effective? biodiversity, that is why they have plan. They need alternative livelihood to reduce treat to MPA, so this is what SEA did in the area, how to increase community awareness and develop their resources. So, their objectives is for conservation practices that the success will pay off later. Also with the financial support for what will be alternative to support their economic level or opportunity. • Marine Change is looking for fund to • Community groups and fishers association for Fair Trade under MDPI. They have annual report and also POKMASWAS (Surveillance), for community surveillance and enforcement. In West Papua, there is RARE and ILLMA in Maluku.” • Marine Change is developing private partnership in tourism with dive operator promoting diving in Lease Island. In Maluku, with ILMMA facilitated the organization of community groups for non￾fishing livelihood activities and partnered with private sector for more income partnering with other NGOs including local NGOs on non￾fishing livelihood activities in their sites. • For key informants (ILMMA, RARE, and Marine Change) implementing or supporting non￾fishing livelihood activities in selected Project sites, they informed that such activities have just started, which include soap￾making, handicraft making, and eco-tourism. This means that villagers have not yet fully enjoyed or realized the expected economic benefits, but the potentials for good economic returns are quite achievable tools (e.g., EAFM, MPA, MSP, law enforcement) are being introduced by SEA Project. • Most respondents (6 of 9), however, indicated that they do not have sufficient information to comment on the success or impact of non-fishing livelihood activities. Remaining respondents (3 of 9) considered livelihood activities as effective in achieving behavior change and improved marine conservation. • The “socialization”, local champions￾building, funding support, and process (expert inputs, consultations, coaching and mentoring) used are recognized as effective enabling factors. villagers demonstrating change in behavior, reduction in threats and improvement in marine conservation. •In North Maluku, number of champions increased to about 50, who dedicated their life for conservation. Increase in number of conservation group members is also noted. •Key informants in Maluku and West Papua also indicated an increase in number of local champions, and members of conservation groups, but they have no idea of the exact numbers. •The conditions under which non￾fishing livelihood activities are effective as evidence of behavior change, reduction of threats, and improvement of marine conservation. These activities are:  Changes in human behavior: -Both Males and Females: maintain clean marine environment through practice of proper waste disposal, participation in cleaning of ocean, refrain from taking marine resources, and not mining beach sand. -Males: getting involved in marine ecosystem protection, and engage in alternative non￾fishing livelihood activities. -Females: planting flowers, and obeying positive behavior, reduced threats and improved marine conservation in the project sites (see for example documents [4], [5], and [9]). The effective and successful work of MDPI on fishing livelihoods were also noted. These are reflected in positive MEL targets achievement related, in whole or in part, to this EQ (e.g., SA1.1a-c, SA3.1, SA3.2, SA5.1, SA5.2). Positive conditions noted include the project process employed (from socialization to coaching/mentoring ) and the wide local participation and effectivity of local champions. Hindering conditions noted include the substantive amount of time needed to gain effective buy-in 224 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments support group activity. • Reef Check also for livelihood projects. generation. Also in Widi Island, there is a call for investment of the private sector to contribute, fair trade program with MDPI, Anova. There is a local private partner for selling the tuna ‘Fair Trade’ with a company called PT. Harta Samudera, • With Reef Check Indonesia, ILMMA and RARE they document community accomplishment in numbers by indicators. • There is some evidence of increase in income of the community through home based soap making, pottery, eco-tourism, and partnership with private sector like given time. • However, social benefits are already starting to emerge. In villages engaged in non-fishing livelihood activities, women organization learned to report their livelihood activities, and include them in their village plan with budget (Dana Desa). Villages have allocated funds for the first trial and sample production of soap and handicrafts. The other benefit includes local empowerment to develop and market their own soap products and eco-tourism services. • Hindering factors: Need for timeliness of funding support and local staff availability. identified by key informants include: (i) Facilitation, training and technical support (processing and marketing of products, and marine tourism) for communities by partner NGOs; (ii) Provision of support facilities for fishers for non-fishing livelihood activities; (iii) Development of and/or support for community empowermen t program (organization of processing and marketing groups to participate in Fair Trade). conservation regulations.  Reduction of threats: -Both Males and Females: mobilizing community to look after cleanliness of ocean, take care of seaweeds, and prevent use of fish nets that catch small fishes, and other destructive fishing methods.  Improvement of marine conservation: -Both Males and Females: creating/ increasing community awareness on conservation regulation and the importance of marine conservation, protecting marine resources. -Males: planting mangroves. -Females: maintain and on-time availability of appropriate staff given the remoteness of target sites/ communities (see document [5] and [9]). 225 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments hotels and local stores. • There is a huge evidence of change behavior towards marine resource conservation and non￾destructive fishing practices in the communities with non-fishing livelihood activities. • Still very early to see change in economic condition happening. The MPA is just beginning to be established. It will take a few years before the MPA areas can show social, ecological, and economic benefits. I think we are going there. cleanliness of marine environment. • Conditions under which these activities are effective: - Both Males and Females: socialization, non-fishing livelihoods, regulations. - Males:, training, CBO organization, and village government support. - Females: community cooperation and participation. 4. To what extent do USAID‐supported fishermen with more secure marine tenure in fisheries apply sustainable fishing • SEA Project is trying to minimize conflict with local fishermen and • Key Informants responded that these activities were sub￾contracted with national and • Most key informants responded that they do not have secure marine tenure fisheries • All key informants recognized the utility of more secure marine tenure for • Most key informants (5 of 8) responded that they have no sufficient information on • Majority of male and female participants in Pasinalo, Taniwel, West Seram, Maluku; and • * Secure marine tenure Project documents indicate that fishers with more secure marine tenure in • 226 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments practices and reduce destructive fishing practices (e.g., Territorial Use Rights for Fishing [TURF] in Raja Ampat and Local Marine Management Area [LMMA] in Maluku)? Under what conditions are these activities effective? communities. It is crucial how NGO contractors facilitate among the fishermen, how to deal with fishermen from outside of Maluku or Papua, like flying fish fishermen in Papua coming from Sulawesi. • For economic impact there is huge gaps and big difference between the earnings of local fishermen and fishermen from outside the Papua Barat. • But the assumption to establish village regulation that is more effective and for ILLMA the integration of local NGOs, to work with community to organize themselves. This work includes formulation of policies and regulations in order to get legal mandate from the gvernment (local, provincial and national). These partners also look at the possibility that the customary mandate will be supported and acknowledged by regional government. • For example, TURF provides right for local community fishermen. ILMMA and TURF increase the secure of marine tenure but the conflict from outside is bigger than inside. Local community wants to and local marine management area activities, although WWF, WCS, CTC, and MDPI mentioned partnering with other NGOs and local NGOs relative to these activities in their sites. • For those key informants (ILMMA, and RARE) that are implementing TURF in Raja Ampat, West Papua and LLMA in West Seram, Maluku are one in responding that their work in these sites is quite successful in changing attitudes, behaviors, and practices towards applying sustainable fishing practices and reducing destructive fishing practices. • These changes in behavior that include fisheries sustainability, improving local equity versus external fishers, and effecting improved marine conservation. • Most key informants (6 of 9), however, concede that they do not have enough information about SEA Project work on more secure marine tenure (as such responsibility belongs to other directorates) - pointing to the need for more effective project reporting and coordination across MMAF units. • The rest of key informants (3 of 9) think that the work on TURF (Rajah Ampat, West Papua) and LMMA (West Seram, Maluku), including MDPI the extent of contributions of more secure marine tenure to applying sustainable fishing practices and reducing destructive fishing practices. The rest of key informants indicated that the work on more secure tenure in West Seram, Maluku (LLMA) and Raja Ampat, West Papua (TURF) is effective. • In Maluku, key informants indicated that villages were able to come up with their regulations, establish their “Sasi” and change their behavior relative to destructive fishing practices. Local fishers stop using bomb, compressor. They now use fishing gears that Waisai, Raja Ampat, West Papua Province claimed that local fishers now apply sustainable fishing methods, and stop destructive fishing practices, by citing the following: (i) Use the right size of fish net; (ii) Stop the use of bomb, poison and other destructive and illegal fishing tools; (iii) Catch fish in designated fishing zone; (iv) Place banner, poster and/or billboard to inform all people about conservation regulations (v) Conduct marine surveillance. • Conditions under which these activities are effective: TURF in Rajah Ampat and LMMA in Maluku largely exhibit more positive behavior (i.e., use of sustainable fishing practices and avoid destructive fishing)(see for example document [4], [5], [7] and [9]). The positive achievement in SA3.3 (89%) reflect this success, with the project on the way to fully achieving targets in the remaining project lifespan (see for example document [6]). Enabling conditions noted include existence of strong customary/tradition al management and good local champions and participation. Hindering conditions cited mainly pertain to “substantive time to success and achievement of impacts”, local staff availability, and 227 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments the traditional/ customary beliefs and regulations to the scientific marine practices was found to be quiet effective. • For the success of the project, initiative, support from provincial level is a must. protect their area by conducting fishing activity toward conservation but the threat from the fisherman outside the area is bigger and the community don’t have the capacity to reduce these threats. • TURF based on cluster (zonation area) while ILMMA work based on customary rights. But still partly effective even if the awareness where we applied TURF and ILMMA increase because the threat outside is bigger than inside So still not really fully effective. Both RARE and ILMMA made sure that the protection and maintenance of their “Sasi” or no-take zone, adherence and adoption of their village regulations, maintaining cleanliness by not throwing trash and garbage is very effective because of the following reasons: 1.Awareness and acceptance of villagers that using bombs and illegal fishing gears are destructive to their marine resources and their livelihood. 2.Willingness and commitment of villagers to change based on their customary laws that support marine science and conservation 3.Support of village government for SEA Project initiatives on FADs, are highly successful and effective. • Positive conditions cited include good socialization, effective local champions, strong community involvement, and a tradition of customary/traditi onal rights and management. • Hindering factors: Time and budget, sustainability, site access and communication by staff. • Work on MPAs in marine tenure areas should look at its enhancement of fisheries (instead of marine tourism). do not destroy corals. • In North Maluku, key informants cited Sula Island and Tugulufa Site where protected areas are maintained, and economic benefits are derived by villagers from eco-tourism program. • Conditions under which these activities are effective: (i) Capacity of local government to monitor and evaluate changes in local conditions as a result of interventions, but it requires further improvement. (ii) Socialization with communities to teach them how to use fish net wisely that do not - Males: training and information provided by SEA Project; community participation, including women and children, in marine surveillance; local people’s observance of conservation regulations, and securing support of village government and traditional leader. - Females: training and information provided by SEA Project; and integration of customary tenure approach to conservation regulations (Sasi area). - Both Males and Females: the observed growth or local community harmony/relations (see for example document [5], [7] and [9]).” 228 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments community feel ownership of the project and the local community appreciate this very much. • Another approach is with MPA. For ecxample 5 MPAs in North Maluku under facilitation by WCS. With CI in West Papua. They facilitated agreement with the community for “no take zone”. This zone before and during the implementation with SEA is bothered by the fishers from outside like from Sulawesi that used bomb and poisons for fishing. • Very effective because of the good work and commitment of the partner NGO, ILMMA which really 4.Support of NGO partners and SEA Project to village activities 5.Institutionalizati on of policies and regulation as well as the establishment of no-take zone by getting approval from village council and including it in their village plan with annual budget. 6.Organizations of conservation groups, women for livelihood activities, POKMASWAS for marine surveillance, and POKDARWIS for tourism management 7.Identification, development and training of champions for more and sustainable socialization 8.Deployment of young local staff that stays and damage corals. However, the Provincial Government in North Maluku still feels the need for increasing people’s awareness, particularly in terms of law enforcement. (iii) Ability and participation of villagers to learn how to make and enforce village regulations reappearance of many and varied species of fish and other marine fauna. 229 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments work with the community and made sure that everyone in the organization are involved. ILMMA also are quite committed with the project because it provides its own funding through their own organizations budget or through other sources or other partners reside in the site. Hiring of young people with strong commitment towards achievement of SEA Project objectives 9.Villagers ownership of SEA Project interventions: they implement, monitor and report the interventions. They know that if they are successful, their marine resources will increase and bring more and more benefits to all of them. They are all looking forward to the benefits that they will enjoy in the future. 5. Given that USAID SEA addresses key marine threats and drivers through a complex project comprised of multiple components/ strategic approaches • Coordination is an effective strategy for all partner NGOs. Individual partners work • Most respondents agree that to be more effective, SEA Project must continue its relations and • All KII respondents pointed out that SEA Project Management is quite effective. All the Project • Most respondents (8 of 9) opined that SEA Project management is largely effective and successful How well are all the components/ approaches working together? North Maluku: • Effective - Strong • On strategic approaches working well together, both male and female participants to FGD sessions, • Project Management and Synergy Project documents indicate that project management and • 230 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments implemented by a large number of partners, how well are all the components/ approaches working together? What aspects of overall management are working well and what can be improved? To what extent are there capacity gaps among the partners, which might be hindering effective implementation? independently from each other but they need to discuss, sit down and talk to each other including the government, and lead by MMAF. Not easy to merge the different groups due to the kind of plan the project has. • For all management to be improved, interaction with USAID, and Tetra Tech must be improved. There is a need for a better management plan for this. continue getting updates from all partners and others. SEA Project activity plan approved by the government counterpart (MMAF). Including with Bappenas (National Planning Board), Ministry of Marine and Fisheries and also provincial level (Maluku, North Maluku and West Papua). • Coordination not only with project team at the provincial levels (project coordnators in every province, Ternate, Ambon and Manokwari) and with One Regional coordnator (for all) office based in Ambon but also with government and strategies and approaches are very relevant because they are all working together for the achievement of the goals and objectives of SEA Project. • Despite many partners, and many institutions and offices involved, all stakeholders work together for achievement of the Project objectives, while working on their respective mandates/ deliverables. • Key informants raised the following recommendation s and challenges that reflect capacity gaps in the Project implementation: • Improve pre￾finance or pre￾contractual arrangement Many NGO partners have across the five project strategic approaches/ components, 15 tasks and 13 NGO partners. This is noted to be true at national, provincial and site level (and noted to be increasing in effectiveness going down through such institutional hierarchy). • Effective synergy and management is maintained via various mechanisms (joint planning, regular coordination meetings, periodic consultations, workshops, working groups, task forces) and project communication/a dvocacy works. • Challenges toward more improved management and coordination and initiatives among and between SEA Project, national and provincial governments and partner NGOs, as demonstrated in the completion of the MSP or Marine and Coastal Zone Spatial Planning (RZWP3K). • Synchronization of planning process. Maluku: • Established a work forum (POKIA) for conservation at provincial, district and sub￾district levels. Through this forum we can separate the team for surveillance, regulation at provincial level, and data collection at district level. • Focus on regulation from indicated the importance of the combination of socialization, training, alternative livelihoods, regulations, and organization/ strengthening of conservation groups as the key to effective implementation of the Project. • In terms of capacity gaps which hinder effective implementation, the key factors identified include: (i) Still lack knowledge and skills on certain aspects of non-fishing livelihood development with emphasis on product development, packaging and marketing, homestay management, and human and synergy across components has been largely successful across strategic components/tasks and implementing partners/NGOs at national, provincial and site/local level (see for example document [5] and [9]). Challenges have evolved through the project implementation lifespan from March 2016 to date. Project management has been responsive through this period to address the evolving operational/tactical and strategic issues through, among others, joint planning, regular coordination platforms/ meetings (i.e., working groups, task forces, steering committees), technical workshops, relations-building, expert inputs, 231 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments other local private partners. • The need to continue what is it doing and to strengthen the alliance and partnership with NGO partners and government. The need for better communication with weekly, monthly meeting and reporting. • It’s very effective because of the coordination not only at national level but also at provincial and community sites. There are tasks forces working at central and provincial government including also fora and community groups, like fair trade groups and reimbursable system and not fixed contract. It is very hard for partners because it slows down implementation of activities while waiting for approval of activities; otherwise if we conduct activities without prior approval and later disapproved by the Project, then NGO partners will end up bearing the cost. Because partner NGOs need to finish their activities as planned, then they are sometime forced to conduct activities even if the budget is not approved yet. • We need more time to implement SEA Project interventions if our goal is to change attitude, behavior and synergy include addressing the following: (1) perception of “exclusivity” of NGO partners; (2) perception of “little direct benefits” to MMAF and its component units; (3) prospect of “sequential implementation” of project components; (4) limited time to achieve impacts/benefits; (5) “local emphasis” in selection of partners and staff deployment; (6) “more grants instead of subcontracts”; (7) improved “platform” for project coordination, reporting, monitoring and evaluation across implementing partners; and (8) administrative/fin ancial streamlining (for kementerian kelautan dan perikanan (KKP) for conservation. • SEA Project really helps us to never give up and encourage the community to change their behavior by using sustainable tools and protecting the coral reefs. West Papua: • Effective - strong coordination and synergy. • Training and technical assistance in making regulation, and MPA establishment. What aspects of overall management are working well and what can be improved? North Maluku: What are working well: organization values. (ii) Still lack knowledge and skills on the use of fishing gear and modern ways of fishing, including lack of supply of the right fishing gears for fishers involved in secure marine tenure or managed access to fisheries resources. • In Waisai, Raja Ampat, West Papua Province, both male and female participants raised the issue of lack of awareness/educ ation of local people about conservation regulations, and lack of time and interest to participate to the Project capacity-building and coaching/mentoring . The noted main strategic challenges remaining pertain largely to “government capacity and engagement level” and “time to success/impacts” sufficient to achieve project targets. Concerns have been raised that certain targets (i.e., SA2.2, 2.4 and 2.5) may not be achieved within the curret project lifespan due to these strategic challenges (see for example document [5] and [9]). 232 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments POKDARWIS (Tourism) and POKMASWAS (Surveillance) • The project is effective because it gets support from field coordinators, support from partners and support from government. Effective because of strong leadership and flexibility. • The project is only partly effective because of partner resource limitation budget & the pressure from people strongly pushing to get project done • Individual strategy with NGO partners. They discuss in every year, they sit down and sharing each other including practice towards marine conservation and non-destructive fishing activities. • The first year of the Project focused on its introduction to key stakeholders to request for their support. At village level, villagers refused to cooperate with the Project at this stage because the Project had no direct benefits to them. • On its second year, the Project worked on socialization and awareness raising, and organization and training of conservation groups. • On its third year, the Project started to implement ground-level interventions with direct economic benefits to the timely support and reimbursement of expenses). • Coordination between provincial and central government is improved. Also with NGO partners. Coordination in North Maluku is better day by day. • SEA Project also has contributed in improving the communication at provincial level including motivated us to take responsibilities. What can be improved: • Continue support of SEA Project because we have just started with MSP or RZWP3K (Marine and Coastal Zone Spatial Planning). Maluku: What are working well • We from Surveillance section in DKP activities due to other important family/househol d related obligations. 233 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments wth Government, of the authority and lead by Ministry of Marine and Fisheries Affairs (MMAF). • About the coordination, it is not easy to merge due to the kind of plan. For example, fisheries resources management unit at MMAF, has their own needs or idea and sometimes not matched or not relevant with SEA concerns, for example with fish reserve management. So, this is not easy for SEA to deal with this kind of ‘mismatch’. • For all management needs to be improved by interaction with community. However, NGO partners need more time to enable fishing villages to fully enjoy or realize the benefits from the Project. • Three or four years are not enough. More time is needed to succeed. • Need to deploy young and hardworking staff who should stay and reside in the village so that they can maintain personal contact with villagers. • Improve and open communication. It is difficult to connect the three provinces without good communication. • Need for all NGO partners to meet even only once a year. This will help NGO partners to know what the others are doing, and if thank for Sea Project for helping us in our financial need, and to help community. • Training of local champions. Like now, there is a workshop about champion. It is really helpful. What can be improved: • The size of FMA 715 is huge yet SEA Project is only covering a portion of it. There is a need to cover management of the whole FMA 715. West Papua: What are working well: • Coordination and communication between SEA Project, Provincial Government, and NGO partners. What can be improved: 234 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments USAID, and with the Tetra Tech main office, even though sometimes quite complicated. USAID and Tetra Tech main office doesn’t fully understand the big pictures of SEA Project. There is a need for a comprehensive program management. • In general, the project team understands each other. So they just keep going and working on over and over again. Team members are pretty open, difficulties around and how to solve it. For example, our experience with the travel budget moratorium (cannot travel needed can work or support each other especially that the local stakeholders are the same. There is also a need for more cohesion among NGO partners especially those working in the same site so there will be no overlap of activities, but rather they should complement each other. • At this time, however, the establishment of MPA must be given priority by the Project. That is SA2 relative to the establishment of MPAs because of the planned number of MPAs not one have been established. • The process of authorizing and adopting policies for MPAs is both done in local and national levels. • Only coordination issue, to maintain to inform and coordinate with the Provincial Government, even though it is already good. 235 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments until August) from the Main Office. Also our problem with ‘Ministry of Home Affairs’ is sometime complicated with the issues of RZWP3K (Provincial Zoning Plan), and the problems with some NGO partners, are complicating the project work but the project team handle them efficiently, • In terms of capacity gaps, of NGO partners, for the national, their capacity is OK but on the ground still lacking • We have improved our relationship in technical issues. • In some places, our partners face difficulties, lack of project management for There is a need to bridge the gap in raising the village or community regulation to be adopted in district level, then to provincial, and then to the National Level (MMAF). • Work closely with local and provincial government. In all the capacity building that NGO partners are providing, local and provincial governments must be aware and invited to participate. • It is really a challenge for all NGO partners to come up with so much reports: monthly, quarterly, annually, etc. We recommend to reduce the number of reports required from us because 236 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments some of our NGO partners. • Some NGO partners do not have permanent position for ME and L which create a lot of problem in reporting data and monitoring their progress relative to their target indicators and deliverable. • Each partner has weak and strong point. The gaps scientific aspect for example at stock assessment • Technical skill still needs to be improved for the NGO, such as WWF good for awareness but not for technical skill. • UNIPA, UKIP and NGO also get trained by NOAA. The university also still needs support for technical skill. the time spend for accomplishing them should be spend in the community. • Partnering and coordinating with academe for scientific works/products, research data, training, and resource partners. 237 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments • Buget allocation is a bit problematic for some partners. Programmatic applied for that kind of budget and large project locations, affecting also like travel and bureaucracy issues. Travel expense to sites (e.g. West Papua) Is very expensive. • The Project performance can still be improved but the limitation come from experts, time, and budget. • Recommenda￾tions:  Fishing and non-fishing livelihood activities – tourism, sustainable fisheries should be part of the implementati on and 238 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments establishment of MPA.  Replication of fair trade model, the partnership between Harta Samudra and MDPI which involves the partnership of SEA Project and MDPI, partnership of MDPI and Harta Samudra, the partnership of Harta Samudra and fish suppliers, and the partnership of fish suppliers and the local fishermen. Each partnerships must be strong and the influence of SEA project is carried and adopted in each level of relationship 239 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments and partnership.  Partners advocating stronger relations with the government through public-private partnerships in application and adoption of new system or model they have developed and tested in their areas in SEA project implementati on. These model or system can be adopted by the government in running sustainable marine resource conservation  Common/ uniform indicators/ concepts, methods of measurement 240 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments , study design in conducting M&E of the project (baseline, mid-term and end-of￾project) for all partners. 6. Based on the answers above about the conditions under which USAID SEA Project strategies are or are not effective, what are the recommended areas (thematic and/or geographic) of expansion/contraction for USAID? • What we need is a broader network with other donors like Walton . • Coordinaton with Kajiskan (National Committe fo Stock Assesmenet) to evaluate and monitor. • Coordination is happening but among the highest level but should be encouraged, • In Project location/site. The need to strengthen coordination between Chief of Party with these two SDI Marine • All the project strategies and approaches are still very relevant except for Marine Spatial Planning because this is already done and completed. Every aspect is complementing each other, such as capacity building that needs to be parallel with awareness and MPA can also serve for EAFM. • Improvement in sequence of approach such as fisheries area based on management plan. • Now SEA team must continue working on • For most key informants, all the Project strategies are still very relevant. • For some SA2– MPA, MSP and EAFM, SA3, and SA5 are relevant; and SA1 and SA4 can be combined/merge d into one activity - awareness raising can be combined and merged with the identification, formation, and training of local champions. Local champions will lead socialization and awareness raising in the community. • All key informants are one in saying that SEA Project must Thematic Areas: • All thematic and technical areas of the Project are still deemed relevant, although consideration of sequential implementation (for tractability) and recognition of “lag time to achieve impacts” was suggested in future projects. Geographic Areas: • Coverage of more sites in the provinces of North Maluku, Maluku and West Papua is recommended, and coverage of the 3 other provinces bordering FMA 715 which are not included in • Thematic Areas: • MSP: More technical assistance in planning on zoning management • Monitoring system to cover not only conservation but also socio￾economic development • Extension of SEA Project for 1-2 years to ensure its sustainability, which to a large extent depends on the provincial capacity and initiatives; • Need for community empowerment • More time for SEA Project to • Most of the recommendation s made by FGD participants (both males and females) are focused on capacity gaps of local communities/ villages involved in non-fishing livelihood activities, and secure marine tenure or managed access to fisheries resources. These capacity gaps include training and logistical support (transport and communication equipment, fisher gears, business and tourism related equipment, • Thematic/geographi c expansion or contraction for USAID Project documents indicate that the project strategic components/tasks all remain relevant for sustainable fisheries and improved marine biodiversity in Indonesia. No explicit mentions of thematic contraction areas were evident from the documents. Geographic expansion has been noted to be desirable to cover the three provinces adjoining FMA 715 outside the scope of the project, as well as expanding activities to include more • 241 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments Fisheries Management under Capture Fisheries of MMAF and KKHL including Bappenas. Coordination among the contractors or sub￾contractors • Communicati on strategy for the project. • Improving the capacity of officers, recruit qualified local people. sustainable fisheries, and law enforcement. • MPA is on track but cannot move faster – Its success is very limited. Policy is very slow because of its complexity after the implementation of Decree No. 23/2014 and the need to have RZWP3K in the area, • Resources of SEA has been allocated to support the cmpletion of this document even though this was not so easy to do due to lack of capacity and budget from the provincial level. • Completion of RZWP-3-K required the transfer of budget, personnels and document continue, extended and must not end because of the important role it plays in marine resource management and conservation in Indonesia. • Here are the key informants’ recommendation s: 1.Work with the groups that are already established in the village and no need to form or create a new group. For example, there is already a group that does surveillance and enforcement of regulations in the village no need to form a new group (e.g. POKMASWAS) work with the existing group 2.USAID to act as bridge between the community and local government and the SEA Project was recommended. Replication capacity to cover the 3 other provinces (not currently covered by SEA Project) in the future is a concern. • Sustainability and wider replication of current efforts in North Maluku, Maluku and West Papua may require extension of SEA Project support. build the capacity and competence of the Provincial Government staff on conservation; • Strengthen the capacity of government at provincial level for collaboration with other government units, local partners (NGOs and CBOs • More SEA support to increase the capacity of ASN (civil servant) to get GIS training, diving, MPA, and the preparation of making regulations. Geographic Areas: • Expand area coverage not only the three provinces within FMA 715 but the entire FMA 715, including FMA 714 and 718; organizational development of CBOs, and product packaging, marketing and promotion. • More distinct recommendation s made by male participants involved in secure marine tenure or managed access to fisheries in Waisai, Raja Ampat, West Papua Province include: the use of local champions to dissemination/ spread information, share lessons learned from other areas; and use easily understandable information materials. sites/replicates within the three provinces currently covered by the project. Positive impacts were expected from more complete and expanded coverage of provinces adjoining FMA 715 given the nature of coastal resources and dynamics in the area.” 242 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments (P3D) so, SEA significantly contributed in this issues and counted as good achievement. • Provinical Government especially like North Maluku appreciates this as compliment despite thier experiences in marine and fisheries management as ‘new’ province’. • Travel cost is so high and has affecting the allocation of personnels and work plan. • About project management for all, all levels, to reduce the complexity of bureaucracy of the project. There is a concern about financial management. The Tetra Tech Main office should stay here other coastal communities. The need to bridge the gap between the local government and the coastal communities. 3.Zone Planning – villages with their own regulations and zone maps should be integrated into MSP. SEA Project work at the provincial level – but they must work at the district level and push or bridge the district to endorse the zoning of the village level. 4.Village regulations do not contradict the government regulations therefore the need to be integrated with the district and provincial level regulations. 243 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments better management and relevant control. • Recommendatio ns: Expand the area instead of 3 provinces to 6 provinces such as small pelagic because some data take at Bitung Port and Ternate Port. • More coordination with partner, more synchronize with partner, stronger leadership and holding hand together to achieve the goal of the project. • Engage the activity as many as possible, talk with people who don’t know detail about this project. Solve the communication with people who has different personality 5.Acknowledgeme nt and recognition of the local government of these villages is important. That they should be seen by other villages as good example and role model so other villages will follow. 6.The issue of sustainability and continuity of local champions, POKMASWAS, POKDARWIS, and the other local organization in the villages/commun ity, in particular, the sustainability of local SASI-no￾take zone and the maintenance of their local regulations - when SEA Project ends. The NGO partners and Universities recommend that the project 244 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments problem and find other means/media of communicating with different groups • Recommendatio ns: the project must obligate partner for the inclusion of gender empowerment and development in their budget because most of the activities conducted by partners are for the benefit of men. • To encourage public-private partnership, the model of investing in the livelihood activities, where one local person is showing a successful business venture may elicit positive response to other residents to do the same should look seriously and make the issue of sustainability a priority before the end of SEA Project. 7.Increase the frequency of coordination and communication between SEA Project, NGO partners, provincial government and the community. For example for vessel registration, before the activity, there is a need for intensive communication to the community so they is a need for massive information campaign so they understand the benefit of vessel registration and explain the paperwork involve that they 245 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments or follow suit, The best example is the homestay activity for local tourism activity in ILMMA sites which originally the places to stay were provided or run by foreigners but when a local person started doing it and others observed how profitable it is, how successful is the business with many guest coming, they follow suit. That is how homestay started in Taniwel. • Law enforcement because the process is too complicated in the national level. The process should be simplified so the policies can then be need to fill up and upload. 8.Time and efforts spend for all the Project activities in the communities need logistical, financial, and technical supports 9.Time is of the essence. if our goal is to change attitude and behavior and practice towards marine resource conservation and non￾destructive fishing activities. These are long term goals. In Indonesia, the first year of the project is still introducing the project and requesting for their support which usually because there is no direct benefit and impact to the community, the first time 246 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments authorized and adopted in localities • The project restriction on travel, the focus must be on targets that have not been achieved yet (e.g. policy and regulations. • To look into the areas of marine debris/ marine waste inclusion in surveillance of POKMASWAS they will refuse to cooperate. The second year of the project is organizing and socialization and awareness raising, and training, etc. And in the third year, we begin to implement the project so for the impact to be felt by the community we need longer time. Three and four years is not enough. The project need more time to in order to accomplish its target and succeed. 10. Shorten and simplify the bureaucracy, especially for the activity which involved cross cutting stakeholders. Need time and effort to build the emotional 247 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments relationship between them. Further, in the process of adoption and authorization of policies, the process at the national level is so slow and the bureaucratic red tape should be shortened. 11. Increase the capacity of students in universities to do research and publication. Recommend that more students get involved to gather and analyze data for their thesis. Need more reliable data and more scholarly works and publications for the topic 12. Push for more strong 248 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments connectivity between regions. For example, South Sorong, Bintuni, and Sorong must be connected because of the need to study their shrimp stock which until now the data does not exist 13. The Project must seek cooperation, coordination, and partnerships with the institution (e.g. the whole University) and not individual (a particular faculty in Marine Science Department). With individual contract, the impact is only for the benefit of the 249 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments individual and the institutional impact is quite minimal. 14. Thematic recommendati on: do not just focus on flying fish or shrimps, or anchovy, etc (or a particular species of marine resource) because in one site, there are many other potential resource to be developed e.g. reef fish, pelagic fish, etc. 15. Improved awareness of the community to protect endangered species, e.g. turtle. Studies and science in combination with the communities 250 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments customary/tra ditional knowledge must be used in messaging and communicatio n so villagers understand how important it is to protect endangered species. • In Rajah Ampat areas, there is a strong need to involve women in the project and expand their role. The project must find out how to support and empower women – more training, more involvement in fishing or non￾fishing livelihood activities, more involvement in community organization, local champions, etc.  What is the overall FGD Outputs 251 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Evaluation Questions Analysis by Data Gathering Method Synthesis KII FGD Desk Review USAID TETRA TECH/SEA Project NGO Partners MMAF Local Governments performance of SEA Project, and the factors and conditions that enhanced or diminished the achievement of the Project results at midterm? (Evaluation Objective 1: Review of midterm performance, and analysis of factors and conditions that enhance or diminish the achievements of the Project results)  Overall, both male and female participants to the FGD sessions have generally expressed satisfaction from the performance of the Project, particularly the benefits gained from their participation to the Project.  In terms of conservation and improvement of marine and fisheries resources, majority have recognized the change in local fishers’ behavior to stop destructive fishing practices, and apply sustainable fishing methods. Overall, they also noted the change in local villagers’ behavior to get involve in the cleanliness of the seashore and ocean, not throwing garbage indiscriminately; and the conduct of marine surveillance.  In villages involved in non-fishing livelihood activities, the introduction of alternative sources of livelihood, supported by the socialization, training and regulations are perceived to be effective to encourage and equip local people to apply sustainable fishing methods and stop destructing fishing methods.  In villages involved in secure marine tenure or managed access to fisheries resources, regulations built on customary law/rights (Sasi) and made through village government and/or traditional leader, and supported by socialization and training, are perceived to be effective for local fishers to apply sustainable fishing methods, stop destructive fishing methods, and observe/implement the regulations.  In both groups of villages, while changes in behavior of local villagers’/fishers’ have largely improved, many FGD participants raised concerns over the continued intrusion of fishers into the no-take zones (protected areas), particularly from Sulawesi, who are still using bomb and poison for fishing. POKMASWAS and other local law enforcement groups lack the capacity (people, transport and communication equipment) to control these fishers from other villages. * Document References [1] SEA Project. 2018. Indonesia Sustainable Ecosystems Advanced (SEA) Project: Monitoring, Evaluation and Learning (ME&L) Plan (5th Revision). The SEA Project, Jakarta, Indonesia. 115 p. [2] SEA Project. 2017. Indonesia Sustainable Ecosystems Advanced (SEA) Project: Integrated Communication and Coordination Plan. The SEA Project, Jakarta, Indonesia. 84 p. [3] SEA Project. 2017. Indonesia Sustainable Ecosystems Advanced (SEA) Project: From Awareness Raising to Behavior Change - Considerations in Support of Strategic Approach # 1. The SEA Project, Jakarta, Indonesia. 35 p. [4] SEA Project. 2018. Activity Report: Buano Islands Rapid Baseline Behavior Assessment Pre-Test (West Seram, Maluku, Indonesia; 22-28 May 2018. The SEA Project, Jakarta, Indonesia. 16 p. [5] SEA Project. 2019. USAID Sustainable Ecosystems Advanced (USAID SEA) Project Annual Report (October 2018 - September 2019). The SEA Project, Jakarta, Indonesia. 147 p. [6] SEA Project. 2019. USAID Sustainable Ecosystems Advanced (USAID SEA) Project FY20/21 18-Month Work Plan (October 1, 2019 to March 22, 2021). The SEA Project, Jakarta, Indonesia. 147 p. 252 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV [7] SEA Project. 2019. Activity Report: Forum/Workshop on Promoting & Strengthening Local Fisheries Management in Indonesia’s West Papua Province (Manokwari, West Papua, Indonesia; 12 February 2019). The SEA Project, Jakarta, Indonesia. 38 p. [8] SEA Project. 2019. Training-Workshop for Researchers and USAID SEA Partners on Stock Assessment and Catch Monitoring (Bogor, July 29-August 2, 2019 and Sorong, August 5-8, 2019. The SEA Project, Jakarta, Indonesia. 18 p. + 2 annexes. [9] SEA Project. 2020. USAID Sustainable Ecosystems Advanced (USAID SEA) Project Quarterly Report: First Quarter of FY2020 (01 Oct - 31 Dec 2019). The SEA Project, Jakarta, Indonesia. 94 p. [10] DCAFS. 2012. Technical Guidelines for Evaluating the Management Effectiveness of Aquatic, Coasts and Small Islands Conservation Areas (E-KKP3K). Directorate for Conservation of Area and Fish Species (DCAFS), Directorate General of Marine, Coasts and Small Islands, Ministry of Marine Affairs and Fisheries, Jakarta, Indonesia. 61 p. + ix.” 253 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV ANNEX M. SUMMARY OF TRAINING COURSES PROVIDED BY THE PROJECT TO KEY STAKEHOLDERS, FY2016 – FY2019 Training Title Level Year Participants Male Female EAFM Y1 Stock Assessment and Harvest Strategies for MMAF Researchers 2017 17 6 Fishery Data Collection Training and Profiling Initiation 2017 11 2 Intensive Course on Fish Stock Assessment and Harvest Strategy 2017 3 2 Capacity Building Visit to Western Pacific Fishery Management Council 2017 6 3 37 15 Y2 Fisheries Enumerator Training Maluku 2017 8 6 Sustainable Use of Fisheries Resources by Applying Blue Economy to Support the Community-Based MPA Compliance North Maluku 2017 11 8 Sustainable Fisheries West Papua 2018 30 6 Village Regulation Preparation West Papua 2018 23 0 Formulation of Reference Points for Fisheries Management National 2018 11 5 Stock Assessment and Fisheries Management Framework National 2018 35 22 118 47 Y3 Sustainable fisheries management West Papua 2018 45 5 Capacity building and development of Village Customary Regulation on Natural Resources Management Maluku 2019 24 1 Training on Certificate of Proficiency of Fish Handling National 2019 61 4 Refreshment training on CPUE Data Collecting Maluku 2019 2 3 Refreshment training on CPUE Data Collecting Maluku 2019 2 3 Refreshment training on CPUE Data Collecting Maluku 2019 2 3 Training-workshop on Concept, Theory and Methodology to Set Operational Objectives of a Harvest Strategy North Maluku 2019 17 5 Fisheries Product Training West Papua 2019 0 30 Refreshment training on CPUE Data Collecting Maluku 2019 3 2 Refreshment training on CPUE Data Collecting Maluku 2019 2 2 Household Product Certification Training for Fisheries Processed Product Maluku 2019 0 23 TOT SKKNI Fisheries Onboard Observer National 2019 24 0 Training on Inspired Light Tool for Tuna Value Chains National 2018 3 5 SKPI Ternate North Maluku 2017 10 0 195 86 350 148 MPA Y1 Principles of MPA for MPA Managers 2016 16 4 Principles of MPA for MMAF Fisheries Extension Officers 2016 16 5 Principles of MPA 2017 13 2 45 11 Y2 Signing Blue Assessor Maluku 2017 8 3 First Responder Initiation Workshop West Papua 2018 24 2 254 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Training Title Level Year Participants Male Female Training of Trainers (TOT) for Marine Protected Area (MPA) Design Using MARXAN Maluku 2018 16 9 MPA101 for Community in Lease Islands Batch 1 Maluku 2018 20 6 MPA101 for Community in Lease Islands Batch 2 Maluku 2018 21 4 MPA101 for Community in Sula Batch 1 North Maluku 2018 26 1 MPA101 for Community in Sula Batch 2 North Maluku 2018 20 1 MPA Design and OSPC for Sula, Buano and Lease Stakeholders Maluku 2018l 13 6 Fundamental Marine Protected Area Training For Fisheries Extension Officers In FMA 715 Maluku 2018 12 13 MPA101 for Community in Buano Maluku 2018 17 5 MPA Design and Open Standard for The Practice of Conservation Training for Central Java Government Staff National 2018 17 5 MPA101 for POKMASWAS and Fisheries Extension Officers National 2018 19 7 233 62 Y3 OSPC Open Standard for MPA management planning National 2018 11 1 POKMASWAS Training North Maluku 2018 34 2 OSPC Miradi training and field visit Nusa Penida for POKJA North Maluku National 2018 9 9 Basic Marine Conservation Training for Local Communities Maluku 2018 14 2 Basic Marine Conservation Training in Sorong Selatan West Papua 2018 10 2 Basic Marine Conservation Training for Local Communities Maluku 2018 19 2 OSPC Miradi Training and Field Visit to Nusa Penida for Marine Conservation Task Force (POKJA) North Maluku 2018 8 1 MPA & MPA Networks Design Training Module First Course National 2019 5 1 MPA101 Training for West Papua Provincial Government Officials West Papua 2019 5 11 MPA101 Training for Regional Government Officials West Papua 2019 16 2 Fundamental Marine Protected Area (MPA) Training for Communities/Stakeholders in Fatkauyon, Sula, North Maluku Province North Maluku 2019 17 2 Fundamental Marine Protected Area (MPA) Training for Communities/Stakeholders (Head of Soa And Government Village Staff) In Buano Islands Maluku Province. Maluku 2019 14 1 Fundamental Marine Protected Area (MPA) Training for Communities in Porto-Haria, Lease Islands, Maluku Province Maluku 2019 19 4 Training and Competency Test for Assessor in Marine Conservation Planning National 2019 3 4 2nd Training of Trainer (TOT) for Designing and Evaluating MPA and MPA Networks in Indonesia North Maluku 2019 13 6 Coaching Clinic on the Development of MPA Management and Zoning Plan Maluku 2019 14 7 Resource use monitoring (RUM) and community-based fisheries surveillance group (POKMASWAS) in Sula North Maluku 2019 20 0 240 57 518 130 MSP Y2 Marine Spatial Planning Development and Implementation 2017 8 5 Training Workshop on Marine Spatial Planning Data Management 2017 6 6 14 11 Y3 Training of Trainers of MSP Geoportal Database National 2018 5 4 Utilization of MSP Spatial data for Developing the Online Geoportal System in Maluku Province Maluku 2018 13 13 Training of Trainers II SEANODE MSP Geoportal RZWP-3-K Online National 2019 10 6 28 23 255 | USAID SEA PROJECT MID-TERM PERFORMANCE EVALUATION – FINAL MTPE REPORT USAID.GOV Training Title Level Year Participants Male Female 42 34 LE Y1 Port State Measures 2016 33 2 Field Study on VIIRS Validation through Inter-comparison VIIRS Radar Data 2017 13 2 VIIRS Internship at NOAA 2017 4 0 40 4 Y2 Training of PSM Inspector to Implement PSM Agreement National 2018 31 2 31 2 Y3 Training on marine surveillance West Papua 2018 50 0 Sorong Selatan Community Based Surveillance Group (POKMASWAS) Training West Papua 2018 9 2 POKMASWAS Training North Maluku 2018 13 2 POKMASWAS Study Visit West Papua 2019 3 0 POKMASWAS Training West Papua 2019 31 1 POKMASWAS Training North Maluku 2019 25 4 Training of Trainer (TOT) Community Based Marine and Fisheries Surveillance Group (POKMASWAS) National 2019 12 7 Surveillance SOP training West Papua 2019 35 3 POKMASWAS training for improving and strengthening Marine Surveillance Group (POKMASWAS) in Serutbar MPA Maluku 2019 13 0 191 21 262 27 Y1 Tourism/Communication/Others Community Perception Monitoring in MPAs for Fisheries Extension Officers at FMA-15 2017 13 6 13 6 Y2 Communication and Outreach Training Workshop West Papua 2018 25 4 First Responder Training on Stranded Marine Mammals Maluku 2018 21 2 46 6 Y3 Fisheries Product Processing – Part 2 (Tubir Tolu group) Maluku 2018 0 20 Fisheries Product Processing – Part 2 (Batnata group) Maluku 2018 0 18 Dive Training for Community Leaders and Partners Maluku 2018 20 1 Training workshop to develop village based potential tourism package Maluku 2019 12 4 Household Food Industry Certification Training West Papua 2019 0 17 Dive Training for Community Leaders and Partners in Sula Islands Regency North Maluku 2019 19 2 Organizational and Bookkeeping Training North Maluku 2019 5 0 Organizational and Bookkeeping Training North Maluku 2019 5 0 Organizational and Bookkeeping Training North Maluku 2019 3 0 Champion training North Maluku 2019 28 3 92 65 151 77 OVERALL =1,739 1,323 416