ii Evaluation Mid-term Performance Evaluation of the Caribbean Marine Biodiversity Program iii Mid-term Performance Evaluation of the Caribbean Marine Biodiversity Program MAY 2018 Report authors: Alex Arrivillaga, Rich Wilson, Robert Simmons, Kasia Grasela, Rudo Udika and Paul Fedner Zamy Cover photo: Marine park vessels and yachts in the Tobago Cays Marine Park (Rich Wilson) Prepared under: Contract number Contract No. AID-517-C-17-00001. USAID/Caribbean Regional Development Program's Caribbean Marine Biodiversity Program (CMBP) Alter Modus International Corp. 1615 M Street, NW Second Floor Washington, DC 20036 DISCLAIMER The views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. iv Table of Contents List of Acronyms, Figures and Tables ........................................................................................ 1 Executive Summary ...................................................................................................................... 3 Evaluation Purpose ................................................................................................................... 3 Program Background ............................................................................................................... 3 Evaluation Design ...................................................................................................................... 3 Findings, Conclusions and Recommendations ....................................................................... 4 Program Background ................................................................................................................... 8 Background ................................................................................................................................ 8 Description of the Problem, Development Hypothesis, and Theory of Change ................ 11 Summary of Evaluated Activity ............................................................................................. 15 Evaluation Purpose, Methods and Limitations ........................................................................ 15 Findings, Conclusions and Recommendations by Seascape ................................................... 22 Expected Result 4: Effective Management and Governance of MPAs .............................. 22 Samana Bay Seascape (Dominican Republic) ................................................................... 22 Grenadine Bank Seascape (Grenada and St. Vincent and the Grenadines) .................. 29 North Coast of Hispaniola Seascape (Haiti) ...................................................................... 38 Pedro Bank and Southwest Coast Seascape (Jamaica) .................................................... 43 Expected Result 3: Sustainable Fisheries and Sustainable Livelihoods for Fishers ......... 51 Samana Bay Seascape (Dominican Republic) ................................................................... 51 Grenadine Bank Seascape (Grenada and St. Vincent and The Grenadines)................... 8 North Coast of Hispaniola Seascape (Haiti) ...................................................................... 13 Pedro Bank and Southwest Coast Seascape (Jamaica) .................................................... 18 Sub-grantee Organizational Capacity Assessment .............................................................. 24 Center for the Conservation and Eco-Development of Samana Bay and its Surroundings ........................................................................................................................ 25 Sustainable Grenadines, Inc. .............................................................................................. 29 Foundation for the Protection of Marine Biodiversity ..................................................... 31 Caribbean Coastal Area Management Foundation C-CAM ........................................... 34 Regional Findings, Conclusions and Recommendations ......................................................... 37 Summary of Performance Towards Program Targets ........................................................ 37 Overarching Conclusions ....................................................................................................... 40 Regional-level Recommendations .......................................................................................... 42 v Annex I: Evaluation Matrix ....................................................................................................... 45 Annex II: MPA Management Plan Evaluation Tool ............................................................... 57 Annex III: Synthesis of TNC Annual Progress Reports.......................................................... 73 Annex IV: Sources of Information ............................................................................................ 90 Plans, Reports and Technical Documents ............................................................................. 90 Interviews and Focus Group Discussions ............................................................................. 92 CMBP Sites Visited ................................................................................................................. 93 Annex V: Data Collection Instruments ..................................................................................... 96 Field Interview Questionnaire Guide .................................................................................... 96 Evaluation Questionnaire – Government Officials ............................................................ 101 Evaluation Questionnaire – Sub-grantees/MPA Authorities............................................ 111 Evaluation Questionnaire – Fishers and Relevant Stakeholders ...................................... 121 Evaluation Questionnaire – Program Implementer .......................................................... 129 Annex VI: MPA Management Plan and Action Plan Scores ................................................ 139 Action Plan for the Protection of Corals in Samana Bay .................................................. 139 Action Plan for the Protection of Marine Mammals in Samana Bay ............................... 148 Action Plan for the Estero Hondo Marine Mammal Sanctuary ....................................... 159 Adaptive Mooring Management Plan – Sandy Island Oyster Bed Marine Protected Area ................................................................................................................................................. 172 Adaptive Mooring Management Plan – Tobago Cays Marine Park ................................ 189 Sandy Island Oyster Bed Marine Protected Area Management Plan ............................. 208 Three Bays National Park Management Plan .................................................................... 221 Pedro Cays and Surrounding Waters Management Plan ................................................. 240 Annex VII – TNC Statement of Differences ........................................................................... 250 Annex VIII – Table of monitoring indicators ........................................................................ 276 1 List of Acronyms, Figures and Tables AFADFA Association des Femmes Actives pour le Developpement de Phaeton AGRRA Atlantic and Gulf Rapid Reef Assessment AMI Alter Modus International Corporation ANAP Agence Nationale des Aires Protégées BBFFS Bluefields Bay Fishermen’s Friendly Society CARSEA Caribbean Sea Ecosystem Assessment C-CAM Caribbean Coastal Area Management Foundation CCI Caribbean Challenge Initiative CDP Caribbean Development Program CDCS Country Development Cooperation Strategy CEBSE Centro para la Conservación y Ecodesarrollo de la Bahía de Samana y su Entorno CFF Carriacou Fisher Folks CIBIMA Centro de Biologia Marina, Universidad Autonoma de Santo Domingo CMBP Caribbean Marine Biodiversity Program CODOPESCA Consejo Dominicano de Pesca y Acuicultura CPMWTA Carriacou and Petite Martinique Water Taxi Association CRDP Caribbean Regional Development Program E3 Bureau for Economic Growth, Education, and Environment FGD Focus Group Discussion FONDOMARENA Fondo Nacional Para el Medio Ambiente y Recursos Naturales FOPROBIM Foundation for Protection of Marine Biodiversity FSAP Fisheries Sector Action Plan GEF Global Environment Facility HACCP Hazard Analysis and Critical Control Points IDB Inter-American Development Bank IEDIH Integrated Development in Haiti IR Intermediate Results JDF Jamaica Defense Force LAC Latin America and the Caribbean MARENA Ministerio de Medio Ambiente y Recursos Naturales MARNDR Ministere de l’Agriculture, des Ressources Naturelles et du Développement Rural METT Management Effectiveness Tracking Tool MMA Marine Managed Area MPA Marine Protected Area MOCPMA Ministry of Carriacou and Petite Martinique Affairs NBSAP National Biodiversity Strategic Action Plan NEPA National Environment and Planning Agency (Jamaica) OCA Organizational Capacity Assessment OFM Office of Financial Management OPBL Organisation des Pécheurs pour la Protection de Bord de mer de Limonade PA3B Three Bays Protected Area RDCS Regional Development Cooperation Strategy 2 RFSA Rapid Fisheries Sector Assessment SFCA Special Fisheries Conservation Area SIOBMPA Sandy Island Oyster Bed Marine Protected Area SUSGREN Sustainable Grenadines TCMP Tobago Cays Marine Park TNC The Nature Conservancy TOC Theory of Change UNDP United Nations Development Program USAID United States Agency for International Development Figure 1 – CMBP Theory of Change Figure 2 – CMBP Results Framework Figure 3 – Phases of the Evaluation Process Figure 4 – OCA Scoring System Table 1 – Evaluation: Action Plan for Protection of Corals in Samana Bay Table 2 – Evaluation: Action Plan for Protection of Marine Mammals in Samana Bay Table 3 – Evaluation: Action Plan for Estero Hondo Marine Mammal Sanctuary in Samana Bay Table 4 – METT results for Parque Nacional de los Haitises Table 5 – METT results for Manglares del Bajo Yuna Table 6 – METT results for Bancos de la Plata y Navidad Marine Mammals Sanctuary Table 7 – Observations and Recommendations: Expected Result 4 Samana Bay Table 8 – Evaluation: Adaptive Mooring Management Plan for the Sandy Island Oyster Bed Marine Protected Area Table 9 – Evaluation: Adaptive Mooring Management Plan for the Tobago Cays Marine Park Table 10 – Evaluation: Sandy Island Oyster Bed Marine Protected Area Management Plan Table 11 – METT results for Sandy Island Oyster Bed Marine Protected Area Table 12 – METT results for Tobago Cays Marine Park Table 13 – Observations and Recommendations: Expected Result 4 Grenadine Bank Table 14 – Evaluation: Three Bays National Park Management Plan Table 15 – METT Results for Three Bays National Park Table 16 – Observations and Recommendations: Expected Result 4 Three Bays Table 17 – Evaluation: Pedro Cays and Surrounding Waters Management Plan Table 18 – METT Results for South West Cay Special Fishery Conservation Area Table 19 – METT Results for Bluefields Bay Special Fishery Conservation Area Table 20 – METT Results for Galleon St. Elizabeth Special Fishery Conservation Area Table 21 – Observations and Recommendations: Expected Result 4 Pedro Cays and Southwest Coast Table 22 – Observations and Recommendations: Expected Result 3 Samana Bay Table 23 – Observations and Recommendations: Expected Result 3 Grenadine Bank Table 24 – Observations and Recommendations: Expected Result 3 Three Bays Table 25 – Observations and Recommendations - Expected Result 3 Pedro Cays and Southwest Coast Table 26 – Score Results: CEBSE organizational capacity assessment Table 27 – Score Results: SUSGREN organizational capacity assessment Table 28 – Score Results: FOPROBIM organizational capacity assessment Table 29 – Score Results: C-CAM organizational capacity assessment Table 30 – Interviewed Stakeholder Groups 3 Executive Summary Evaluation Purpose The purpose of this mid-term performance evaluation (hereafter “performance evaluation”) of USAID/Caribbean Regional Development Program’s Caribbean Marine Biodiversity Program (CMBP) is to provide the United States Agency for International Development (USAID), its implementing partner The Nature Conservancy (TNC), and other stakeholders with robust and independent findings, conclusions, and recommendations regarding CMBP performance since its inception. USAID and its implementing partner are expected to use the findings, conclusions, and recommendations generated by this evaluation, as appropriate, to make any needed mid-course adjustments for the remaining life of the activity, highlight successes to date, and bring to bear evidence for design and implementation of similar and/or future regional marine biodiversity conservation activities in the Caribbean and other regions. Program Background CMBP aims to reduce threats to marine-coastal biodiversity in priority areas across the Caribbean (hereafter “seascapes”), which include high biodiversity ecosystems like coral reefs, mangroves, and seagrasses. CMBP’s main goal is to increase the sustainability and resilience of these marine resources and the nearby communities where program activities are implemented. Activities to date have targeted Marine Protected Areas (MPAs) through many different mechanisms and approaches, including strengthening legal frameworks, filling information gaps, developing or refining management plans, and regularly assessing management effectiveness. Additionally, activities were designed to assess the state of each participating countries’ fisheries and then prepare action plans that inform updates to national fisheries policy. Moreover, these Fishery Sector Action Plans (FSAPs) identified priorities and actions for engaging fishers, promoting more sustainable fishing practices, and improving compliance and support for no-take MPAs and fish sanctuaries across the region. Under CMBP, TNC is carrying out USAID-funded ocean and coastal activities across five countries in the Caribbean: Haiti, the Dominican Republic, Jamaica, and Grenada and St. Vincent and the Grenadines. CMBP is design to implement activities in four seascapes:  Samana Bay Seascape (Dominican Republic)  Grenadine Bank Seascape (Grenada and St. Vincent and the Grenadines)  North Coast of Hispaniola Seascape (Haiti)  Pedro Bank and Southwest Coast Seascape (Jamaica) Evaluation Design USAID and TNC decided to focus the performance evaluation on two CMBP Expected Results:  Expected Result 3: “Sustainable Fisheries and Sustainable Livelihoods for Fishers,” and  Expected Result 4: “Effective management and governance of Marine Protected Areas and/or Marine Managed Areas (MPAs/MMAs)” The following overarching questions guided the performance evaluation: 1. To what extent has CMBP met its targets under Expected Result 4: Effective management and governance of MPA/MMAs? What implications does CMBP performance on Expected Result 4 4 have for achieving CMBP’s overall objective of reducing threats to marine-coastal biodiversity? 2. To what extent has CMBP met its targets under Expected Result 3: More sustainable fisheries sector? What implications does CMBP performance on Expected Result 3 have for achieving CMBP’s overall objective of reducing threats to marine-coastal biodiversity? 3. What is the capacity (financial, managerial, programmatic/technical) of CMBP’s local implementing partners (i.e. sub-grantees)? A multi-person team under contract with Alter Modus Inc. (hereafter the “Evaluation Team”) was assembled to facilitate all elements of the performance evaluation and prepare this report. At the outset, the Evaluation Team developed a matrix which describes various data sources, methods and tools utilized to answer the evaluation questions. Application of the matrix was in three primary phases: planning and desktop data collection; field-based data collection; and analysis, presentation, and reporting. The Evaluation Team conducted interviews and focus group discussions (FGDs) with a range of CMBP partners and beneficiaries; utilized USAID’s Organizational Capacity Assessment Tool to evaluate the capacity each sub-grantee; and developed and applied a rubric to evaluate the quality of MPA management plans and action plans from each seascape. Findings, Conclusions and Recommendations The main findings of the CMBP performance evaluation include the following: Expected Result 4: Effective Management and Governance of Marine Protected Areas Performance indicators  Indicator 3 (new laws, policies and plans) has produced mixed results: exceeded in Jamaica and fully achieved in Haiti, but only 50% towards the performance target in the Grenadine Bank and Samana Bay.  Indicator 5 (hectares under improved management) is showing adequate progress, with full target achievement in Samana Bay, Haiti and Grenadine Bank, and 60% of target achieved in Jamaica.  Indicator 8 (number of person hours of training), which applies to both Expected Results 3 and 4, has exceeded its targets in all four seascapes. MPA management plans and action plans1  The action plans for coral and marine mammal conservation in Samana Bay are of sub-standard quality and require significant improvements.  The Protected Area of Managed Natural Resources of the Three Bays (PA3B) Management Plan is the most comprehensive and well written of all the plans reviewed. Haiti’s Agency for Protected Areas considers the management plan a good reference to structure the development of marine protection for other needed areas. 1 Caveat lector: please read important information in the Evaluation Limitations section regarding the interpretation of management plans evaluations results for Action Plans: “CMBP partners and contractors who developed or refined MPA management plans and actions plans did not have advance notice that the quality of these plans would be evaluated by the tool developed for this performance evaluation. Each plan evaluation is therefore acknowledged as a robust and useful desktop exercise, but one that lacks a deep analysis of the local context from which the plan grew.” 5  Good quality rating for the updated Pedro Cays and Surrounding Waters Management Plan, Jamaica, with some need for improvement. The process of management plan development for the South West Cay Special Fishery Conservation Area has begun.  Good interagency and stakeholder collaboration for Pedro Cays and Surrounding Waters Management Plan development process and advancing towards government approval Jamaica. However, mixed responses as to whether lead implementing agency is clear to all parties.  Good quality rating for adaptive mooring management plans in the Grenadine Bank, and implementation occurring in both Tobago Cay Marine Park (TCMP) and Sandy Island – Oyster Bed Marine Protected Area (SIOBMPA). MPA governance and management, including patrolling and outreach  Successful creation of six new no-take zones in Samana Bay, declared through a resolution of the Ministry of Environment.  Increased capacity for patrolling, outreach and education, monitoring and general management reported at South West Cay and Bluefields Bay in Jamaica. However, limited patrolling capacity still inhibits anti-poaching enforcement efforts at both sites.  Ongoing implementation of a mooring buoy user fee system at TCMP and SIOBMPA helps secure a sustainable revenue stream. Nevertheless, some stakeholders perceive too much focus on collection of user fees to the detriment of other priority management actions at TCMP. Management effectiveness tracking tool (METT) scores  In the Samana Bay MPA, only Bancos de la Plata y de la Navidad Marine Mammals Sanctuary has shown an increase in METT scores; all other MPA scores have largely stayed the same.  Substantial improvements seen in METT scores in the Grenadine Bank MPAs and modest improvements in the Protected Area of Managed Natural Resources of the Three Bays in Haiti.  METT performance scores have not changed substantially at the three Jamaica assessed Special Fishery Conservation Areas (SFCAs), South West Cay, Bluefields Bay and Galleon St. Elizabeth Fish Sanctuaries.  Most organizations are applying WWF’s 2007 version of the METT, or a modified version. It does not appear that organizations have consulted or are aware of the 2004 “marine” version of the METT developed by the World Bank. Expected Result 3: Sustainable Fisheries and Sustainable Livelihoods for Fishers Performance indicators  Performance indicator 6 (number of persons with tangible and/or perceived increases in economic benefits derived from improved resource management) has met its target in Haiti for Year 3. Results are mixed in other seascapes, with great results among Jamaican men (94% completion of the target), but almost null success among Grenadine Bank women (7% completion of the target).  Performance indicator 7 (number of fishers that have applied new technologies or management practices) has had poor results. Percentage achievement of this indicator’s targets was lowest in Samana Bay and Haiti (0% of the target completion for both men and women) with a slightly better result in the Grenadine Bank (32% completion of the target for men and 4% for women). Results are better in Jamaica, with 71% completion of the target for men and 40% for women. 6  Indicator 8 (number of person hours of training), which applies to both Expected Results 3 and 4, has exceeded its targets in all four seascapes. Fisheries sector action plans  Successful development of a Fisheries Strategic Action Plan for Pedro Bank and the Southwest Coast of Jamaica seascape provided guidance for capacity building of fish sanctuaries.  The Fisheries Sector Action Plan developed for Samana Bay was received positively by the Ministry of Environment and CODOPESCA.  Successful development of a Fishery Sector Action Plan for Grenadine Bank, which helped guide and inform program activities.  A complete Fisheries Sector Action Plan developed for the Three Bays, Haiti area. Capacity of fisher associations  Administrative capacity of fisher associations in Samana Bay has increased and will facilitate sustainable management of cooperative common funds.  In Jamaica the organizational and technical capacities of Bluefields Bay Fishermen’s Friendly Society (BBFFS) limit the type of financial support they can receive and administer to small, fixed-price grants. No fisher association for the Pedro Cays has been organized as yet.  Community leaders on some islands across the Grenadine Bank report that fisher associations are experiencing difficulties getting organized, and thus most associations are in an early stage of development. Building fisher associations into truly organized and active groups is challenging, particularly on Union Island and Mayreau.  Successful capacity building efforts were implemented by TNC and FOPROBIM for fisher associations in the Three Bays, Haiti, area with fishers organized, educated and motivated. Fisher support for sustainable management, fisher association capacity building and rising environmental awareness  Samana Bay fishers and Dominican government officials are optimistic the cooperation network created by CMBP will last far beyond the project timeline, resulting in tangible, long￾term benefits for ecosystems and local communities. CMBP actions have led to more formal participation of local organizations in the co-management of Bajo Yuna’s natural resources.  A wide range of trainings conducted in Jamaica were well received and generally had balanced participation of men and women. Nevertheless, outreach and education activities face the challenge of limited literacy among fishers and some wardens, and the high turnover of fishers in Pedro Cays.  In the Grenadine Bank increasing Sustainable Grenadine (SUSGREN) and MPA staff engagement has led to improving relationships with fishers, tour operators, and local communities, but key user groups, especially fishers in communities near both TCMP and SIOBMPA, show limited support and some poaching in no-take zones may still be occurring. Notably, leaders of fisher associations in Carriacou, Union Island and Mayreau are well aware of the need for fishers to embrace more sustainable practices. Fisher support for sustainable fishing and increased compliance due to better patrolling  Awareness of the need for more sustainable fishing practices has increased in Samana Bay and in Haiti; however, raising awareness has so far not proved to be enough to change the way local communities use marine resources. 7  HACCP training and equipment for cold storage of catches provided to over 50 fisherfolk in the Grenadine Bank. 70% of participants reported tangible economic benefits as a result of the activity, which responded to stakeholder needs and the Fisheries Sector Action Plan.  In Jamaica, trainings and innovative outreach events led by Caribbean Coastal Area Management Foundation (C-CAM) have helped educate Pedro Bank fishers on sustainability concepts, and provided tools to enable fishers to put concepts into practice; however, it is unclear what level of impact the training and outreach programs have had on getting fishers to adopt sustainable practices and reduce threats to key species and marine biodiversity. CMBP funding for staff and patrolling activities provided to C-CAM and BBFFS has led to increased patrolling and outreach efforts in the SFCA at South West Cay and Bluefields Bay.  Emerging partnerships between C-CAM and the Jamaica Defense Force (JDF) have helped advance programs in the remote Pedro Cays, and show promise for further capacity building, conservation action and lessons learned for the full network of 17 fish sanctuaries in Jamaica.  Safety-at-sea and product handling trainings were broadly received and appreciated by fishers in the Grenadine Bank. However, small boats owned by most fishers cannot fish in pelagic waters, thus it is yet unmeasured and unclear whether an effort shift in the fishery has occurred, and if this has reduced fishing pressure in nearshore waters. Some stakeholders noted unsustainable fishing practices are still common amongst Union Island fishers. Using USAID’s Organizational Capacity Assessment Tool (OCA), the evaluation of CMBP sub￾grantee organization capability found that CEBSE is at a Moderate Level of organizational capacity, and SUSGREN, C-CAM and FOPROBIM at a Strong Level of organizational capacity. The Evaluation Team concludes that all sub-grantees organizations have an adequate level of managerial, financial and programmatic capacity to support CMBP’s efforts to achieve expected results. Moreover, sub-grantees have the organizational capacity needed to sustain activities into the future. Each seascape section of this report describes findings, conclusions and recommendations for the seascape considered. These conclusions and recommendations provide strategic direction and specific action that USAID and CMBP partners may consider building on progress to date, address challenges, and inform programmatic priorities and focus under the remaining CMBP timeframe. Moreover, lessons learned may inform future planning, as needed, for CMBP or similar regional initiatives. While CMBP fell short on some of its original goals, important conservation work has taken place across the region which can and should be built upon. Overall performance evaluation conclusions include the following:  The original program goals were overly ambitious, yet important foundational work has taken place across the region.  CMBP has yet to fully evolve into a regional initiative that connects all seascapes.  Workshops, trainings and associated equipment distribution are considered one of the most valuable elements of the program to date.  MPAs are steadily building capacity, advancing management activities and increasing protections at key CMBP sites across the region.  Some MPA authorities and local sub-grantees are not using the METT correctly.  The impact of capacity building and training efforts targeted at fishers and fisher associations is limited at this stage. 8  CMBP has aligned well with national and local level conservation planning.  Local sub-grantee capacity is relatively strong; however, each organization has critical weaknesses it still needs to address. Recommendations include the following:  Continue supporting local partners and targeting persistent challenges.  Revisit and retrain local partners in management effectiveness evaluation concepts, tools and practices.  Learn from and build upon evaluation of current management plans and action plans and start/continue the implementation of the actions therein.  Continue to prioritize stakeholder engagement and effective communication.  Develop learning, information sharing and peer exchange networks.  Learn lessons from others in the Caribbean who are facing similar conservation challenges.  Consider strengthening cost-sharing among sub-grantees. There is no priority or logical sequence to the aforementioned recommendations. Each should be given equal consideration. Many can and should be integrated and thereby mutually reinforcing. If proactively implemented, it is expected that these recommendations will help CMBP partners to prioritize remaining activities, focus actions, and create conservation impact across each seascape that extends and evolves well into the future. Readers are encouraged to explore the detailed rationale, purpose and potential benefits associated with each recommendation in the final narrative section of this report. Finally, the Evaluation Team acknowledges the absence of interviews to the current and former CMBP Chiefs of Party (COP) and TNC’s grants manager. Nevertheless, the team conducted a thorough review all official project progress reports submitted by the COPs to USAID, and facilitated interviews with key TNC's field personnel in all four seascapes. Program Background Background Introductory note to reader: The narrative text in the following three sub-sections is intentionally drawn from original CMBP documents to provide proper context for this report:  Background.  Description of Problem, Development Hypothesis, and Theory of Change.  Summary of Evaluated Activity. Tourism is the major economic driver across the Caribbean region, with 25 million annual visitors and one out of six jobs linked to the tourism sector. The fisheries sector is another 9 important economic driver, and a critical provider of food security. A conservative estimate would be that more than 1.5 million people in the Caribbean rely on fishing for their livelihoods and food security.2 That said, Caribbean marine and coastal resources face growing threats, such as overfishing, unsustainable coastal tourism, and climate change:  75% of the region's coral reefs are categorized as degraded or threatened. 3  Approximately, 60% of living coral in the Caribbean has been lost in the past 30 years.4  All the major commercially important fish species and groups of species in the region (such as lobster, conch, and grouper) face intense fishing pressures, with some already considered to be overexploited.  Growing, unsustainable coastal tourism has resulted in a dramatic loss of mangroves. Links to “America First” The Caribbean Marine Biodiversity Program promotes the “America First” philosophy of the Administration by promoting the security, safety and economic interests of the United States government as well as core American identity values. It is important that neighboring countries have strong economies and employment opportunities. By protecting and conserving marine and coastal resources and strengthening the management of marine protected areas USAID is also promoting sustainable fishing practices. A thriving and sustainable fishing sector will allow for increased economic opportunities for coastal community members and reduce illegal immigration to the United States. CMBP has also contributed to increased maritime security in the Caribbean which leads to increased security along the coastlines of the United States. The Caribbean Sea is commonly used by traffickers of many illicit products which often end up in the United States. CMBP has worked to further develop marine spatial plans, and, as this report demonstrates, strengthen the management of marine management areas, hire additional marine park wardens and procure necessary equipment to increase enforcement capabilities, as well as create effort shift in the fishing industry away from marine managed areas. Furthermore, the increased management measures and the presence of marine park wardens with the necessary supplies, including boats and communications devices, also deter the flow of narcotics and other illegal substances being transported into the United States. Lastly, many U.S.-based companies have businesses which depend on the rich natural resources of the Caribbean region, including American-owned hotels, restaurants, cruises and tour operators. Many U.S. companies import seafood from the Caribbean, including Caribbean spiny lobster and conch, which depend on healthy and resilient marine habitats. Links to Regional Strategy and Bilateral Mission Strategies This activity, while not tied to any one mission’s or operating unit’s Country/Regional Development Cooperation Strategy (CDCS/RDCS), was designed with the Regional 2 CARSEA (2007). Caribbean Sea Ecosystem Assessment (CARSEA). A sub-global component of the Millennium Ecosystem Assessment (MA). J. Agard, A. Cropper, K. Garcia Eds. Caribbean Marine Studies, Vol. 8:1-85 3 Burke, L. (2011). Reefs at Risk: Revisited. World Resources Institute, 11. Retrieved March 3, 2017, from https://www.wri.org/sites/default/files/pdf/reefs_at_risk_revisited_executive_summary.pdf. 4 Burke, L. (2011). Reefs at Risk: Revisited. World Resources Institute, 16. Retrieved March 3, 2017, from https://www.wri.org/sites/default/files/pdf/reefs_at_risk_revisited_executive_summary.pdf. 10 Development Cooperation Framework, and Caribbean missions’ (including Dominican Republic, Haiti, Jamaica, and Barbados/Eastern and Southern Caribbean (ESC), in mind. With the exception of the LAC/RSD Regional Development Cooperation Framework, these strategies do not directly address biodiversity conservation, but instead focus environmental investments on global climate change. While global climate change is not a main focus of this activity, the development and strengthening of MPAs will help protect the reefs and the species they house to build resilience to recover more effectively and rapidly from external stressors. Research has shown that improved fisheries management can increase coral reef resilience to climate impacts; this would then strengthen resilience of coastal communities. The relevant regional and bilateral strategies with which the activity will interact are described below.  LAC/RSD’s Regional Development Cooperation Framework recognizes the importance of both global climate change and biodiversity conservation in LAC Missions’ work and notes, “Climate change poses a serious threat to prosperity and poverty reduction, yet it also creates opportunities for innovation and for investments that will deliver long￾lasting environmental and economic benefits. Biodiversity loss and natural resource degradation also threaten to undermine progress towards sustained economic prosperity and democratic stability. In recognition of the importance of these issues, LAC governments are increasingly calling for proven tools and methodologies to help them respond to climate change and address threats to biodiversity.” Biodiversity work falls under Development Goal 4: “Transition to climate-resilient, low-emission, sustainable economic growth accelerated.”  USAID/Dominican Republic’s CDCS, which covers FY 2014-FY 2018, identifies the need to align land use planning with the realities of climate change and, as such, will work towards achievement of the Development Objective: “Increased resilience of people to the impact of climate change.” Noting that proper land use planning can protect vulnerable populations and coastal communities from the adverse effects of climate change, USAID/Dominican Republic’s strategy will work to increase the capacity of select municipalities to develop and implement participatory land use plans.  The RDCS for Barbados/Eastern and Southern Caribbean recognizes important linkages between climate change and vulnerable populations, and focuses programming on strengthening the application of climate science for decision-making.  In Jamaica, USAID supports increased institutional capacity to mitigate emissions and manage impacts of climate change, and t o help targeted populations withstand and respond to negative impacts of climate change.  In Haiti, USAID works with United States government partners to better manage natural resources, including biodiversity, enhance food security and to promote technologies that increase the country’s capacity to adapt to climate change. Links to Global Strategies Other applicable and relevant strategies and guides that informed the activity design include:  USAID Biodiversity Policy: The policy identifies two mutually reinforcing goals: (1) conserve biodiversity in priority places, and, (2) integrate biodiversity conservation as an essential component of human development. 11  USAID Biodiversity Code: In order to ensure that the Agency meets the intentions of legislation in support of biodiversity conservation, USAID has a “Biodiversity Code” that defines four criteria required of programs which use funds designated for biodiversity. The four criteria are: (1) the program must have an explicit biodiversity objective; it isn’t enough to have biodiversity conservation result as a positive externality from another program; (2) activities must be identified based on an analysis of drivers and threats to biodiversity and a corresponding theory of change; (3) site-based programs must have the intent to positively impact biodiversity in biologically significant areas; and, (4) the program must monitor indicators associated with a stated theory of change for biodiversity conservation results.  USAID Gender Equality and Female Empowerment Policy: The Agency recognizes the important role gender plays in core development objectives and its relation to sustainable development outcomes; this policy informs all activities under design.  Sustainable Fisheries and Responsible Aquaculture: A Guide for USAID Staff and Partners-June, 2013. The Guide’s recommended actions will serve as Best Management Practices.  National Oceanic and Atmospheric Administration’s (NOAA) Coral Reef Conservation Program International Strategy: 2010-2015; which identifies the Caribbean as a priority region, specifically the Strategy’s Priority Goal 1: Work with regional initiatives to build MPA networks and strengthen local management capacity to improve and maintain resilience of coral reef ecosystems and the human communities that depend on them and Priority Goal 2: Strengthen local and national capacity and policy frameworks to reduce impacts of fishing on coral reef ecosystems. Description of the Problem, Development Hypothesis, and Theory of Change Problem Statement The purpose of this activity is to reduce threats to marine-coastal biodiversity in priority areas in the Caribbean, which will include high biodiversity ecosystems such as coral reefs, mangroves, and seagrass meadows. The main threats to marine and coastal biodiversity in the Caribbean are coastal and urban development, unsustainable fishing practices, pollution, unsustainable tourism/recreation practices and invasive species. This activity contributes to achievement of the purpose via two sub-purposes: 1) Targeted marine protected areas and associated coastal/fisheries resources more sustainably managed, and 2) Enabling conditions enhanced for sustained biodiversity conservation across a network of marine protected areas. USAID has worked to increase the sustainability and resilience of marine resources and coastal communities in the Caribbean by helping to strengthen existing MPAs, enhance regional coordination among MPAs, and build capacity for MPA and fisheries management in the region. By simultaneously broadening the constituency and strengthening the capacity of MPA stakeholders and increasing the effectiveness of MPAs as a tool for conservation, USAID has sought to improve marine biodiversity and benefit marine-resource- dependent communities. Theory of Change 12 In summary, the main CMBP goal is to increase the sustainability and resilience of the marine resources and communities where program activities are being implemented. The original purpose stated is similar to that of the Caribbean Challenge Initiative (CCI),5 (Need to note the national conservation trust funds linked to the CCI have been slow to come online across the region, but still show promise and are very much needed; may help further advance work that has occurred under this initiative.) to effectively conserve and manage at least 20 percent of the marine and coastal environment. The activities envisioned targeted strengthening managed marine areas (MMAs) and developing new MMAs through a number of different mechanisms. This includes strengthening legal frameworks, filling any information gaps and developing management plans, where needed. Additionally, activities were designed to address updating national fisheries policy, and establishing a sustainable financing mechanism, including leveraging additional public and private funding, all of which lead to strong management and enforcement capability among a network of MMAs across the Caribbean. This goal is measured through a number of different criteria, including human well-being factors, biodiversity richness and the health of marine and coastal habitats including their ability to provide essential ecosystem services. It is important to note that the Theory of Change (TOC) presented below was theorized and developed during the original design of CMBP. As TNC began to implement and design activities in annual work plans some of components of the original TOC shifted due the need for adaptive management. Specifically, components of the TOC related to climate change adaptation and the control of invasive species have not been strong areas of focus during activity implementation. Figure 1: CMBP Theory of Change 5 http://www.caribbeanchallengeinitiative.org/ 13 Results Framework According to the cooperative agreement between USAID and TNC, the overall objective of CMBP is “to reduce threats to marine-coastal biodiversity in priority areas in the Caribbean— including high biodiversity ecosystems such as coral reefs, mangroves, and seagrass beds—in order to achieve sustained biodiversity conservation, maintain critical ecosystem services, and realize tangible improvements in human wellbeing for communities adjacent to marine protected areas.” To achieve this objective, CMBP is structured around four inter-related Expected Results, as shown in figure 2. Figure 2: CMBP Results Framework 14 The CMBP Results Framework points to a more sustainable fisheries sector (Expected Result 3) and effective MMA/MPA management (Expected Result 4) as key elements for reducing threats to marine-coastal biodiversity in key Caribbean areas. Expected Result 3 is shown to be accomplished by: (1) developing fishery sector action plans, (2) strengthening the capacity of local fisher associations, (3) implementing pilots for sustainable fishing practices and sustainable livelihoods, (4) increasing coverage and support for fish sanctuaries (aka no-take zones), and (5) adopting and implementing supportive fishery policies. Likewise, Expected Result 4 is shown to be accomplished by: (1) establishing MPA/MMA management plans, (2) operationalizing governance mechanisms, and (3) putting in place management infrastructure and implementing management actions. 15 Summary of Evaluated Activity CMBP was designed out of USAID\Washington's Latin America and the Caribbean Bureau’s Regional Sustainable Development Team in consultation with the four USAID Mission’s in the region and technical staff from the Bureau for Economic Growth, Education, and Environment (E3). In September 2014, TNC was awarded CMBP cooperative agreement (No. AID-OAA￾A14-00064). At that time, management responsibilities remained with USAID/Washington. At the outset of 2015 the Caribbean Development Program (CDP) Operating Unit (OU) was officially created, housed in the USAID\Dominican Republic Mission and the CMBP became the first activity under the CDP OU. Management of the cooperative agreement was then transferred to the USAID/DR Mission, from where it continues to be managed. Through this award TNC carries out USAID-funded ocean and coastal activities across five countries in the region including Haiti, Dominican Republic, Jamaica, Grenada and St. Vincent and the Grenadines. As discussed above, the main objective for this regional activity is to reduce threats to marine and coastal biodiversity in priority areas in the Caribbean – including high biodiversity areas such as coral reefs, mangroves, and seagrass meadows – in order to achieve sustained biodiversity conservation, maintain critical ecosystem services, and realize tangible improvements in human wellbeing for communities adjacent to marine protected areas. To achieve this result an emphasis is placed on creating an enabling environment for long-term marine protected area success (sustainable conservation financing), developing effective marine spatial plans and seascape governance mechanisms, establishing a more sustainable fisheries sector (strengthening the capacity of local NGOs, conducting fisheries sector analysis, supporting local fisher associations, and sustainable fisheries pilot programs) and promoting effective management and governance of MPAs across the region. Evaluation Purpose, Methods and Limitations The purpose of this mid-term performance evaluation is to provide USAID, its implementing partner TNC, and other regional stakeholders with robust and independent findings, conclusions, and recommendations regarding CMBP performance since inception. Recipients of the final performance evaluation report will include:  USAID missions involved in managing CMBP (i.e., Dominican Republic, Jamaica, Haiti, and Eastern and Southern Caribbean)  USAID/Washington bureaus and offices, including the Bureau for Latin America and the Caribbean (LAC) and the Forestry and Biodiversity (FAB) Office in the Bureau for Economic Growth, Education, and Environment (E3)  Implementing partner (The Nature Conservancy)  Other stakeholders (e.g., national and regional government ministries and offices) USAID and TNC are expected to use this report, as appropriate, to make needed mid-course adjustments for the remaining life of the activity, highlight successes to date, and bring to bear evidence for design and implementation of similar and/or future regional marine biodiversity conservation activities in the Caribbean and other regions. USAID also expects that the findings, conclusions, and recommendations of the performance evaluation will contribute to the design of any final CMBP performance evaluation. 16 USAID and TNC jointly decided to focus this performance evaluation on two of the activity’s Expected Results:  Expected Result 3: “Sustainable Fisheries and Sustainable Livelihoods for Fishers,” and  Expected Result 4: “Effective management and governance of Marine Protected Areas and/or Marine Managed Areas (MPAs/MMAs).” The Evaluation Team’s investigations were guided by a set of overarching questions and associated sub-questions designed to assess the degree to which CMBP has achieved Expected Results 3 and 4. Moreover, the performance evaluation considered what implications progress to date, or lack thereof, has on the overall CMBP objective of reducing threats to marine biodiversity in the four priority Caribbean seascapes.  Jamaica (Pedro Bank and Southwest Coast Seascape)  Haiti (North coast of Hispaniola Seascape)  The Dominican Republic (Samana Bay Seascape)  St. Vincent and the Grenadines and Grenada (Grenadine Bank Seascape) Specifically, the CMBP mid-term performance investigated the following evaluation questions: 1. To what extent has CMBP met its targets under Expected Result 4: Effective management and governance of MPA/MMAs? What implications does CMBP performance on Expected Result 4 have for achieving CMBP’s overall objective of reducing threats to marine-coastal biodiversity? a. What is the status (e.g., draft, approved, implemented, etc.) of existing and/or new MPA/MMA management plans in the four priority seascapes? What is the level of quality of these MPA/MMA management plans? How are the MPA/MMA management plans being used, and by whom? b. In the four priority seascapes, which MPAs/MMAs are using the Management Effectiveness Tracking Tool (METT) or similar tools? How are the results of the METT being used? What data or evidence does the METT provide regarding changes in MPA/MMA management effectiveness over time? Finally, using this tool, has MMA management effectiveness changed (i.e., increased, decreased, or stayed the same) since CMBP’s inception? What factors or conditions (e.g., enabling environment, presence or absence of financing, local capacity, etc.) help explain why or how MPA/MMA management effectiveness has changed? c. What types of management infrastructure has CMBP provided? What plans has CMBP put in place to ensure that the infrastructure provided will continue to be used beyond the life of CMBP? 2. To what extent has CMBP met its targets under Expected Result 3: More sustainable fisheries sector? What implications does CMBP performance on Expected Result 3 have for achieving CMBP’s overall objective of reducing threats to marine-coastal biodiversity? a. What do fishers and relevant stakeholders, such as local authorities and local fisher associations, identify as sustainable or unsustainable fishing practices? How is CMBP strengthening and measuring changes in local fisher associations’ capacity over time? b. How is CMBP measuring changes in marine biodiversity and human well-being as a result of these pilots? Based on this evidence, what are the prospects for successful expansion? 17 c. What are CMBP, local authorities, and/or fishery associations doing to reinforce appropriate fishing practices and discourage inappropriate practices in the area? What are the obstacles and/or gaps? d. To what extent are governments scaling up pilot activities implemented by CMBP that promote sustainable fishing practices? 3. What is the capacity (financial, managerial, programmatic/technical) of CMBP’s local implementing partners (i.e. sub-grantees)? a. What implications does sub-grantees’ overall capacity have on CMBP’s ability to achieve expected results? b. What implications does sub-grantees’ capacity have for the sustainability of the activities implemented and results achieved by these organizations? At the outset of the consultancy, the Evaluation Team developed a matrix that describes the various data sources, methods and tools utilized to answer the aforementioned evaluation questions (Annex I). Application of this matrix came in three primary phases: (1) planning and desktop data collection; (2) field-based data collection; and (3) analysis, presentation, and reporting (Figure 3). Figure 3: Phases of the Performance Evaluation Process Following completion of field work, the Evaluation Team organized, synthesized and analyzed data by seascape and from a regional perspective. Data from all sources was then compared to verify initial findings and highlight areas requiring further investigation, thereby improving the quality and reliability of conclusions and recommendations presented in this report. A. Desktop Document Reviews USAID, TNC and other regional stakeholders assisted the Evaluation Team in the initial collection and organization of key program documents. Early document review included, but was not limited to, project solicitation documents and TNC’s original proposal; various TNC reports (e.g. annual progress; monitoring and evaluation); and key program outputs (e.g. Fishery Action Plans, MPA/MMA management plans, action plans and METT results from across the region). A few additional documents were collected in the field and were integrated into this desktop data set. Initial review of program documents helped orient the Evaluation Team to CMBP goals, expected results, indicators and targets. Moreover, review of program documents and outputs provided, in some cases, partial answers to evaluation questions which were later explored with program partners and stakeholders in the field. Finally, early desktop work helped the Evaluation Team gain insight on CMBP progress and challenges to date, finalize work planning efforts, and prepare for field-based research. (Annex IV includes a full list of documents consulted for this performance evaluation.) Planning and desktop data collection Field-based data collection Analysis, presentation, and reporting 18 B. Field-based Interviews and Focus Group Discussions A multi-person Evaluation Team was assembled to facilitate all elements of the performance evaluation and prepare this report. Most team members also conducted field-based interviews across all seascapes. The full team consisted of the following individuals:  J. Robert Simmons, overall contract management and sub-grantee capacity assessments  Alejandro Arrivillaga, project team lead and overall project management  Rich Wilson, MPA management plan/action plan evaluation design and oversight, and field-based research (Grenadine Bank and Jamaica)  Rudo Udika, field-based research (Grenadine Bank and Jamaica)  Katarzyna Gracela, field-based research (Dominican Republic)  Paul Fedner Zamy, field-based research (Haiti)  Ian Drysdale, management plan evaluation (tool development and plan evaluations) The Evaluation Team prepared tailored questionnaires for four key interview groups (Annex V) including Government agencies/officials, Sub-grantees/MPA managers, Fishers and relevant stakeholders, and Program implementer (TNC) At the beginning of the field work the Evaluation Team met with USAID staff at the Dominican Republic Mission to ensure alignment of USAID’s interests with the proposed approach to conducting interviews across the region and address any outstanding questions. The meeting helped clarify USAID’s particular interest in expected results 3 and 4, and to further stress the important of linking outcomes to date to the long-term potential for conserving marine biodiversity in each seascape. As a last step to prepare for field-based interviews, a questionnaire guide was developed and closely reviewed in the Dominican Republic with all team members present. All team members participated in a series of interviews and focus group discussions (FGDs) in Samana Bay as a means to standardize the methods before splitting into sub teams. The guide was designed to ensure efficient and effective data collection, consistency of approach, and high-quality interviews across the region. Specifically, it provided evaluators with guidance on how to: use a common set of opening talking points; generate candid interviewee responses; accurately and comprehensively collect notes; properly save data; and summarize notes following the interview. (See Annex V for the interview questionnaire guide.) Finally, it is important to note that TNC staff assisted the Evaluation Team across the region to both set up interviews and help with logistical arrangements. In some cases, such as in the Dominican Republic and Grenadine Bank, staff from sub-grantee organizations provided additional on-site assistance and helped secure meetings with important stakeholder groups. (See Annex IV for a full list of interviewees by seascape) C. Evaluation of Marine Protected Area Management Plans and Action Plans In addition to field-based interviews, the Evaluation Team conducted a desktop analysis of submitted management plans and action plans from each CMBP seascape. Linked to this effort, the team reviewed annual METT scores from the 2015 – 2017 for priority MPA/MMA sites in the 19 four seascapes. To assess the quality or new or existing MPA/MMA management plans, the team developed a rubric or assessment tool that included criteria for global climate change adaptation or risk mitigation plans. The adapted tool was subsequently reviewed and approved by USAID. A three-step process guided the quality assessment of MPA/MMA management plans and action plans developed or revised by local CMBP partners. 1. Literature review. Identified leading publications that describe recognized standards and processes for developing protected areas management plans and implementing management actions; 2. Prioritization exercise and development of evaluation tool. Narrowed the list of supporting resources and developed a comprehensive plan evaluation tool; 3. Plan evaluation. Applied the tool in MS Excel, generated rankings by management category and an overall score for each plan, and integrated three years of METT scores. After a comprehensive literature review, the Evaluation Team selected the below publications for additional review and consideration in developing the plan evaluation tool:  Lee Thomas, and Julie Middleton (2003). Guidelines for Management Planning of Protected Areas. IUCN Gland, Switzerland and Cambridge, UK. ix + 79pp. (Link here)  Salm, Rodney V., John R. Clark, and Erkki Siirila. Marine and coastal protected areas: a guide for planners and managers. IUCN, 2000. (Link here)  National Oceanic and Atmospheric Administration (date unknown). NOAA MPA Management Planning Guidelines: Planning Steps for Developing Effective Management Plans. Part 1 4.1: Overview of Management Planning (Link here)  Mesoamerican Barrier Reef Systems Project (2003). Training Manual on Design and Development of Management Plans for Marine Protected Areas. (Link here)  Ervin, Jamison (2003). WWF: Rapid Assessment and Prioritization of Protected Area Management (RAPPAM) Methodology. (Link here) The Evaluation Team then conducted a ranking exercise to identify one document that would serve as the primary resource for developing the plan evaluation tool. The exercise involved development and review of nearly 40 primary and secondary management planning topics, followed by a three-step process to determine which publication provided the most comprehensive and germane set of information for the purposes of this plan evaluation. Ultimately, the team selected the publication by Lee and Middleton, Guidelines for Management Planning of Protected Areas as the primary source for the tool. Once the first draft of the tool was developed, each of the other four publications was again reviewed to help identify gaps and ensure development of a comprehensive plan evaluation tool. The team then set about to construct and apply the tool. Six priority management topics and 36 associated sub-categories (indicators) were established by which each plan would be evaluated. Subsequently, a range of evaluation criteria and a high/medium/low ranking scheme was developed for each indicator. An additional column was created so the evaluator could identify and describe supporting evidence that explains rankings across each indicator. Each of the six submitted plans were carefully analyzed, with an emphasis on identifying how well the plan addressed the 36 indicators. Plans received a 0 - 3 numeric ranking for every assessed 20 indicator. For some plans, namely the action plans, a small selection of indicators was determined as not relevant. As such, the possible total score varies by plan, even though the same evaluation tool was used for every plan that TNC submitted for review. The final version of the plan evaluation tool is presented in Annex II. Based on overall score achieved, each plan received one of the following quality ratings:  Excellent quality plan: >80%  Good quality plan: 60% - 80%  Average quality plan: 40% - 60%  Sub-standard quality plan: <40% D. Assessment of Sub-grantee Organizational Capacity In order to gain insight on the current capacity and future leadership potential of CMBP sub￾grantee, the Evaluation Team visited each organization’s Caribbean offices and applied USAID’s Organizational Capacity Assessment Tool (OCA). The OCA helps organizations conduct a self￾assessment of key areas of organizational capacity:  Governance and legal structure  Fiscal management and internal control systems  Administration and procurement systems  Human resources systems  Program management  Project performance management  Organizational management and sustainability An assessment of the organizational capacity of each CMBP sub-grantee was conducted using USAID’s Organizational Capacity Assessment (Facilitator’s Guide Version Without NUPAS Items tool (Link here). The organizational capacity assessment explored the following overarching question and associated sub-questions:  Overarching question: What is the capacity (financial, managerial, and programmatic/technical) of CMBP’s local implementing partners (i.e., sub-grantees)? o Sub-question: What implications does sub-grantees’ overall capacity have on CMBP’s ability to achieve expected results? o Sub-question: What implications does sub-grantees’ capacity have for the sustainability of the activities implemented and results achieved by these organizations? At each sub-grantee meeting, the Evaluation Team helped participants understand the objective of the assessment and how to use the OCA tool, then assisted each in conducting a self-assessment of their respective capacity. Each area of organizational capacity described above included a sub￾set of questions. The Evaluation Team helped participants explore the questions, identify accurate responses, and then convert responses using the following scoring system: Figure 4. OCA Scoring System Capacity Level Score Low Capacity 1 21 Basic Capacity 2 Moderate Capacity 3 Strong Capacity 4 Results of the organizational capacity assessment for each sub-grantee are described following the seascape sections below. These results were compared to responses provided during field-based interviews that included other stakeholders in order to gain an overall sense of each sub-grantees capacity, and what this means for CMBP’s present and future potential to effectively build MPA/MMA management capacity, foster sustainable fishing practices and conserve marine biodiversity in priority areas across the Caribbean. E. Evaluation Limitations This report acknowledges certain limitations of the methodology used to conduct the CMBP performance evaluation and prepare this report.  Scheduling and meeting constraints. TNC and sometimes staff from sub-grantees helped ensure the Evaluation Team met with a comprehensive set of government officials and other stakeholder groups in each seascape. Nevertheless, many parties, especially high￾level ministerial and fisheries department officials, often had limited time available, thus allowing for exploration of only a few key questions. In Haiti, the Evaluation Team was unable to meet with the Fisheries Authority despite several attempts. In some counties, such St. Vincent and the Grenadines and Grenada, meetings took place with individual leaders of fisherfolk associations rather than with groups of fishers.  Language barriers. The region evaluated is home to three languages (English, Spanish and French), with many also speaking local creole dialects. During some interviews it was difficult for all Evaluation Team members to understand interviewees, especially fishers. Moreover, at times it also proved difficult to translate interview responses from Spanish or French into English without losing key concepts and sentiments conveyed.  Limited validity of evaluated MPA management plans and action plans. CMBP partners and contractors who developed or refined MPA management plans and actions plans did not have advance notice that the quality of these plans would be evaluated by the tool developed for this performance evaluation. Each plan evaluation is therefore acknowledged as a robust and useful desktop exercise, but one that lacks a deep analysis of the local context from which the plan grew.  Break up of evaluation team field research. Due to time and budget constraints, no Evaluation Team member was able to visit all four seascapes, thus limiting how well any one member analyze the program from a regional perspective.  Time limits on field-based research. Although focused and comprehensive in its approach, all field-based research was conducted in a rapid manner, thus limiting depth of analysis and validation of core activities such as patrolling and monitoring in MPAs or shadowing fishers to better understand their fishing practices. Furthermore, the evaluation team acknowledges the lack of an interviews to the current and former CMBP Chiefs of Party (COP) and TNC’s grants manager. Nevertheless, the team conducted a thorough review all official project progress reports submitted by the COPs to USAID, and facilitated interviews with key TNC's field personnel in all four seascapes. 22 Findings, Conclusions and Recommendations by Seascape Expected Result 4: Effective Management and Governance of MPAs To what extent has CMBP met its targets under Expected Result 4? What implications does CMBP performance on Expected Result 4 have for achieving CMBPs overall objectives of reducing threats to marine biodiversity? Samana Bay Seascape (Dominican Republic) Summary Analysis of Findings The Evaluation Team concludes that CMBP has almost fully achieved expected result 4, effective management and governance of MPA/MMAs in the Samana Bay seascape. Management infrastructure is partially in place, but some items are still pending. Key achievements in the Samana Bay seascape include:  CMBP activities facilitated the creation of six new no-take zones in Samana Bay, declared through a resolution of the Ministry of Environment.  METT scores for the Bancos de la Plata y de la Navidad Marine Mammals Sanctuary changes have increased.  CMBP Indicator 5 (Number of hectares of biological significance and/or natural resources under improved natural resource management as a result of USG assistance), and indicator 8 (Number of person hours of training in natural resources management and/or biodiversity conservation supported by USG assistance), associated with Expected Result 4, have met or exceeded the targets.  CMBP implementation has led to more formal participation of local organizations in the co-management of Bajo Yuna’s natural resources.  A cooperative MPA governance mechanism enabled a concession agreement for ecotourism activities in Cayo la Farola, which will be conducted by local stakeholders.  Fishers and government officials perceive that the cooperation network created by CMBP will last far beyond the project timeline, resulting in tangible, long-term benefits for ecosystems and local communities. Key challenges in the Samana Bay seascape include:  Action plans for coral and marine mammal’s conservation are of sub-standard quality and require significant improvements.  The only Samana Bay MPA that has showed an increase in METT scores (management effectiveness) is Bancos de la Plata y de la Navidad Marine Mammals Sanctuary.  CMBP performance indicator 3 (Number of new laws, policies, strategies, plans, or regulations addressing biodiversity conservation officially proposed or adopted as a result of USG assistance, has reached half of the expected results. Observations and recommendations listed at the end of this section reflect improvement opportunities and potential future actions that may enhance CMBP performance in Samana Bay under Expected Result 4. 23 Evaluation of action plans prepared under CMBP in Samana Bay Three protected areas are found in the Samana Bay Seascape: Los Haitises National Park, Bancos de la Plata y de la Navidad Marine Mammals Sanctuary (MMSBPN), and the Bajo Yuna Mangroves National Park. Since both Los Haitises and MMSBPN already had recently approved management plans (2013 and 2015, respectively), there were no new management plans developed for these areas under CMBP. Nevertheless, other documents that guide management of natural resources in Samana Bay were developed, including the Fisheries Sector Action Plan (FSAP) for Samana Bay, the Action Plan for the Conservation of Corals and Marine Mammals in Samana Bay and marine mammals in the Estero Hondo Marine Mammals Sanctuary and Santuario Marino del Norte. These action plans were drafted for discussion and proposed for approval to the Ministry of Environment in FY 2016. The plans were well received but will require both funding and political support to be implemented. Even though Manglares del Bajo Yuna National Park does not have a management plan yet, CMBP did not include plan development for this protected area among its activities; instead CEBSE and TNC are guiding conservation actions using TNC’s Conservation Action Planning methodology, developed prior to CMBP. (Annex VI includes a comprehensive list of scores for all plans reviewed under this CMBP performance evaluation.) The Evaluation Team conducted a formal desktop review and evaluation of three documents designed to help guide marine resource management in the Samana Bay seascape:  Action Plan for the Protection of Corals in Samana Bay  Action Plan for the Protection of Marine Mammals in Samana Bay  Action Plan for the Estero Hondo Marine Mammal Sanctuary It is important to note that these documents were crafted as action plans and not as traditional resource management plans. A summary of evaluation results for each plan is included in tables 1, 2 and 3. Similarities exist among each of these action plans. For example, each lays out a clear set of strategic objectives, and addresses some priority management topics well, as described below. Nevertheless, each plan still scores low in many areas across the broad range of management categories and sub-categories assessed during the desktop evaluation, and thus each requires significant improvement. The Action Plan for the Protection of Corals in Samana Bay received an overall 37% rating (sub￾standard quality plan). Some strengths in this plan include the description of the legal context in which the plan operates, the education program, and the analysis of baseline biophysical information and description of monitoring requirements. In all other areas the plan scored only moderately well or received a low evaluation score. The Action Plan for the Protection of Marine Mammals in Samana Bay received an overall 34% rating (sub-standard quality plan). Strengths in this plan include a robust threats analysis, the education program, the presentation and analysis of baseline biophysical data, and maps, tables and graphics to illustrate where actions are needed. In all other areas the plan scored only moderately well or received a low score. The Action Plan for the Estero Hondo Marine Mammal Sanctuary received an overall 38% rating (sub-standard quality plan). Strengths in this plan, like the other marine mammal plan, include a 24 robust threats analysis, presentation and analysis of baseline biophysical data, and maps, tables and graphics to illustrate where actions are needed. In all other areas the plan scored only moderately well or received a low score. Table 1. Evaluation of the Action Plan for the Protection of Corals in Samana Bay. Management Category Possible Score Plan Score % Management goals and objectives 6 5 83 Legal and institutional framework 12 3 25 Management planning and activities 30 9 30 Stakeholder analysis and engagement 15 3 20 Monitoring, evaluation and adaptive management 15 8 53 Style/presentation of content 6 3 50 Overall Score/Percentage 84 31 37 Overall Plan Rating: Sub-standard quality plan that requires significant improvements Table 2. Evaluation of the Action Plan for the Protection of Marine Mammals in Samana Bay. Management Category Possible Score Plan Score % Management goals and objectives 6 5 83 Legal and institutional framework 12 5 42 Management planning and activities 36 8 22 Stakeholder analysis and engagement 15 6 40 Monitoring, evaluation and adaptive management 15 3 20 Style/presentation of content 6 4 67 Overall Score/Percentage 90 31 34 Overall Plan Rating: Sub-standard quality plan that requires significant improvements Table 3. Evaluation of the Action Plan for the Estero Hondo Marine Mammal Sanctuary in Samana Bay. Management Category Possible Score Plan Score % Management goals and objectives 6 5 83 Legal and institutional framework 12 7 58 Management planning and activities 36 7 19 Stakeholder analysis and engagement 15 6 40 Monitoring, evaluation and adaptive management 15 5 33 Style/presentation of content 6 4 67 Overall Score/Percentage 90 34 38 Overall Plan Rating: Sub-standard quality plan that requires significant improvements 25 Tracking management effectiveness CMBP through TNC helped the Ministry of Environment and Natural Resources (MARENA) complete METT evaluations in 2015, 2016, 2017, using a slightly modified version of the WWF 2007 METT scorecard. A summary of METT scores for Los Haitises National Park, Bajo Yuna Mangroves National Park, and Bancos de la Plata y de la Navidad Marine Mammals Sanctuary are included in Tables 4, 5 and 6. METT score changes reveal that the only protected area that has showed an increase its total score is Bancos de la Plata y de la Navidad Marine Mammals Sanctuary. The total possible score for each site is 102 points. Table 4. Results of the application of the Management Effectiveness Tracking Tool (METT) to Los Haitises National Park, for years 2015 – 2017, based on self-evaluation, and using the terrestrial version of METT Scorecard. Year 2015 2016 2017 Total Possible Score 102 102 102 Total Score Received 61 35 59 Table 5. Results of the application of the Management Effectiveness Tracking Tool (METT) to Bajo Yuna Mangroves National Park for years 2015 – 2017, based on self-evaluation, and using the terrestrial version of METT Scorecard. Year 2015 2016 2017 Total Possible Score 102 102 102 Total Score Received 35 35 35 Table 6. Results of the application of the Management Effectiveness Tracking Tool (METT) to Bancos de la Plata y de la Navidad Marine Mammals Sanctuary, for years 2015 – 2017, based on self-evaluation, and using the terrestrial version of METT Scorecard. Year 2015 2016 2017 Total Possible Score 102 102 102 Total Score Received 41 33 62 Most of the management improvements are attributed to actions under the CMBP. For example, within the management effectiveness context section, improvements were documented in the enforcement of whale watching regulations and in the capacities of local personnel. Consistent improvements can also be seen in both the planning and management sections, where management and annual workplans are being implemented. Improvements have also been realized in protection mechanisms; research program implementation; as well as an increase in the local communities’ awareness, particularly for businesses that depend on protected area resources. According to field￾based interviews and facilitated focus groups, improvement of management effectiveness has been due in part to factors such as CMBP financed capacity building for park administrators and rangers, and changes implemented by the Ministry of Environment in response to METT results and CMBP recommendations. According to interviewees, the three MPAs in the Samana seascape have been using the METT every year since 2015. The total possible score for the METT scorecard applied is 102 points; 26 however, in 2017 there was a mistake in the calculation of the total possible score, which is incorrectly listed in the report to be 145. Even though MARENA, as the responsible agency for managing protected areas, uses the METT results to adjust annual MPA workplans, stakeholders agreed that MARENA’s implementing capacity is limited. MARENA understands that METT is the right tool to use and that MPA managers need to participate in METT workshops. The sub￾grantee was able to give details on the use of METT results, indicating that, although scores do not reflect improvement, there is enhanced management in Los Haitises and in Manglares del Bajo Yuna National Parks. Moreover, MARENA said they are using the results of the METT to update the Manglares del Bajo Yuna National Park information required by Ramsar Convention. Proposals have been submitted to Fondo MARENA, a government fund created by the Dominican Republic Government for environmental management, to request funding to improve management effectiveness. Finally, another impact of the METT is that the fishers who participate in METT have a better understanding of their role in management processes. Interviewees also indicated that, even though METT scores have increased in just one MPA, improvements can be seen in the administrative capacity and in more transparent management across all MPAs. Other notable changes include a more formal participation of local NGOs in management, as is evident from the memorandum of understanding signed between TNC, CEBSE, Propagas Foundation and MARENA for the co-management of Bajo Yuna. METT results also show changes in management of the Bancos de la Plata y de la Navidad Marine Mammals Sanctuary, where CMBP is actively supporting continuous whale monitoring and whale watching activities, through better surveillance. There is also improvement in management effectiveness at Los Haitises National Park, from very unsatisfactory to unsatisfactory, most likely due to an increase in the number of staff and capacity building activities. Fishers noted that enforcement of fishing regulations in Los Haitises National Park has been effective and should continue. Progress towards performance indicators and strengthening collaborative management CMBP monitoring reports indicate that, of the three indicators associated with Expected Result 4, performance under two have met or surpassed the targets, while one has not yet reached results. CMBP Indicators 5 and 8 (number of hectares of biological significance and/or natural resources under improved natural resource management, and number of person hours of training in natural resources management and/or biodiversity conservation, respectively), have reached their targets. According to the September 2015 CMBP Performance Management Plan report, indicator 3 (number of new laws, policies, strategies, plans, or regulations addressing biodiversity conservation officially proposed or adopted) had a target of 2 by the end of FY16. Even though the Year 3 Monitoring and Evaluation Report lists zero as the number of policies, plans and regulations achieved, the six new no-take zones appear in the narrative of Year 3 Annual Performance Report. The resolution that creates six no-take zones in Samana Bay, was enacted by the Dominican Republic Ministry of Environment, and resolutions emitted by the government should be considered as “adopted”. There is no question that these new no-take zones were declared based on sound technical analysis and with the participation and full support of all stakeholders (e.g. fishers, government authorities and the sub-grantee). Even though these no-take zones have no management plans yet, their creation provides legal mechanism to protect shrimp nursery areas. 27 Participation of non-governmental entities in the management of protected areas is very important, given limited capacity of government agencies to effectively monitor, control, and sanction illegal/unsustainable activities. As noted above, a notable change resulting partially from CMBP implementation, is the participation of local organizations in management, as exemplified by the August 2017 memorandum of understanding signed between TNC, CEBSE, Propagas Foundation (NGO funded by a Dominican company) and MARENA, for the co-management of Bajo Yuna. This co-management agreement was legalized on January 11th, 2018, with partners agreeing to begin work developing a management plan for the area, and annual work plans. Another cooperative MPA governance mechanism, developed under CMBP, is a concession agreement forged between MARENA and Espiritu Santo Cooperative (Los Cacaos) for ecotourism activities in Cayo la Farola. Different actors perceive that the network of cooperation created by CMBP in the Samana Bay area will last far beyond the project timeline, resulting in great benefit for ecosystems and local communities. No-take zones created in Samana Bay One of the key achievements of CMBP in Samana Bay is the creation of six new no-take zones, declared through a resolution of the Ministry of Environment. Nevertheless, government agencies interviewed noted that these new no-take zones still lack management plans. Even though the areas declared constitute a shrimp fisheries management intervention, the participation of the CODOPESCA was limited. The limited participation of the CODOPESCA was due in part to a perceived conflict in jurisdiction with the Ministry of Environment and, apparently, the management plans for these no-take zones will be developed by MARENA and not CODOPESCA. CMBP should facilitate a dialog between these two Dominican Republic government agencies to ensure both are equally involved and support these efforts. Government agencies consulted agreed that the process of establishment of no-take zones took advantage of a proposal that had begun many years ago. Nevertheless, some fisher associations and other stakeholders indicated that perhaps new consultations should have taken place, considering the length of time that has passed since the development of the original proposal. The original consultation process included government agencies, local communities, experts, and members of fisher associations. Interviewees agreed that CMBP conducted an excellent facilitation process, resulting in sound technical analysis with adequate location and boundaries of the no-take zones. However, an interviewee from CODOPESCA indicated that their participation was somehow limited in the final technical process, and that CODOPESCA and MARENA have not reached an agreement when it comes to the responsibility for managing the no-take zones. For example, CODOPESCA questions the legal status of the no-take zones, as they believe these no￾take zones should have been created as fisheries reserves, with relatively flexible boundaries that can be adjusted, and with a mechanism for discussing and deciding what activities should be permitted or not within those limits. Because these no-take zones were declared by MARENA, the no-take zones became protected areas and will be managed by MARENA, not CODOPESCA. Besides, the resolution that created the no-take zones can be overruled by a new minister, whereas with CODOPESCA the no-take zones would have been declared by a decree. This way, the management of the no-take zones can be modified for technical reasons, but the existence of the reserves would be more permanent. Finally, another issue raised by a government agency was the 28 fact that the language in the decree is more legal than technical, and that some terms may not be clear to laymen. Government agencies agree that the FSAP developed under CMBP should be implemented by CODOPESCA; however, it appears that CODOPESCA has the will, but lacks the capacity do to so. The action plan for coral reefs in Samana Bay should be implemented by the Vice-ministry of Coastal and Marine Resources, in coordination with Vice-ministry for Biodiversity and Protected Areas for those reefs located within protected areas. CMBP, through TNC, should also explore the development of co-management mechanisms between the private sector such as dive shops and cooperatives for the management of coral reef nurseries. The marine mammal action plans should be implemented by MARENA’s Vice-ministries of Coastal Marine Resources and Biodiversity and Protected Areas. Enabling factors and conditions Several actions by CMBP have resulted in positive outcomes for marine biodiversity in the Samana Bay area. These include the signing of an agreement between Fondo MARENA and the Caribbean Biodiversity Fund, which resulted in increased funding for the area. Similarly, an agreement signed between TNC and the Ministry of Environment resulted in the organization of a team, including the Ministry’s Planning, Coastal Marine Protected Areas, and Environmental Management Divisions, for the reforestation of mangroves and the installation of a mangrove nursery in Arroyo Barril. A third example can be observed with the concession at Cayo La Farola, granted to a local cooperative, for its use as an observation platform for whale watching tourism. Other conditions that enabled improvements in MPA management effectiveness included the improved condition of the road between Punta Cana and Sabana de la Mar, which resulted in an increase in visitation in Los Haitises. With this increased visitation, the need and incentive for better management of the area was created. In addition, according to CMBP Annual Reports, the project has carried out a series of activities that contributed to overall changes in MPA management effectiveness, including organizational assessments and training, and in improvements in the technical, institutional and administrative capacity of fisher organizations. Management infrastructure CMBP has also contributed to improvements in management infrastructure with equipment and supplies donations. There is also the donation to CEBSE from the German Cooperation that was inspired by the work done by CEBSE under CMBP, which resulted in the construction of an observation deck and floating platform in Cayo La Farola. Even though fishers sometimes have trouble identifying what support comes from CMBP or from prior USAID programs, they acknowledge the provision of scuba diving equipment, suripera nets, kayaks and training in using the new equipment by CMBP. In terms of sustainability beyond CMBP, the cooperative that has the concession at Cayo La Farola indicated they will save some funding to continue after the project ends. TNC acknowledges that CODOPESCA’s participation in project activities is vital for the sustainability of the management infrastructure, as it will increase fishing communities’ adoption of sustainable fishing practices. Observations and Recommendations 29 Based on findings, conclusions and progress towards targets described above, table 7 summarizes observations (improvement opportunities) and recommendations (potential future actions) for Expected Result 4 in the Samana Bay Seascape. Table 7. Observations and Recommendations - Expected Result 4 Samana Bay Observations Recommendations  The Year 3 Monitoring and Evaluation Report list zero for the performance indicator 3 (number of new laws, policies, strategies, plans, or regulations addressing biodiversity conservation officially proposed or adopted) by the end of FY17, even though six new no￾take zones are listed in the Year 3 Annual Performance Report.  TNC should amend the Year 3 Monitoring and Evaluation Report to include the six new no-take zones under indicator 3.  TNC is using the terrestrial version of the METT  In the future TNC should use the marine version of the METT to measure changes in MPA management  Six new no-take zones were declared in Samana Bay through a resolution of the Ministry of Environment  Management plans for the six no-take zones should be developed  Overlapping jurisdictions among government agencies related to fisheries are precluding full collaboration  Jurisdiction and competencies between MARENA and CODOPESCA should be clarified to ensure appropriate management of fisheries resources in Samana Bay, particularly regarding management responsibility of the fisheries exclusion zones. Grenadine Bank Seascape (Grenada and St. Vincent and the Grenadines) Summary Analysis of Findings The Evaluation Team concludes that CMBP has partially achieved Expected Result 4, effective management and governance of MPAs, in the Grenadine Bank seascape. Important management frameworks are in place and being actively implemented, but significant challenges remain, including the need to continually strengthen working relationships with the fishing community. Key achievements in the Grenadine Bank seascape include:  Development and implementation of adaptive mooring management plans for both Tobago Cays Marine Park (TCMP) and Sandy Island - Oyster Bed Marine Protected Area (SIOBMPA).  Good quality rating for both mooring plans (per the desktop evaluation methodology).  Increasing Sustainable Grenadines (SUSGREN)/MPA staff engagement and improving relationships with fishers, tour operators, and local communities (e.g. education, patrols, collaborative planning). 30  Ongoing management and improvement of user fee systems as a sustainable revenue stream at TCMP and SIOBMPA.  Substantial improvements in management effectiveness reported at both TCMP and SIOBMPA over three years of applying the METT. Key challenges that have limited progress towards program targets include:  Limited support from key user groups, especially fishers, in communities near both TCMP and SIOBMPA, and some poaching in no-take zones may still be occurring.  Lack of a SIOBMPA manager for a two-year period, resulting in ad hoc implementation of management activities.  Challenges in building fisher associations into truly organized and active groups, particularly on Union Island and Mayreau.  Some perception of too much focus on collection of user fees to the detriment of other environmental management actions in TCMP.  TNC identified CARIBSAVE (a conservation non-profit with a regional focus in the Eastern Caribbean) suddenly closure as a limitation to the continuity of CMBP Project’s interventions; for example, delaying the hiring of a manager for SIOBMPA. Observations and recommendations listed at the end of this section reflect improvement opportunities and potential future actions that may enhance CMBP performance in the Grenadine Bank under Expected Result 4. Evaluation of adaptive mooring and SIOBMPA management plans Management plans for the two core CMBP MPAs in the Grenadine Bank, TCMP (in St. Vincent and the Grenadines) and SIOBMPA (in Grenada), were originally drafted years prior to the launch of CMBP in the region. Under CMBP, SUSGREN facilitated development of adaptive mooring management plans for each MPA, working in collaboration with boards and staff at each site, as well as local dive shops and yacht charter companies in St. Vincent and the Grenadines and Grenada. A summary of evaluation results for each plan is included in tables 8, 9 and 10 below. (Annex VI includes a comprehensive list of scores for all plans reviewed under this CMBP performance evaluation.) The Evaluation Team assessed two adaptive mooring management plans and one traditional management plan in the Grenadine Bank seascape, including the following:  Adaptive Mooring Management Plan – Sandy Island Oyster Bed Marine Protected Area  Adaptive Mooring Management Plan – Tobago Cays Marine Park  Sandy Island Oyster Bed Marine Protected Area Management Plan A similar structure for each mooring plan demonstrates SUSGREN’s strategic and forward￾thinking approach to maximize available resources for plan development; address common challenges to the sustainable management of tour operators and visiting yacht activities in both protected areas; and design an adaptive management framework flexible to the unique circumstances facing each site, and changing conditions over time. Each mooring plan, given the similarities in both structure and content, received an overall score of 78% (good quality plan), yet both can still realize some improvements. The SIOBMPA 31 Management Plan, revised in 2015 with support from NOAA’s Coral Reef Conservation Program, scored significantly lower overall at 54% (average quality plan), and requires more substantial improvements than either mooring plan. The TCMP management plan was not submitted for review during this evaluation. Creation of the two mooring plans, as strategic management frameworks, represents progress of 50% towards desired performance targets under CMBP indicator 3 Number of new laws, policies, strategies, plans, or regulations addressing biodiversity conservation officially proposed or adopted as a result of USG assistance, for Grenada and St. Vincent and the Grenadines. Each mooring plan received a high 90-100% score in the following overarching management categories (and associated sub-categories): management goals/objectives, legal and institutional framework, stakeholder analysis/engagement, and style/presentation of content. Each plan received lower scores, between 60%-70%, in the categories of management planning and activities, as well as monitoring, evaluation and adaptive management, thus revealing needed improvement areas in each document. Moreover, demonstrated mooring plan implementation at TCMP and SIOBMPA is particularly noteworthy, as anchor damage from visiting yachts and tour boats is a widespread problem throughout the Grenadine Bank. The mooring plans, as TNC has reported, reinforce and support responsible boating practices and reduce the risk of anchor damage to benthic habitats in both MPAs, thus reducing threats to marine biodiversity on a daily basis. SIOBMPA staff, stakeholders and supporting consultants engaged in a highly participatory workshop to inform the 2015 plan revision. Much useful information is included in the SIOBMPA Management Plan. Moreover, it is clear that the document revision builds on important organizational capacity development and priority management actions implemented, or not yet implemented but still a priority, since the plan was originally developed in 2007. However, many plan elements are written like a report of past deliverables, rather than as a strategic framework designed to guide management of the SIOBMPA during the 2015–2025 period. A re-framing of existing, goal driven information in the plan, including greater detail on management and monitoring activities, and how these activities are linked to management goals, may quickly result in a significantly higher plan score. For example, the next revision may wish to re-visit and replicate aspects of the original 2007 plan, specifically the sections which go into substantial detail regarding the linkages between management objectives and strategic actions. Table 8. Evaluation of Adaptive Mooring Management Plan for Sandy Island Oyster Bed Marine Protected Area. Management Category Possible Score Plan Score % Management goals and objectives 6 6 100 Legal and institutional framework 12 12 100 Management planning and activities 42 28 67 Stakeholder analysis and engagement 15 14 93 Monitoring, evaluation and adaptive management 18 11 61 Style/presentation of content 6 6 100 Overall Score/Percentage 99 77 78% Overall Plan Rating: Good quality plan that requires some improvements 32 Table 9. Evaluation of Adaptive Mooring Management Plan for the Tobago Cays Marine Park. Management Category Possible Score Plan Score % Management goals and objectives 6 6 100 Legal and institutional framework 12 12 100 Management planning and activities 42 28 67 Stakeholder analysis and engagement 15 14 93 Monitoring, evaluation and adaptive management 18 11 61 Style/presentation of content 6 6 100 Overall Score/Percentage 99 77 78% Overall Plan Rating: Good quality plan that requires some improvements Table 10. Evaluation of Sandy Island Oyster Bed Marine Protected Area Management Plan. Management Category Possible Score Plan Score % Management goals and objectives 9 6 67 Legal and institutional framework 12 10 83 Management planning and activities 42 20 48 Stakeholder analysis and engagement 18 8 44 Monitoring, evaluation and adaptive management 21 9 43 Style/presentation of content 6 5 83 Overall Score/Percentage 108 58 54% Overall Plan Rating: Average quality plan that requires significant improvements Evolution of collaborative governance and management infrastructure The planning documents described above have built a foundational management structure that guides MPA authorities, in both Grenada and St. Vincent and the Grenadines, in their respective efforts to regularly engage stakeholder groups and promote sustainable use and management of marine resources within and nearby the region’s no-take MPAs. A review of CMBP’s annual reports, combined with field observations, demonstrates that CMBP support has helped facilitate significant and ongoing SUSGREN and MPA staff engagement with fishers, tour operators, and local communities in both countries. Some stakeholders, however, expressed concern that MPA authorities are too narrowly focused on mooring fee collection to the detriment of a more comprehensive approach to marine resource management and appropriate engagement of resource users. At the same time, several interviewees noted that particular individuals from TNC and SUSGREN are well respected and have played a critical role in helping build working relationships between MPA staff and resources users, especially fishers. Under CMBP, field observations demonstrate that greater collaboration across key stakeholder groups has emerged in the Grenadine Bank as a result of: 1) an increase in MPA staffing capacity, skills development and patrolling efforts; 2) training fishers in safety-at-sea and on-board product handling techniques (Hazard Analysis and Critical Control Points or HACCP); 3) supplying fishers with new equipment and gear; 4) replanting mangroves in Aston Lagoon; and 4) conducting community outreach and engagement. Several examples—including training MPA staff in both countries in mooring installation and maintenance, training and supplying fishers, and junior 33 ranger programs—illustrate achievements reported by CMBP and validated during field observations, that exceed targets under indicator 8 (number of person hours of training in natural resources management and/or biodiversity conservation), with nearly 700% above the target for men and more than 200% above the target for women. During the development of mooring management plans for both TCMP and SIOBMPA, TNC and SUSGREN simultaneously supported park staff at each site in the acquisition and installation of additional moorings, and training in mooring maintenance. The mooring programs, though already in existence prior to CMBP, are a particularly notable outcome, given the importance of marine recreation tourism across the Grenadine Bank, and the financial contribution that mooring fees make towards management of marine resources in the area. Field observations also suggest a possible multiplier effect to the economy, as the local water taxi association, whose members often shuttle visiting tourists to local MPAs, has gotten more organized in recent years as MPAs and natural beauty continue to draw visitors to the area. It appears that steady improvements in managed marine tourism are facilitating positive economic impacts for both MPAs (e.g. sustainable finance) and the wider community (e.g. livelihoods), and therefore, at least indirectly, if not yet measured, contributes to progress towards CMBP indicator 6 (Number of people with tangible and/or perceived increases in economic benefits derived from improved natural resource management and conservation as a result of USG assistance). That said, an investigation of this kind of economic impact was beyond the scope of this performance evaluation. Needed communication and engagement with fishers Several interviewees across all stakeholder groups applauded MPA authorities in both countries for past and current stakeholder engagement. At the same time, many also suggested that more community outreach needs to occur, especially with fishers who lost historical fishing grounds as MPAs were legally established. One leader in the fishing community of L’Esterre, on Carriacou, described a confrontation he had to break up between a fisher and a visiting yacht operator, and said that lack of zoning and necessary communication of rules and regulations by wardens contributes to this kind of problem. He noted positively that SIOBMPA has stepped up patrols, but said enforcement without prior education is “like a slap in the face” to local fishers. He emphasized the importance of having a SIOBMPA manager who is local, knows science, and can communicate effectively with locals. In St. Vincent, one government respondent identified resource management and communication with fishers as implementation weaknesses. He specifically noted that more could be done in terms of communicating the benefits of MPAs in general, while simultaneously investing in communities that lost fishing grounds. Although the Grenadine Bank is considered a single seascape under CMBP, field-based interviews revealed, on some issues, findings that were specific to each country. These distinct findings are listed below. Grenada CMBP trained SIOBMPA wardens in mooring maintenance and facilitated development of an operational plan to help organize and advance priority management activities. The operational plan links goals from the SIOBMPA Management Plan to required actions protected area staff must take to build management capacity, maintain a sustained revenue stream, and build important 34 working relationships with key resource user groups, government agencies and the wider community. Key operational plan focus areas include resource protection, governance, livelihoods, administration and financing, and networking CMBP contributed to hiring four staff persons at SIOBMPA. That said, some interviewees suggested that staffing capacity at both SIOBMPA and TCMP has been a challenge. For example, SIOBMPA went without a manager for nearly two years, even though TNC and partners did all they could to advertise and recruit the position. CARIBSAVE initially recruited a manager, but then CARIBSAVE suddenly closed and was no longer part of CMBP. Then TNC selected a candidate who later turned down the job. Finally, SUSGREN recruited the present manager, who accepted the job in late 2017. In the absence of a manager, necessary activities were still conducted by staff, yet not implemented in the well-organized and systematic manner necessary for meaningful results and sustainable management of SIOBMPA. For example, a manager, TNC noted, was needed for proper planning, scheduling of activities, supervision and monitoring of operations to ensure the greatest efficiency and effectiveness. On-water field observations and interviews suggest that patrolling has increased in SIOBMPA. Relations with visiting yachts were reported by wardens as strong and cordial. However, as noted above, leaders in the nearby fishing community of L’Esterre (Carriacou Fisherfolk Organization) noted that more communication about management zones, as well as rules and regulations, is still needed with the fishing community. As the new SIOBMPA manager comes in to her position, she has robust management framework plans (e.g. management plan and mooring plan) in place and previously established relationships with key user groups, each of which will help improve efficiencies, strengthen relationships and guide management from a strategic perspective. St. Vincent and the Grenadines The Evaluation Team observed TCMP staff conduct organized, on-water fee collection and ensure proper mooring usage from visiting yachts and tour operators in the park, demonstrating implementation of the mooring management plan is already occurring. TCMP staff have established rules to manage illegal cooking in the park and are considering new zones to better manage improper waste disposal from visiting yachts. One water taxi operator noted, anecdotally, that the reef in the TCMP has not recovered as much as expected in recent years, and that some poaching at night is likely still occurring. The Evaluation Team received mixed information about the latest status of the TCMP management plan. One government official believed it had been updated, while park staff said the management plan is in need of revision, particularly in the areas of sustainable finance, stakeholder engagement and zoning. Some noted that SUSGREN, TCMP staff, and MPA/MMA staff across Grenadine Bank could benefit from training in consensus building and conflict management, given the challenges of managing fishers, local tour operators and visiting yachts. Some fisher association leaders on Union Island who are well versed in sustainability concepts expressed skepticism that fishers are broadly embracing sustainable fishing practices. One noted the collapse of parrotfish near the island as a perfect example of how fishers will continue using practices that ultimately put themselves out of business. Leaders in the fishing community on both Union Island and Mayreau noted that it has been difficult to bring fishers together in an organized 35 fashion, and to then work in cooperation with TCMP staff. In the face of these challenges TCMP staff is growing its outreach efforts, mentioning during interviews that a park office may soon open on the island of Mayreau. Support from national government High-level government authorities in Grenada and Carriacou spoke positively of how CMBP aligns with national development goals (e.g. protect fisher livelihoods), and the goals of international agreements like the Caribbean Challenge Initiative that both St. Vincent and the Grenadines have signed onto (e.g. 20% of coastal areas protected by 2020). Government authorities in both countries regularly referenced the importance of effective marine resource governance and shared responsibilities, noting they believe the CMBP seascape approach has increased communication and collaborative planning between MPA authorities from each country, with SUSGREN playing a central role in connecting and supporting these groups. Perhaps not surprisingly, given their respective high-level roles in government, officials in both countries commonly lacked detailed knowledge of specific CMBP activities. Officials on Carriacou were an exception, and possessed awareness of a range SIOBMPA management activities and engagement efforts with the local fishing community. At the same time, these officials noted a dearth of needed communication from CMBP to their offices at the outset, yet expressed satisfaction that information sharing improved over time. Further, they noted that the initiative has greater potential to bring people together if it has the backing and, when possible, direct support and involvement of key government agencies. This is a notable observation, wherein effective communication strengthens collaborative governance, and positive lessons may be drawn from the Carriacou experience. Tracking MPA management effectiveness TNC introduced a modified version of the 2007 WWF METT under CMBP, trained key stakeholders (e.g. resource managers and users, police, dive shops, customs etc.) in how to apply the tool, and helped both TCMP and SIOBMPA staff facilitate the METT exercise in 2015, 2016 and 2017. Application of the METT at SIOBMPA took place in the absence of a manager, though at least one interviewee noted that the new manager will likely continue using this tool to monitor ongoing changes in management effectiveness. It is unclear, based on field interviews, how extensively managers and staff are analyzing METT results and using that analysis to inform management. Among all interviewees, local TNC staff demonstrated the deepest understanding of the METT’s purpose and benefits, and how the tool is helping both TCMP and SIOBMPA to date. The Evaluation Team reviewed METT scores submitted by CMBP for both SIOBMPA and TCMP. SIOBMPA staff realized improvements in several key management areas: resource inventory, management objectives and plan implementation, education and awareness, and integration into a larger network of MPAs, among others. TCMP staff realized improvements in the areas of resource inventory, stakeholder awareness and concern, management processes and associated outputs (e.g. enforcement, outreach and education), and achievement of management objectives, among others. For SIOBMPA, lack of proper gazetting was cited as a major obstacle to proper enforcement. In 2017 TCMP identified annual biophysical monitoring as a critical need. 36 A summary of SIOBMPA and TCMP scores are shown in tables 11 and 12 below and reflect a steady increase in score over a three-year period. Some MPA staff noted verbally that the METT helps staff and stakeholders understand management strengths and weaknesses. However, little information or evidence was provided during these interviews which demonstrated actual changes in management effectiveness at TCMP or SIOBMPA. TCMP staff noted they are trying to improve, but some things are beyond the staff control (e.g. patrol boat with no engines). When discussing changes in performance over time, TNC cited AGRRA (Atlantic and Gulf Rapid Reef Assessment) monitoring, an increase in TCMP staff, and a reduction in reported incidents of illegal fishing in both MPAs as evidence that demonstrates modest improvements in management effectiveness. Some interviewees noted development of social, biophysical and governance indicators, but remarked that most monitoring evidence at this stage is anecdotal. A few interviewees at TCMP and SIOBMPA noted that the recent increase in staff numbers lays the foundation for improved management effectiveness over time. Table 11. Application of the Management Effectiveness Tracking Tool (METT) to the Sandy Island Oyster Bed Marine Protected Area for years 2015 – 2017. Year 2015 2016 2017 Total Possible Score 145 145 145 Total Score Received 38 61 84 Table 12. Application of the Management Effectiveness Tracking Tool (METT) to the Tobago Cays Marine Park for the years 2015 – 2017. Year 2015 2016 2017 Total Possible Score 139 N/A 145 Total Score Received 47 N/A 102 Progress towards performance indicators, stakeholder outreach and collaborative planning CMBP monitoring reports show that, among the indicators associated with Expected Result 4, indicator 3 (number of new laws, policies, strategies, plans, or regulations addressing biodiversity conservation officially proposed or adopted) has achieved 50% of its target. Indicator 5 (number of hectares of biological significance and/or natural resources under improved natural resource management) has fully achieved its target. Indicator 8 (number of person hours of training in natural resources management and/or biodiversity conservation) has significantly exceeded its target (700% above the target for men; 200% above the target for women.) Notwithstanding the aforementioned positive CMBP outputs, several groups in both Grenadine Bank countries, including fishers, government officials, and MPA staff, noted that more stakeholder outreach is needed to reduce unsustainable fishing practices and build support for compliance with the no-take regulations in TCMP and SIOMPA. The Grenadines Network of Marine Protected Areas—a support network led by SUSGREN with personnel from TCMP, SIOBMPA and other MPAs, key government agencies and other stakeholders—appears to be the most positive and observable manifestation of collaborative governance. This network of MPA practitioners from both countries convenes annually to discuss challenges, improve management 37 frameworks and plan for the future. CMBP has supported and strengthened this network by financing two annual meetings and enabling SUSGREN to hire a MPA coordinator who facilitates collaboration between member MPAs year-round, rather than just at annual meetings. Management infrastructure Common responses to the question of management infrastructure under CMBP included acquisition of mooring buoy equipment for both TCMP and SIOBMPA, and safety-at-sea equipment for fishers. One interviewee who is involved with partners across the seascape noted mooring supplies for MPAs and described the range of supplies provided to fishers (e.g. ice boxes, vhf radios, lifejackets, flare guns, etc.) to make their work further out at sea safe and economically sustainable. Some MPA staff received SCUBA training to assist with mooring maintenance, but no actual SCUBA equipment. When asked about sustainability of management infrastructure beyond the life of the program, a number of interviewees pointed to the design and intention of the mooring management plans. TNC staff described key steps that are taken to ensure infrastructure and equipment lasts beyond the life of the project, for example: 1) ensure equipment is something the recipient wants/needs; 2) provide equipment that generates income, thereby creating an incentive to maintain it; and 3) secure support from the relevant ministry in how to maintain certain equipment (e.g. beehives). One interviewee noted that TNC has forms which need to be signed for some equipment. Another noted that verbal requests made to recipients. Still another interviewee noted a transfer agreement for services, but not equipment. Enabling factors and conditions A main theme that surfaced when exploring enabling factors and conditions is the importance of people. Most responses focused on two types: 1) users and beneficiaries of marine resources and 2) staff who manage and run the MPAs. For the former, interviewees noted that it is critical to involve and not exclude people, and to make sure they see and receive benefits from marine resource management. For the latter, interviewees noted that it is important to acquire and develop the right staff to manage the MPAs, and to give them the right training for the job. Several specific examples of staff training needs were provided, including consensus building and conflict management, monitoring, enforcement, CPR and engine maintenance among others. Another stressed the need for resource managers to incorporate traditional knowledge. Other interviewees noted that TCMP wardens need to focus on environmental management and not just collecting fees. At least one high level government official in St. Vincent and the Grenadines noted that SUSGREN’s efforts to cultivate junior rangers in the community is very important. Finally, some interviewees generally noted that MPA capacity at sites across the Grenadine Bank remains a key challenge (i.e. human resources, equipment and technical skills). Observations and Recommendations Based on findings, conclusions and progress towards targets described above, table 13 summarizes observations (improvement opportunities) and recommendations (potential future actions) for Expected Result 4 in the Grenadine Bank Seascape. Table 13. Observations and Recommendations - Expected Result 4 Grenadine Bank Observations Recommendations 38  Broad awareness of management plans among most stakeholders, but limited understanding of specific management priorities, rules and regulations  Leverage established forums (e.g. MPA annual network workshops) and outreach channels (e.g. fisher associations) to improve messaging and public awareness  Significant outreach to fishers occurs, though more is needed, specifically for fishers who lost fishing grounds  Train SUSGREN and MPA staff in facilitation, consensus building, conflict management and effective communication  Support TCMP efforts to open an office on Mayreau  SIOBMPA Management Plan needs substantial improvements  TCMP Management Plan needs revision  Mooring action plans rated as good quality, but could still be improved  Hold a series of internal (i.e. SUSGREN, park staff and boards) and external (i.e. stakeholders) planning sessions and use plan evaluations to explore weaknesses and facilitate needed improvements  TCMP management activities and staff time may be too narrowly focused on user fee collection  Consider innovative ways to engage TCMP Board and senior staff to address the issue via a participatory MPA management planning process  Limited awareness of specific CMBP goals and activities among high-level government officials and stakeholders  Develop a communication and outreach plan as an amendment to existing TCMP and SIOBMPA management plans  Little evidence seen of early launch of protect parrotfish campaign  TNC parrotfish campaign needs more support and could learn from Belize; future efforts could include use of social media, on-the-ground campaign materials and banners installed in selected areas. North Coast of Hispaniola Seascape (Haiti) Summary Analysis of Findings The Evaluation Team concludes that CMBP has partially achieved expected result 4, effective management and governance of MPAs/MMAs in the North Coast of Hispaniola Seascape. Seven out of 15 CMBP key activities planned for 2017 were completed, including the Protected Area of Managed Natural Resources of the Three Bays (PA3B) Management Plan, along with a zoning system and priority action plans for 2017-2027. Key achievements in the North Coast of Hispaniola seascape include:  The PA3B Management Plan is the most comprehensive and well written of all plans reviewed during this CMBP evaluation.  METT scores for the PA3B increased 8 points from 2016 to 2017. The 3 performance indicators associated with result 4 have met or surpassed targets (Indicator 3, Number of new laws, policies, strategies, plans, or regulations addressing biodiversity conservation officially proposed or adopted as a result of USG assistance, Indicator 5, Number of hectares of biological significance and/or natural resources under improved natural resource management as a result of USG assistance, and Indicator 8, 39 Number of person hours of training in natural resources management and/or biodiversity conservation supported by USG assistance). Key challenges that have limited progress towards program targets include:  Securing the Minister of Environment’s signature in the foreword of the PA3B Management Plan has been difficult. That last step is needed to allow for wide official dissemination.  Financing for the implementation of the PA3B Management Plan activities and strategies has not been secured. Observations and recommendations listed at the end of this section reflect improvement opportunities and potential future actions that may enhance CMBP performance in Three Bays seascape under Expected Result 4. Evaluation of PA3B management plan All the associations and stakeholders the Evaluation Team met with in the field expressed a strong desire to work together to achieve tangible results from this program. This is no doubt a result of CMBP’s achievements to date, including the mobilization of the communities, fishers’ associations, local and central authorities for the development of the PA3B Management Plan. The Evaluation Team conducted a formal desktop review and evaluation of the PA3B Management Plan. Development of the management plan was guided by the sub-grantee, FOPROBIM, and the CMBP Team, with support from Global Parks/International Caucus Foundation. This plan is the most comprehensive and well written of all the plans reviewed during this CMBP evaluation. Scores and percentages across all assessed management categories for the PA3B Management Plan are summarized in Table 14 (Annex VI includes a comprehensive list of scores for all plans reviewed under this CMBP performance evaluation). The PA3B Management Plan scores particularly well across two-thirds of all overarching management categories and associated sub-categories assessed. There are several notable strengths throughout the plan, not the least of which include:  States clear goals and objectives  Integrates national laws, development plans and international agreements  Includes a robust threats analysis and informed planning  Clearly describes management zones, rules and regulations and an enforcement plan  Identifies staff training programs and links to nine conservation programs  Describes the role of the stakeholder advisory committee  Prioritizes local education  Understands baseline biophysical data  Contains strong overall content, use of graphics and presentation style Unlike other CMBP management plans and action plans assessed, the PA3B Management Plan requires minimal improvements. Overall, the plan received an 85% rating (excellent quality plan). The plan could further define its biophysical and socio-economic monitoring requirements and monitoring protocols. Moreover, given there is a functioning stakeholder advisory committee, a collaborative process to define a vision and mission for management of the area may be a useful 40 next step. The Evaluation Team recommends enhancing the management plan and complementing it with a detailed operational plan and a realistic budget focused on targeted actions for marine areas in the PA3B. Table 14. Evaluation of the Protected Area of Managed Natural Resources of the Three Bays Management Plan. Management Category Possible Score Plan Score % Management goals and objectives 9 8 89 Legal and institutional framework 12 12 100 Management planning and activities 42 36 86 Stakeholder analysis and engagement 18 17 94 Monitoring, evaluation and adaptive management 21 13 62 Style/presentation of content 6 6 100 Overall Score/Percentage 108 92 85 Overall Plan Rating: Excellent quality plan that requires minimal improvements Even though the PA3B Management Plan has been validated by the government, final approval, through the Minister of Environment’s signature in the foreword of the document, is still pending. This last step is needed to allow for wide official dissemination of the plan. The government agency in charge of protected areas, National Agency for Protected Areas (ANAP), considers the completion of this plan a key achievement of CMBP, and intends to use it as a reference document for the development of future management plans. Nevertheless, it is unclear how ANAP will take full ownership of the plan to develop governance mechanisms and implement the plan’s activities and budget. CMBP should collaborate closely with ANAP to identify the support required to make implementation of the PA3B Management Plan effective. Significant achievements under Expected Result 4, such as the completion of the management and zoning plans and the training of protected area wardens, provide a sound basis to guide the planning and financing of management activities. If implemented, the PA3B Management Plan can significantly contribute to CMBP’s goal of a significant reduction of threats to marine biodiversity and an increase in income of the fishers, due to a sustainable management of marine and coastal resources. Nevertheless, the goal cannot be attained, unless resources are sufficiently allocated to the pursuit of objectives and identified actions in the 3BNP Management Plan. Much remains to be done prior to being able to achieve measurable results, such as the implementation of the fishing gear exchange program, procurement of basic monitoring equipment, production of environmental education materials, and institutional support to ANAP. During evaluation interviews, TNC reported that the PA3B Management Plan is being used a reference document for all stakeholders and funding agencies in the protected area, including IDB and UNDP, who use it to guide their interventions. For the sub-grantee, the activities currently under implementation come from a table of priority actions in the management plan. For some members of fisher associations “the plan is not yet being implemented in the field, except for FOPROBIM trainings, and the government should ensure the implementation of the plan activities in the field for the protection of PA3B. According to ANAP, it is too early to clearly indicate how the plan is being used since it has been just recently completed and is pending final approval. 41 For the management plan to be implemented, there is a first need to build capacity on the ground. ANAP is the manager for the MPA, and they already have a protected area manager. FOPROBIM is the ad hoc manager of the protected area and have requested to be co-managers. A co￾management agreement has been drafted, but not yet approved, mostly due to constant changes in ANAP’s staff. Tracking management effectiveness CMBP, through TNC and FOPROBIM, helped ANAP complete METT exercises for the PA3B during the three years of the Program implementation. The results of these METT exercises are presented in Table 15. The PA3B obtained an increase of 8 points in its METT score from 2016 to 2017. This increase is a result of the completion of the management and zoning plans, and to FOPROBIM’s wardens being hired. Although the wardens are not yet fully trained and functional, they are now actively patrolling. According to TNC and FOPROBIM, METT workshops have been attended by representatives of local organizations and stakeholders like the mayor’s office, ANAP, NGOs intervening in the protected area, departmental (state) directors of ministries of agriculture and environment, the protected area manager, and the UNDP representative. All these three METT reports were made available to the evaluation team. The representative of the ANAP interviewed by the Evaluation Team was aware of only one-time application of the METT, and the new technical director of ANAP has not yet received the METT results. Table 15. Results of the application of the Management Effectiveness Tracking Tool (METT) to the Protected Area of Managed Natural Resources of the Three Bays, for years 2015 – 2017, based on self-evaluation, and using the terrestrial version of METT Scorecard. Year 2015 2016 2017 Total Possible Score 96 96 96 Total Score Received 32 31 39 It is important to note that the METT applied in PA3B is the first version of the terrestrial METT, which has a total of 30 questions with scores from 0-3, and other sections with extra points for a total possible score of 96. METT sessions for the PA3B were conducted with various stakeholders, with the leadership of FOPROBIM and TNC staff, as part of CMBP implementation. ANAP has been a participant in METT sessions but has not led them. Since the program has been implemented for over three years now, it is time to progressively transfer the leadership and responsibility for the METT process to ANAP’s local and national authorities. TNC should consider building ANAP’s capacity to monitor management effectiveness. With this capacity ANAP, as Haiti’s national agency for protected areas, can lead and facilitate METT processes in other areas in Haiti where FOPROBIM does not work. To the question of how the results of the METT are being used, FOPROBIM mentioned during the interview that results are used in the development of the CMBP annual work plan, and the findings in the METT are also used by FOPROBIM for the management of the protected area. According to TNC, the first time the METT was applied participants did not seem to understand the merits of the process. Nevertheless, the third time the participation was more active and visible, the mayors and the representatives from the organizations became more familiar with the tool, and were more aware of the importance of the tool in monitoring for protected area management 42 effectiveness. Since ANAP has not yet received the METT results, they were not able to share how ANAP is learning from and utilizing the METT results. Nevertheless, ANAP mentioned that a local committee has been put in place to monitor progress on management plan implementation, including using the results of the METT. According to TNC and FOPROBIM, the factors that helped increase PA3B’s METT scores include the development of the management plan, monitoring actions, a budget that was made available for the protected area, and wardens and other staff that was hired to carry out awareness and community education activities. All these can be considered as examples proving that the METT accurately and effectively measures changes in management effectiveness over time. According to the sub-grantee, the METT is a useful tool and an important gauge of changes in management effectiveness, and that the primary party responsible for the follow-up of METT results is ANAP. Other interviewees indicated that improvement of management effectiveness is a result of planning and financing activities. ANAP could not comment on the reasons for METT score changes since, as it was mentioned above, they have not received the METT reports. According to FOPROBIM, UNDP funding for hiring a protected area manager, and the fact that local mayors and not only national authorities are collaborating with PA3B have also helped increase management effectiveness. Progress towards performance indicators and strengthening collaborative management Regarding the indicators associated with Expected Results 4, performance indicators 3, 5 and 8 have met or exceeded the targets. CMBP indicator 3 (number of new laws, policies, strategies, plans, or regulations addressing biodiversity conservation officially proposed or adopted), had a target of 2, and thus far CMBP has two policies proposed or adopted in Haiti. One conservation agreement, outlining Haiti’s official participation in the Caribbean Challenge Initiative, was endorsed in September 2016, and the management plan for the PA3B, which includes a zoning plan, was validated by the government. CMBP indicator 5 (number of hectares of biological significance and/or natural resources under improved natural resource management) had a target of 75,279 hectares in Haiti, which is exactly the number achieved to date. Finally, CMBP indicator 8 (number of person hours of training in natural resources management and/or biodiversity conservation) had as targets 1,588 person-hours of training for men and 1,150 for women. The year 3 CMBP monitoring report indicates a total of 4,076 person-hours completed for men and 2,925 for women in Haiti. This is attributed to FOPROBIM’s alternative livelihoods activities, such as an apiculture business for associations, in parallel with capacity building trainings to strengthen the institutional development of community-based organizations. Management infrastructure CMBP has contributed to improvements in management infrastructure through funding equipment and supplies donations. Based on TNC’s re-prioritization exercise, it is estimated that roughly half of the planned activities in the management infrastructure program have been completed. The main achievements under management infrastructure were the training and hiring of eight officers to monitor activities related to mangrove cutting and destructive fishing practices in the PA3B. ANAP promised to ensure that those protected area officers will be paid by the government once the program ends. Despite these achievements, according to TNC, many other expected outputs in the management infrastructure plan, like the purchase of boat engines, are delayed, mostly because of prolonged procurement procedures. Other management infrastructure activities that have not 43 been completed include purchasing of monitoring equipment, installation of signage, and environmental education dissemination materials. The Evaluation Team recommends accelerating the mechanisms for purchasing supplies and equipment, such as motorboats and other equipment, for the day-to-day monitoring of PA3B activities. According to TNC and the sub-grantee, CMBP has already provided diving equipment for the monitoring of marine resources in the PA3B, such as tanks and a compressor. Apicultural equipment and supplies have been already granted to two fishers’ associations, and CMBP paid for half of the cost of a vehicle, according to the sub-grantee. ANAP reported no knowledge of management infrastructure or equipment provided by CMBP. To ensure that the infrastructure and equipment will continue to be used beyond the life of CMBP, TNC selected FOPROBIM as CMBP partner, an organization with a longstanding reputation. Numerous training sessions were provided by FOPROBIM to fisher associations that would help ensure management infrastructure sustainability. TNC also hired the firm Integrated Development in Haiti (IEDIH) to mentor fisher associations and to provide complementary training in organizational development. Finally, TNC is coordinating with UNDP to develop synergies and joint projects and actions that can go beyond the CMBP. Observations and Recommendations Based on findings, conclusions and progress towards targets described above, table 16 summarizes observations (improvement opportunities) and recommendations (potential future actions) for Expected Result 4 in the North Coast of Hispaniola Seascape. Table 16. Observations and Recommendations - Expected Result 4 Three Bays seascape Observations Recommendations  ANAP needs to secure the signature of the Minister of Environment in the foreword of the PA3B Management Plan. This is a necessary step before the Plan official dissemination  TNC should work with ANAP to identify the support it requires to implement the PA3B Management Plan  METT processes have been led and facilitated by TNC and FOPROBIM  TNC should consider developing activities for building ANAP’s capacity to monitor management effectiveness  Financing for PA3B Management Plan is pending  TNC should seek ways for accelerating the procurement process for supplies and equipment required for day-to-day PA3B activities  A few actions can be developed to further enhance the PA3B management plan  The PA3B management plan could be enhanced and complemented with a detailed operational plan and a realistic budget that focuses on targeted actions for marine areas in the protected area Pedro Bank and Southwest Coast Seascape (Jamaica) 44 Summary Analysis of Findings The Evaluation Team concludes that CMBP has partially achieved Expected Result 4, effective management and governance of MPAs, in the Jamaica seascape. Significant capacity development of fish sanctuary management organizations has occurred; however, a range of challenges remain, including, among other things, the ongoing need to strengthen patrolling, biophysical monitoring and professional development of wardens. Key achievements in the Jamaica seascape include:  Support for finalizing collaborative plan development and advancing towards government approval of the Pedro Cays and Surrounding Waters Management Plan (Pedro Cays Plan).  Good quality rating received on the updated Pedro Cays Plan, with some outstanding need for improvements.  Process started for the development of a management plan for the South West Cay Special Fishery Conservation Area (SFCA).  Implementation of a Rapid Fisheries Sector Assessment (RFSA) followed by development of the Fisheries Strategic Action Plan for the Jamaica Seascape: Pedro Bank and South West Coast (FSAP). This early step provided capacity building guidance to multiple fish sanctuaries, which, collectively, make up a network of no-take MPAs around Jamaica.  A Campaign for Conservation workshop, with broad participation from CMBP partners and other NGOs, that helped small organizations engage in conservation planning and subsequently receive small grants from TNC (e.g. Caribbean Coastal Area Management Foundation or C-CAM and fisher associations, etc.).  A wide range of trainings conducted and well received, with balanced participation of men and women; exceeded performance indicator 8 (Number of person hours of training in natural resources management and/or biodiversity conservation supported by USG assistance) target.  Increased capacity for patrolling, outreach and education, monitoring and management at South West Cay and Bluefields Bay Special Fishery Conservation Areas (SFCAs).  Targets for performance met for indicator 3, Number of new laws, policies, strategies, plans, or regulations addressing biodiversity conservation officially proposed or adopted as a result of USG assistance, and indicator 8, Number of person hours of training in natural resources management and/or biodiversity conservation supported by USG assistance, under Expected Result 4. Key challenges which have limited progress towards program targets include:  Organizational structure at C-CAM was slow to produce results; TNC hopes the hiring of a new sanctuary manager will produce outputs in the remaining program period.  Fisher associations have limited staff capacity and technical skills to administer anything more than a small, fixed price grant.  Limited capacity to conduct 24/7 patrolling and thereby fully stop poaching in South West Cay and Bluefields Bay.  Challenges to building a robust enforcement program at South West Cay, based on close relations between wardens and fishers, and the dangers of patrolling at night.  Outreach and education challenges based on literacy issues among fishers and some wardens; also, high turnover of fishers operating in the Pedro Cays. 45  Application of the wrong version of the METT by CMBP partners; redundancies in how often it is applied and for who; and performance scores have not substantially changed over the 2015–2017 period, and have dropped for the Galleon St. Elizabeth SFCA.  Targets under performance indicator 5 (Number of hectares of biological significance and/or natural resources under improved natural resource management as a result of USG assistance) not met.  The sudden closure of CARIBSAVE has been identified as a key challenge by TNC. Observations and recommendations listed at the end of this section reflect improvement opportunities and potential future actions that may enhance CMBP performance in Pedro Bank and the southwest coast under Expected Result 4. Planning and Building Fish Sanctuary Management Capacity In 2009, the Jamaican government, with broad public support, created nine no-take fish sanctuaries in multiple locations around the island, with more to follow in later years. These sites are currently designated as SFCAs but are also commonly known as fish sanctuaries. At the outset of CMBP, TNC, with consultant support, facilitated development of a Rapid Fisheries Sector Assessment (RFSA) and subsequent Fisheries Sector Action Plan (FSAP) for the Pedro Cays and southwest coast of Jamaica. The action plan was designed to help guide fisheries management capacity building work in this relatively large geographic area. As an early step, TNC catalyzed conservation action by facilitating a Rare lead Campaign for Conservation workshop for several NGOs, including its CMBP partners. This workshop enabled local organizations to plan conservation campaigns, secure grants, and then implement self-designed programs. C-CAM, the primary CMBP sub-grantee, manages the South West Cay SFCA, the only officially declared protected area on the Pedro Bank and included in CMBP. South West Cay is one of what is now a network of 17 fish sanctuaries spread across coastal waters in Jamaica. Under CMBP, TNC has been building C-CAM’s capacity operationally and institutionally to improve fisheries management at this remote site. Specifically, C-CAM received a boat to conduct enforcement, staff supplies (e.g. uniforms, boots etc.), and AGRAA biophysical survey equipment. Staff is now SCUBA certified and will receive AGRRA basic training in 2018. South West Cay demarcation moorings have been replaced twice. Office-based staff were provided computers and software technology to assist C-CAM with sub-grantee reporting requirements. These same staff will soon get training in Quickbooks. Of note, during interviews C-CAM staff shared with the Evaluation Team that a management plan for South West Cay is now under development, and TNC is supporting this effort in CMBP Year 4. The Bluefields Bay Fishermen’s Friendly Society (BBFFS), which manages Bluefields Bay SFCA, received small grant support for staff salaries, capacity building in many areas, various trainings, and now is looking to build ecotourism enterprises. (The Evaluation Team was shown a shallow reef area in the sanctuary that is already receiving tourist visitation and generating local income.) The BREDS Treasure Beach Foundation, a volunteer-based charitable organization run by and for members of the Treasure Beach, Jamaica, community, which manages the Galleon St. Elizabeth SFCA, did not have the capacity to manage a small grant. Draft management plans for both SFCAs, which are located along the south west coast, were prepared by CARIBSAVE and pre-date CMBP. In fact, the Bluefields Bay management plan was developed by the community 46 and used to acquire fish sanctuary status. Some interviewees described in general terms how the loss of CARIBSAVE meant less operational and administrative support for local partners. For example, without CARIBSAVE, the coral garden in Bluefields Bay went unmanaged. CMBP support helped staff revive the garden, which in turn draws tourists to the area. No interviewee described in detail what impact the CARIBSAVE sudden closure had on CMBP’s activities in Jamaica. Finally, it is important to note that women occupy many leadership and mid-level staff positions within the organizational management structure of both C-CAM and BBFFS. Moreover, TNC Jamaica is run largely by women. This high degree of women in leadership positions has helped promote greater inclusion of women in decision making and participation in core CMBP activities in Jamaica, as well as tangible, gender sensitive benefits. For example, in Bluefields Bay, one BBFFS staff person told the Evaluation Team that the income she receives as a fish sanctuary warden has enabled her son to attend high school. These key findings can be viewed in a positive light given the focus on gender sensitivity under CMBP. Evaluation of the Pedro Cays and Surrounding Waters Management Plan (Pedro Cays Plan) The Pedro Cays Plan, updated in September 2015, was originally prepared in 2012 with funding from UNDP’s “Management and Conservation of Reef Biodiversity and Reef Fisheries Pilot Project – Pedro Bank and Cays, Jamaica.” TNC received a grant from the National Environment and Planning Agency (NEPA), the recipient of UNDP funds, and contracted Dr. Susan Otuokan to develop the plan. All government sources interviewed by the Evaluation Team indicated that, from the outset, the process for developing the plan was highly collaborative, with key partners including NEPA, the Jamaica Defense Force (JDF) and the University of the West Indies, among other stakeholder groups. CMBP is credited with supporting document review in 2015, and, under TNC’s leadership, facilitating a series of stakeholder consultations, notably with fishers, that enabled an update and finalization of the plan. For this CMBP mid-term performance evaluation, the Evaluation Team assessed the quality of the Pedro Cays Plan. The plan received an overall score of 67% (good quality plan); however, could still realize some improvements. The plan scores particularly well in defining a clear vision, mission and goals; describing the legal framework and context within which the plan exists; conducting a threats analysis; identifying revenue streams to support implementation; facilitating participatory planning processes; and prioritizing education and stakeholder engagement. Sections that could still be added or improved include a more detailed analysis of limitations and opportunities for management, periodic reporting mechanisms; an enforcement plan; and further development of methods for monitoring governance and socio-economic indicators, and thereby measuring overall plan performance. (Annex VI includes a comprehensive list of scores for all plans reviewed under this CMBP performance evaluation.) Table 17. Evaluation of the Pedro Cays and Surrounding Waters Management Plan. Management Category Possible Score Plan Score % Management goals and objectives 9 8 89 Legal and institutional framework 12 11 92 Management planning and activities 42 29 62 Stakeholder analysis and engagement 18 11 61 47 Monitoring, evaluation and adaptive management 21 8 38 Style/presentation of content 6 5 83 Overall Score/Percentage 108 72 67% Overall Plan Rating: Good quality plan that requires some improvements Fisheries legislation and the status and level of awareness of the Pedro Cays Plan TNC has made significant contributions under CMBP in helping promulgate more effective fisheries policy, including, among other things, providing input on the successful 2015 amendment to the national fisheries law. With the development of the Pedro Cays Plan, and the Marine Spatial Plan (MSP) for the Pedro Bank officially proposed to the Government in March 2017, CMBP has already met its program target for Jamaica under performance indicator 3 (number of new laws, policies, strategies, plans, or regulations addressing biodiversity conservation officially proposed or adopted). Some interviews suggest that the Pedro Cays Plan has been presented before an Inter￾Ministerial Committee, which has signed off on major aspects of the plan. However, others suggest the committee has not yet formally approved the full document as of February 2018. TNC reports receiving a letter from NEPA fully accepting the management plan, which was submitted to an inter-agency GEF-UNDP Project Steering Committee comprised of government and NGO entities. However, the Pedro Bank MSP has not yet been submitted by NEPA to the National Council on Oceans and Coastal Zone Management, an inter-ministerial committee, for review and approval. Interviews with government agencies revealed that although most agencies possess general awareness of the Pedro Cays Plan, many have not actually seen the document, suggesting broader distribution of the final plan is warranted. Some interviewees noted a disconnect between agencies with overlapping mandates in the management of natural resources, and thus inefficiencies. Also, few fishers interviewed are aware of the management plan. All this suggests a renewed focus should aim to generate broader distribution and awareness of the plan, its driving goals and the role many agencies and organizations must play to advance effective management of critical fisheries resources and marine biodiversity in the Pedro Cays. This could be done in remaining CMBP time. The absence of such collaboration may limit the ability for management to be scaled up, and thereby reduce threats to marine biodiversity in priority CMBP sites and across Jamaica. Training and promotion of sustainable fishing practices As will be explained in more detail in the Expected Results 3 section for Jamaica, field-based interviews revealed a number of trainings either involving or led by C-CAM which have been well received. C-CAM staff received trainings in basic biological monitoring, first aid, and CPR. Fishers and C-CAM staff both benefitted from trainings in turtle tagging and monitoring, and received a basic package of tools to conduct various kinds of biophysical monitoring (e.g. reefs, birds, turtles). C-CAM staff has creatively engaged fishers on sustainability issues through DJ music events and workshops, thereby improving fisher understanding of sustainability concepts. Training on how and why to fish lobster sustainably was combined with distribution of calipers to measure length of maturity upon catch and enable fishers to return under-sized lobsters. Fishers, as well as some C-CAM staff, also received safety-at-sea training. Finally, several interviewees noted that important and well received trainings took place out in the cays in 2017, including a training on how to harvest lionfish. TNC plans a renewed campaign to create a lionfish market in the remaining CMBP time period. The combined effort of these trainings, and others mentioned 48 in the Jamaica Expected Result 3 section, illustrates how CMBP has already exceeded its life of project target under indicator 8 (number of person hours of training), with 1,812 hours completed against a target of 1,064. Of note, more than a third of the total number trained were women. Organizational capacity and leadership challenges Notwithstanding the aforementioned activities and associated outputs, CMBP faces a host of challenges in building management capacity at the South West Cay SFCA and nearby waters. Interviews revealed that literacy among fishers at the cays is an ongoing challenge for C-CAM. For example, wardens hired under CMBP, who are often fishers themselves, are at times illiterate. However, these same fishers are often seen as the right persons for establishing good relations with other fishers during patrolling activities. Interviews also suggest that wardens who are part of the fishing community may not want to conduct enforcement on fellow fishers. Moreover, some interviewees noted that wardens are hesitant to patrol at night due to the danger factor. The JDF Coast Guard and Marine Police, and the government more generally, are trying to bring greater organization, safety, waste management and societal norms to Middle Cay and Southeast (Top) Cay, and to the fisher encampments that are a regular presence in these cays, which are near the South West Cay SFCA. As noted, CMBP has supported a patrol boat and other equipment for South West Cay, and thus C-CAM staff and wardens have become a regular presence in the area. Field-based research by the Evaluation Team also revealed that C-CAM’s capacity to complete necessary work has been a challenge. TNC initially followed the approach of how C-CAM manages other sites and assisted the organization in hiring two supervisors to guide and mentor wardens. When expected reporting and monitoring was not forthcoming, TNC strategized with C￾CAM to try a different approach by hiring a sanctuary manager, a dedicated person for the site. TNC is hopeful this new approach will work better in the remaining CMBP time period. At the same time, it is important to note that C-CAM faces many challenging contextual circumstances that would be difficult for any local organization to address. For example, although there is broad interest among many agencies in the management of important natural resources at in the Pedro Cays (e.g. conch, lobster, fish etc.), some interviewees expressed concern that no agency is stepping up and taking responsibility for implementation of the Pedro Cays Plan. TNC reports that in the draft National Cays Policy, NEPA has been tasked to create a Pedro Cays management unit as a governance approach for managing all Cays. However, interviews suggest it is still not clear for some which ministry is ultimately responsible for management. Moreover, one interviewee noted that the MSP recommends the fish sanctuary be three times its current size, however, the at least one interviewee noted that the plan was not approved in that way. Nevertheless, C-CAM is implementing certain aspects of management and staff continue patrolling, educating and enforcing in South West Cay and nearby areas, and are actively working to implement priority management activities. Tracking management effectiveness at burgeoning fish sanctuaries CMBP in Jamaica has not yet met its full life of project targets for indicator 5 (number of hectares of biological significance and/or natural resources under improved natural resource management). As of 2017, fish sanctuary management activities and associated outputs described above have contributed to achieving improved management in 1,737 hectares of the marine environment, against a target of 2,919 hectares. Several fish sanctuary management organizations, including C￾CAM, BBFFS and BREDS -Treasure Beach Foundation (Galleon St. Elizabeth SFCA) applied the 49 METT in 2015, 2016 and 2017. Results for each site are listed in tables 18, 19 and 20 below. Notably, scores largely stayed the same for each organization across three years. Some saw modest improvements due to CMBP support for hiring new staff, increasing patrols, formalizing game warden status for sanctuary staff, and, in the case of Bluefields Bay SFCA, a renewed focus on coral nursery restoration work and professional development training for wardens (e.g. SCUBA). It appears that each site is using the WWF 2007 version of the METT to align with a UNDP-GEF project. C-CAM staff suggested that the METT is not applicable for an environment like the Pedro Cays, yet it appeared senior staff were not aware that there is a marine version of the tool which was developed by the World Bank in 2004. In addition, some interviewees believe fish sanctuary management organizations are simply applying the score card, and not subsequently analyzing and then utilizing results to inform organizational development and staff work planning. Table 18. Results of the Application of the Management Effectiveness Tracking Tool to the South West Cay Special Fishery Conservation Area (Pedro Bank), for the years 2015 – 2017, based on self-evaluation, and using the terrestrial version of METT Scorecard. Year 2015 2016 2017 Total Possible Score 102 102 102 Total Score Received 59 59 60 Table 19. Results of the Application of the Management Effectiveness Tracking Tool to the Bluefields Bay Special Fishery Conservation Area, for the years 2015 – 2017, based on self￾evaluation, and using the terrestrial version of METT Scorecard. Year 2015 2016 2017 Total Possible Score 102 102 102 Total Score Received 72 70 72 Table 20. Results of the Application of the Management Effectiveness Tracking Tool to the Galleon St. Elizabeth Special Fishery Conservation Area, for the years 2015 – 2017, based on self￾evaluation, and using the terrestrial version of METT Scorecard. Year 2015 2016 2017 Total Possible Score 102 102 102 Total Score Received 72 69 66 It is important to also note that in 2017 TNC facilitated, with CMBP support, a management effectiveness evaluation of all 17 sites in the Jamaica fish sanctuary network using the marine METT developed by the World Bank. That said, results of this evaluation were not submitted for review during this mid-term performance evaluation, and thus are not listed in this report. C-CAM, BBFFS and staff at Galleon, St. Elizabeth appear to have applied WWF’s 2007 METT during the same time period as the network-wide evaluation. This suggests there is redundancy in that the same work is apparently being done twice, at the same time, to meet deliverables of different bi￾lateral or multi-lateral donors, It also presents a challenge to integrate past data sets into current approaches to applying the METT or other management effectiveness tracking tools. Following the network-wide METT evaluation, representatives from each organization that manages a fish sanctuary or multiple sanctuaries gathered for the weeklong professional development workshop. Participants collaborated closely to develop minimum performance 50 standards for four priority management activities—human resources, enforcement operations, biophysical monitoring and financial management. In addition, each organization closely reviewed its evaluation results, identified needed performance improvements and began to map out action steps forward that will build capacity and, over time, improve management effectiveness. Evolving partnerships with government Fish sanctuary management organizations, including C-CAM, BBFFS and BREDS-Treasure Beach Foundation (Galleon St. Elizabeth SFCA), have benefitted financially from their relationship with the Fisheries Division and NEPA, as well as evolving partnerships with the national JDF and Marine Police. C-CAM in particular is strengthening its collaboration with the Marine Police, and the JDF commonly provides transport assistance for staff to the Pedro Cays. A logical next step that C-CAM is working towards is development of a memorandum of understanding (MOU) with the Marine Police. This type of agreement may outline the roles, responsibilities and commitment of both parties to a formal partnership. If this MOU comes together, it will further build institutional and operational management capacity in the Pedro Cays and may offer important lessons for other management organizations responsible for Jamaica’s fish sanctuaries. This issue warrants particular attention in the remaining CMBP time period, as wardens from both C-CAM and BBFFS have noted that enforcement in fish sanctuaries is limited in the absence of 24/7 patrols and consistent support from the Marine Police. Enabling factors and conditions Among interviewees who spoke about enabling factors or conditions, the most common responses included funding, capacity, and human resources (i.e. the right people excited about the job who can do the work and bring about public awareness). One interviewee expressed concern that if funding does not continue, CMBP partner organizations will face capacity challenges. BBFFS staff noted that fishers believe spillover is occurring, and it is important that fishers feel this, even if evidence to date is only anecdotal. Another interviewee noted the positive development of a long￾planned conservation trust fund coming into place, but also pointed out that it probably is still several years away from disbursing local grants. Observations and Recommendations Based on findings, conclusions and progress towards targets described above, table 21 summarizes the observations (improvement opportunities) and recommendations (potential future actions) for Expected Result 4 in the Jamaica seascape. Table 21. Observations and Recommendations - Expected Result 4 Pedro Cays and Southwest Coast Observations Recommendations  South West Cay Special Fishery Conservation Area management plan under development  TNC prioritize support for C-CAM to further advance plan development  Bluefields Bay and Galleon St. Elizabeth management plans are both in need of review and possible refinement by management organizations and key stakeholders  Convene workshop with key partners and stakeholders to review plans, identify weaknesses, initiate refinements and develop a workplan to guide process (likely will take time beyond CMBP) 51  Progress made towards more effective patrolling and enforcement in multiple sites, but significant challenges remain  TNC should continue building capacity for patrolling, identifying weaknesses and gaps, and strengthening emerging enforcement programs  TNC strategized with C-CAM to try a different approach and hire a sanctuary manager  USAID should follow-up with TNC to assess if the new staffing approach is proving effective  Lack of clarity on who will lead Pedro Cays management plan implementation (some note NEPA has recently agreed to take this role)  Linked to the effort to secure plan approval, and request NEPA to clarify and broadly disseminate its leadership roles and responsibilities  Emerging formal partnership between C￾CAM and the Marine Police  Prioritize support for a MOU between C-CAM and the Marine Police, and share lessons from partnership with other fish sanctuary management organizations  There is redundancy of effort in that a slightly different version of the METT was applied during the same time by CMBP partners to fulfill requirements of different donors. Some also feel the METT is not the best tool for them.  Explore why some organizations feel the METT is not the best tool for them, and if there are more appealing alternatives. As appropriate, revisit and streamline the process for conducting the METT, and avoid duplication of efforts. Expected Result 3: Sustainable Fisheries and Sustainable Livelihoods for Fishers To what extent has CMBP met its targets under Expected Result 3: More sustainable fisheries sector? What implications does CMBP performance on Expected Result 3 have for achieving CMBP’s overall objective of reducing threats to marine-coastal biodiversity? Samana Bay Seascape (Dominican Republic) Summary Analysis of Findings Achieving Expected Result 3 has been challenging in the Samana Bay area for CMBP. Intended goals under Expected Result 3 have only been partially achieved and are not likely to be completed before the end of the project. Key achievements in the Samana Bay seascape include:  CMBP performance indicator 8 (number of person hours of training in natural resources management and/or biodiversity conservation) has exceeded its target.  Administrative capacity of fisher organizations has increased and will facilitate sustainable management of cooperative common funds.  Roundtable meetings between fishers and authorities are regarded as highly beneficial. 2  The Fisheries Sector Action Plan (FSAP) developed by the project was received positively by the Ministry of Environment and CODOPESCA. Key challenges that have limited progress towards program targets include:  Awareness of the need for more sustainable fishing practices has increased in Samana Bay; however, it is so far not enough to change the way local communities use marine resources.  The number of people with tangible and/or perceived increases in economic benefits from improved natural resource management and conservation (CMBP performance indicator 6, Number of people with tangible and/or perceived increases in economic benefits derived from improved natural resource management and conservation as a result of USG assistance) has fallen short of its target.  No fishers have applied new fishing technologies in Samana Bay.  Economic alternatives and market creation efforts are not achieving expected benefits. Observations and recommendations listed at the end of this section reflect improvement opportunities and potential future actions that may enhance CMBP performance in the Samana Bay under Expected Result 3. Progress in accomplishing CMBP performance indicators and targets Looking at CMBP’s performance indicator 8 (number of person hours of training in natural resources management and/or biodiversity conservation supported by USG assistance) has exceeded its target. Nevertheless, based on the information and data collected in this evaluation, including interviews and focus group discussions with fishers, CODOPESCA and the sub-grantee, it can be concluded that awareness of the need for more sustainable fishing practices is not enough to change the way local communities use resources. Most fishers know which gear types and fishing methods are least harmful to marine organisms and to ecosystems, and they are also aware of fishing bans for different species. However, as most of the fishers in the Samana Bay area are poor, they require viable income alternatives to be able to reduce fishing effort or to change the way they fish, including intensity, timing, and use of different gear. For performance indicator 6 (number of people with tangible and/or perceived increases in economic benefits derived from improved natural resource management and conservation as a result of USG assistance), CMBP falls short in achieving the targets. CMBP had a target of 226 men and 88 women with increased economic benefits; however, the real increase in income due to new opportunities is so far perceived only by a small number of people. Only 51 men and 23 women reported increased economic benefits during the 2016 socio-economic survey conducted by CMBP in Samana Bay. CMBP targets in Samana Bay for indicator 7 (number of fishers and others who have applied new technologies or management practices as a result of USG assistance), are 109 men and 5 women. According to TNC’s 2017 Monitoring and Evaluation Report, no fishers have applied new technologies in Samana Bay. Given the skepticism of most fishers, it will take much longer than expected to achieve the proposed gear adoption goal. Knowledge of sustainable and unsustainable fishing practices Most Samana Bay participants in interviews and FGDs agree that unsustainable fishing methods include the use of non-selective gill nets of inadequate mesh size. Other harmful fishing practices 3 identified by fishers, although not necessarily taking place in Samana Bay, include fishing with the use of poison, the use of licuadora (bottom trawl net), and spear fishing using scuba diving gear or compressors. Fishing in the nursery areas and the physical destruction of mangroves and coral reefs ecosystems, were also mentioned as unsustainable by the fishers. Among sustainable fisheries practices, participants identified methods with a low proportion of juveniles and by-catch, like hand line, free diving spear fishing, cast nets with adequate mesh size, and deep-sea fishing. Interestingly, although CODOPESCA says there is not enough information to establish the adequate mesh size, one of the fisher cooperatives indicated “the recommended mesh size for jack fishing (jurel) is 10 centimeters, and for shrimp fishing the recommended mesh size is 5 centimeters, with gauge 28 thread”. A stakeholder group identified the creation of no-take zones as a good practice, and stated that “gill nets should not be used in river mouths, areas near mangroves, and from the shoreline out to 300 meters”. For CODOPESCA, all artisanal fisheries that respect closed seasons and operate in areas where there are no large human populations, for example lobster and queen conch fisheries for local consumption, are sustainable. At the same time, CODOPESCA sees the use of licuadoras as the only method available that produces adequate shrimp catches, indicating that the damage to the sea floor by the use of licuadoras is non-significant since the areas where licuadoras are used are basically muddy bottoms not affected by this gear. Regarding the protection of parrot fish, even though considerable sale of parrot fish takes place in Samana, local catches of parrot fish are small. CEBSE focuses most of its actions towards parrot fish protection on restaurants and not so much on fishers. The ban on parrot fish was originally proposed by MARENA and is not fully supported by most of fishers and some CODOPESCA interviewees. Fishers say they have no alternatives to parrot fish fishing, even though they admit there are less parrot fish being sold, and acknowledge they see more of parrot fish in the water now. On the other hand, CODOPESCA does not fully support the ban on parrot fish, in part because parrot fish accounts for 40-60% of the country’s fish catch, making it impossible to control. CEBSE pointed out that the Navy, as another authority over the sea, could help to control unsustainable fishing, but Navy officials in Samana have limited knowledge about fisheries legislation and enforcement. There also seems to be lack of clarity among authorities and fishers, about which fishing methods are legal, and which are not. Progress in adopting sustainable fishing practices CMBP, through CEBSE, has promoted an exchange of unsustainable licuadoras for suripera nets for shrimp fishing, but this exchange is perceived as both a success and a failure, depending who is asked, and the exchange process is slow. There seems to be little interest among fishers to use the new gear, as most fishers consider the trials using suripera nets a failure, arguing that the environmental conditions in Samana Bay, such as winds and water currents, do not permit optimal functioning of suripera nets, and that suripera fishing effectiveness is lower compared with licuadoras. On the other hand, TNC, CEBSE and MARENA consider the suripera net trials as positive, and CODOPESCA opinions are divided. It appears that suriperas are cheaper to buy and maintain. The report on the viability of suriperas in Samana Bay, prepared by Mexican facilitators, contradicts the opinion of local fishers, and concludes that suripera nets are a viable option in Samana Bay, even though no side-by-side comparison was made with licuadoras in terms of catch per unit effort. TNC and CEBSE believes suripera nets can be locally made at a lower cost than 4 licuadoras and plans to continue promoting the adoption of suriperas and training CODOPESCA employees on how to use them, and has designed a fishing gear exchange pilot program. To increase the chances of adoption of suripera nets among Samana Bay fishers, the Evaluation Team recommends that, instead of a second visit by the Mexican facilitators as currently proposed, CMBP should consider the possibility of bringing a group of Samana Bay fishers to Mexico, to let them see firsthand the use of suripera nets. This group of Samana fishers can in turn become champions within their local fishers cooperative and promote the use of suripera nets themselves. Fishers also require support to buy or make more sustainable fishing gears and financial support while undergoing fishing gear switches. To increase adoption of sustainable fishing practices, fishers participating in new fishing gear trials indicated the need to receive a stipend to compensate for income loss; however, there is no allocation of funding in the CMBP budget for that. Fishers indicated they need to at least be able to cover any difference in income from the use of the new gear, until they learn how to properly use it. Some fishers mention that they understood that if they participated in new fishing gear trials CMBP would provide an economic compensation for their time and use of their equipment. Other complaints from fishers were the requirement from TNC to sign a document for the release of responsibility in their participation on fishing gear trails and that they have not received the report with the results of these fishing gear trials. According to TNC, the results of the trials were presented during fisherfolk roundtables in November 2016, and during a workshop in Santo Domingo that was attended by CODOPESCA and other stakeholders. Furthermore, TNC indicated the report was later handed out to the fishers who attended. Coordination among government agencies CMBP has promoted the coordination between the Ministry of Environment and CODOPESCA, which has improved, although it still needs improvement. CODOPESCA, despite being the Dominican Republic’s institution responsible for fisheries regulations, has limited personnel in the Samana Bay area. Some fishers mentioned that some CODOPESCA employees are not interested in enforcing sustainable fishing but instead are getting their “share”. Fishers said they follow CODOPESCA’s recommendations when they are presented with alternatives, unlike the Ministry of Environment that mostly talks about prohibitions. The Ministry conducts seizures of illegal fishing gear upon complaints. The Ministry collaborated with CODOPESCA to produce a closed seasons calendar for different marine species that was printed with CMBP funding. CODOPESCA and MARENA have different opinions about the parrot fish ban, and the Navy is currently providing fishers identification cards without CODOPESCA participation. The president’s “surprise visit” to the area, in which he made promises and donations to the fisher association in Sabana de la Mar, is perceived by the implementing partner as one of the reasons for difficulties in working with the fishers. The Sabana de la Mar Fisher Association was promised the reconstruction of their headquarters, a refrigerated truck, and other equipment without analysis of the problems or a business plan. Members of fisher associations that participated in the focus group discussion observed that government agencies are working more intensely and closer with communities in the area because of CMBP efforts. CMBP also initiated periodic Fishermen Roundtable meetings, with the idea of opening the dialogue between fishers and authorities and facilitate coordination, and to find solutions to challenges that arise between fishers and government authorities. The aim is that fishers will 5 continue to use the roundtable meetings on their own in the future, as most fishing groups see the Roundtable as very useful. Roundtable meetings are regarded by authorities and fishers in Los Cacaos and Sanchez as highly beneficial. Fishers in Sabana de la Mar, on the other hand feel the roundtable meetings are just another way for government authorities to control their activities. Strengthening fisher associations Other CMBP efforts to achieve more sustainable fisheries sector, such as strengthening fishers associations and cooperatives, have been more successful. CMBP has promoted organizational strengthening by helping cooperatives to develop purpose and future plans, and by educating fishers on the benefits of being a cooperative member. The administrative capacity of fisher organizations has increased and will facilitate the sustainable management of cooperative common funds. CMBP has provided fisher associations with financial management capacity support, by developing an invoicing system, helping the associations develop their financial and tax reports, and by contracting an auditing firm. Computers with accounting software were provided, as well as printers, and two people in each cooperative were trained in accounting. Fisher cooperatives and the sub-grantee agree that there is still a need to strengthen the fishers associations, including capacity building in the handling of catch and control and maintenance of a temperature-controlled supply chain. If fishers can obtain a better-quality product and sell it at a higher price, the assumption is that there would be less pressure on local marine resources, as fisher could gain more by selling smaller amounts of higher priced catch. These efforts can be successful and have positive impact on the resources, but it would likely be in a time frame beyond the project implementation, and would require additional external support, until the cooperatives become economically and technically sustainable. Business plans for tourism activities for Espiritu Santo and San Lorenzo cooperatives were prepared, and cooperative members have received training in ecotourism, tour guiding, team building, and English language, in preparation for new income opportunities. Diving gear and dive training was also provided to the San Lorenzo fisher association. The building the Women’s Association Artesanas de los Corozos uses was built with funding from CEBSE, and the group received an engine, supplies and tools for handicrafts, and training in the production of coconut handicrafts. Alternative sources of income Regarding CMBP’s activities for economic alternatives and market creation, the Women’s Association Artesanas de los Corozos are clearly not bringing the expected benefits. The production of crafts has stalled for months, and it seems that members of the association are no longer interested in producing crafts made from coconut, an alternative income activity that, according to TNC, started prior to CMBP with funding from the Esperanza Bank Foundation. The association expressed interest in crafts made from different material (e.g. conch), candle making and purses. Nevertheless, the members of women’s association are fervent defenders of the mangrove ecosystem, which is a main activity of the women group under CMBP. There is still a need for further alternative livelihoods training, including training in outboard engine maintenance, tuna fishing, deep-sea fishing, French languages, computer use, and fish processing, storage and export. The Women Association expressed during the interview they still 6 need further training in creating jewelry from materials other than coconut like conch, and other handicrafts. It appears that the Fishermen Association in Sabana de la Mar is the one in most need for support and could benefit from a closer relationship with the sub-grantee. This association expressed interest in developing ecotourism activities around the Magua municipality waterfall, and to use their boats for ecotourism. One of the obstacles for CMBP’s actions sustainability, identified by most respondents, is the lack of economic alternatives for local communities, especially in Sanchez and Los Corozos. Having economic incentives during closed fishing seasons, and increasing social security coverage for cooperative members would help increase compliance of closed seasons. Assessing changes in marine biodiversity and human well-being and potential to scale up CMBP, through TNC, has implemented AGRRA monitoring for coral reefs in Samana Bay every two years, and both CEBSE and CIBIMA (Centro de Biologia Marina, Universidad Autonoma de Santo Domingo) personnel have received training in AGRRA monitoring methods. For mangrove monitoring TNC is using drones and field visits, and the growth of mangroves in the reforested areas near Los Corozos, is measured with participation of the Women’s Association Artesanas de los Corozos. There is no monitoring of fishing resources in general, but the little catch data available in CODOPESCA has been provided at CMBP’s request. This data would be very useful, but unfortunately there is no long-term monitoring data. In the coral nurseries CEBSE monitors coral growth and the increase in associated fauna. All local groups claim that the fish abundance in Samana Bay is decreasing, and that the income of all community members depend on fishing resources. Potential for expansion to new areas Based on CMBP’s performance to date, the sub-grantee and local groups involved in the project believe that conservation efforts in Samana area should be consolidated before considering expansion to other areas. For example, interviewees indicated Yuna and Ozama rivers have environmental pollution issues. The government agencies also agree that there is still much work to be done in Samana Bay area, as there are other areas in the Samana Peninsula that require attention. All agree that there is still a need to strengthen fisher’s associations, and build their capacity in catch handling, so that fishers can obtain better product and sell it at better price, and potentially exert less fishing pressure. Additional requests from fisher’s associations include the donation of a boat to transport tourists required for kayak tourism expansion, and to continue to work with suripera experts from Mexico. When asked about other areas for future expansion, CODOPESCA mentioned Jaragua National Park, where there is very significant lobster overfishing, Montecristi National Park, where the most important coral reef in the Dominican Republic are located, in addition to more traditional fisheries, probably overfishing of ornamental species. MARENA also believes that Montecristi National Park is a good area for expansion, with attention to coral reef habitat. When discussing sustainability and a potential scale up of CMBP actions, stakeholders acknowledged the limited technical and enforcement capacity of the government institutions responsible for regulating the use of natural resources. CODOPESCA has only five ill equipped employees in the Samana Bay area, and only one of them is a formally trained technician. Fishers 7 also suggest that CODOPESCA could for example regulate the import of banned fishing gear to promote sustainability of the regulations. Observations and Recommendations Based on the results from both the introduction of suripera nets and the promotion of income alternatives with the Women’s Association Artesanas de los Corozos, the Evaluation Team recommends that a detailed analysis of the local community’s needs should be conducted, identifying interests and abilities prior to designing conservation actions to be implemented in partnership with these communities. Such a situational analysis is likely to improve the efficacy and effectiveness of any ambitious conservation action, and thereby ensure better adoption by stakeholders. Based on the opinion of different stakeholders, CMBP continued the support to Samana Bay area fisher cooperatives initiated under the previously USAID funded Environmental Protection Program. In a similar fashion, it is also fair to expect that without further support, action plans developed under CMBP will have little implementation, and as a result improvement in the health of marine biodiversity in the area will be harder to achieve. Based on findings, conclusions and progress towards targets described above, Table 22 summarizes observations (improvement opportunities) and recommendations (potential future actions) for Expected Result 3 in the Samana seascape. Table 22. Observations and Recommendations - Expected Result 3 Samana Bay Observations Recommendations  Achieving Expected Result 3 has been challenging in Samana Bay, with intended goals only partially achieved, and seem not likely to be completed before the end of the project  Funding for implementing the FSAP, and the management of the six new no￾take zones, should be secured.  Awareness of the need for more sustainable fishing practices has increased in Samana Bay; however, it is so far not enough to change the way local communities use resources  Instead of a second visit by Mexican facilitators, TNC should explore the possibility of bringing Samana Bay fishers to Mexico to learn the use of suripera nets  Roundtable meetings are regarded by authorities and fishers in Los Cacaos and Sanchez as highly beneficial  Roundtable meetings between fishers and authorities should continue to receive support from CMBP  No fishers have applied new fishing technologies in Samana Bay  Future trials with new fishing gear should be conducted side-by-side with traditional gear, to demonstrate the value of the former in comparison to the latter. This is also likely to increase acceptance of new gear by fishers  Fishers and government agencies complained about the time it takes for CMBP reports to be distributed  TNC could improve communication at all levels and reduce the time it takes for reports to be distributed. TNC could also consider organizing a workshop 8 with stakeholders to share information about projects in Samana Bay  Economic alternatives and market creation efforts are not bringing expected benefits  TNC and local paratners should conduct a detailed analysis of the local community’s needs, interests and abilities prior to designing conservation actions to be implemented in the area. A situational analysis is likely to improve the efficacy and effectiveness of conservation actions, and ensure better stakeholder adoption Grenadine Bank Seascape (Grenada and St. Vincent and The Grenadines) Summary Analysis of Findings The Evaluation Team concludes that CMBP only partially achieved Expected Result 3, sustainable fisheries and sustainable livelihood for fishers. Safety-at-sea and HACCP trainings were well received in multiple locations. That said, it remains unclear if, as desired, a true fishery effort shift to pelagic waters, as a means to relieve fishing pressure on nearshore resources, has yet occurred. Moreover, a lobster pilot project on Mayreau island failed, at least in its initial phase, even though the island’s fishers genuinely attempted to make it work. Key achievements in the Grenadine Bank seascape include:  Rapid Fishery Sector Assessment (RFSA) and subsequent development of a Fishery Sector Action Plan (FSAP) which helped guide and inform CMBP activities.  Exceeded performance indicator 8 (Number of person hours of training in natural resources management and/or biodiversity conservation supported by USG assistance) targets. A wide range of trainings were conducted on alternative livelihoods, new gear design pilot projects, and incentives to create an offshore fishery, far exceeding program targets for number or person hours of training.  HACCP training and equipment for cold storage of catches provided to over 50 fisherfolk. 70% of participants reported tangible economic benefits as a result of the activity, which responded to stakeholder needs and the Fisheries Sector Action Plan. Key challenges that have limited progress towards program targets include:  Although CMBP has led many trainings focused on sustainable fishing practices, limited changes in fishing pressure or threat reduction have actually been measured, with some stakeholders noting that unsustainable fishing practices are still common, particularly amongst Union Island fishers.  An ad hoc approach to selecting and facilitating fisher trainings led to inconsistencies in the training schedule or curriculum used across different sites.  Community leaders on some islands report that fisher associations are having a hard time getting organized, and thus most are still in an early stage of development.  TNC and USAID also identified the sudden closure of CARIBSAVE as a key challenge. 9 Observations and recommendations listed at the end of this section reflect improvement opportunities and potential future actions that may enhance CMBP performance in the Grenadine Bank under Expected Result 3. Strengthening fisheries policy and striving to move towards more sustainable fishing practices Since 2013, Grenada has served as the Caribbean Challenge Initiative (CCI) Secretariat. As such, there is strong interest across high levels in government to collaborate with programs like CMBP in order to expand and improve management of protected areas, strengthen fisheries policies and protect fisher livelihoods while simultaneously promoting more sustainable fishing practices. TNC, with consultant support, conducted a RFSA and subsequently developed a transboundary FSAP for the Grenadine Bank. These early steps helped identify priority actions and tasks that would support the mutually reinforcing goals of more sustainable fisheries and sustainable livelihoods for fishers. The FSAP therefore helped guide work under CMBP, including, but not limited to, trainings, outreach and education, and pilot programs described below. The Grenadine Bank seascape is dotted with a unique mix of fishers, water taxi operators, marine tour operators and visiting yachts, local populations and other stakeholders that exert competing demands on the fisheries sector. CMBP activities implemented under Expected Result 3 in both Grenada and St. Vincent and the Grenadines have achieved limited success to date. Key CMBP activities included an attempt to adopt lobster pots, a range of training and alternative livelihood opportunities, and replanting of mangroves in Ashton Lagoon. Safety-at-sea trainings, combined with supply of needed equipment (e.g. coolers, radios, lifejackets) aimed in part to incentivize local fishers to engage in pelagic fishing, and thereby decrease pressure on nearshore fisheries, especially areas near or within no-take MPAs across the Grenadine Bank. Training fishers and building fisher association capacity The interest in training and skills development is high amongst fishers across the Grenadine Bank seascape. Reported CMBP activities indicate that the life of project target of 600-person hours of training (300 person hours for men and 300 for women) has been exceeded. TNC has reported that 2,827 person hours, including 2,067 person hours for men and 760 person hours for women are already completed. Of note, most of the 2,481 person-hours of training, was delivered during year three of the CMBP. All trainings across the Grenadine Bank were directly linked to the promotion of alternative livelihoods, including safety-at-sea, HACCP for fishers and water taxi operators, ecotourism and apiculture training. However, despite the attainment of person hours of training targets, community leaders from Carriacou, Union Island and Mayreau noted many fishers still engage in unsustainable practices, though a description of the type and scale of such practices was not given. In acknowledging this challenge, some fishers suggested that there remains a need for consumer demand or other economic incentives to encourage more sustainable approaches. Stakeholder interviews across the Grenadine Bank suggest that CMBP has had an ad-hoc approach to selecting any given training, deciding when it would be applied, and where across the CMBP seascape. As a result, some interviewees described a lack of consistency in the training schedule or curriculum used across different sites. Dependence, at times on in-kind support and partnership from government agencies, was also cited by some as a challenge. In some instances, fishers indicated a desire for a particular training they learned about from counterparts based in other sites. Given this context, fishers were sometimes unsure of when they would receive the training or 10 equipment in their particular island. In addition, despite broad awareness of and appreciation for trainings, interviewees (fishers and others) commonly demonstrated confusion in differentiating training or equipment received from CMBP versus other programs. Some interviewees believe a good foundation for the development of active, functioning fisher associations has been laid, but there remains a need, as noted above, to enhance economic opportunities and assist in business development for alternative livelihoods (e.g. boat building) to make more sustainable practices viable. Interviewees praised projects that address local needs and integrate extensive local knowledge, thereby fostering stakeholder collaboration. On the other hand, most interviewees were unsure of any measure of capacity building of recently created fisher associations throughout the Grenadine Bank. TNC staff noted that, at this point in time, different fisher associations demonstrate significantly different capacity and maturity. Perceptions of sustainability For fishers interviewed across the Grenadine Bank, notably leaders of local fisher associations, sustainable fishing was described as allowing juvenile fish to age to maturity, versus unsustainable which means harvesting juveniles. These same fishers also identified sustainable fishing practices as those that maintain fish stocks and do not deplete them. Fishers generally understand why there are size limits, and know why there are regulations and restrictions within protected areas. Fishers also have an appreciation of pollution issues. Most fishers do not object to managed areas, if such areas are able to demonstrate benefits and help them transition to fishing more sustainably. Application of new approaches, technologies and equipment Program activities have led to partial fulfilment of targets associated with CMBP indicator 7, the number of fishers who have applied new technologies or management practices. For the 2017 target of 75 persons (48 men and 27 women), 32 males and 4 women indicated application of new technologies. Notably, the number of people applying new technologies disproportionately favors men. It would therefore be useful to prioritize gender mainstreaming when designing any remaining or future CMBP trainings. Of the beneficiaries applying new technologies or approaches to fishing, several indicated that opportunities grew directly from CMBP trainings and equipment distribution, namely the safety-at-sea-training and HACCP trainings, but not the lobster pot pilot initiative. Given that a significant proportion of interviewed fishers indicated interest in more sustainable technologies, it could be further motivation if opportunities were created for fishers to experience best practice approaches and sustainable technologies at demonstration sites within the wider region or internationally. One positive observation is that on Union Island the head of the local fisher association proudly showed the Evaluation Team the mangrove restoration area near Ashton Lagoon, including the location where bee hives would be soon set up and operational. Hundreds of mangrove seedlings have been planted, have successfully taken root, and are thriving. Four previous restoration efforts were unsuccessful in Grenada until the change in method from PVC pipes to bamboo tubes (biodegradable and safe for environment) as conduits to help plantlets take root. Plantlet survival rates are high compared to other efforts. Moreover, many community volunteers assisted this effort and some feel the trail through the area provides future ecotourism opportunities. Perceptions on economic benefits 11 In considering indicator 6, the number of people with tangible and/or perceived increases in economic benefits derived from improved natural resource management and/or biodiversity conservation, CMBP only partially met its set targets. By 2017, 40 persons (33 men and 7 women) out of a target of 70 persons (48 men and 27 women) indicated that CMBP activities resulted in tangible and/or perceived increases in economic benefits. This figure translates to 40% of the life of project targets (33% for men and 7% for women). Notably, several beneficiaries from across the Grenadine Bank suggested that economic opportunity is a key determinant of the viability of any approach towards more sustainable fisheries. The potential to achieve the life of project targets therefore requires action to improve tangible or perceived increases in economic benefits, particularly among women. Limited success in achieving sustainable fisheries Fishers no doubt have benefitted from CMBP trainings, and many interviewees noted that more training and alternative livelihood opportunities are needed in order to reduce threats to nearshore ecosystems. That said, key activities aimed at promoting sustainable fishing practices have either failed or not yet proven successful. For example, fishers, though interested and open to try an alternative, abandoned the lobster pot project, at least in its initial stage, owning to the fact that the trap design did not perform well, and two pots per fisher during the pilot was not enough to secure adequate catch. TNC reported that the lobster gear activity needed more time to follow up on the results of the pilot. Further, TNC realized it would take at least two years to address key issues, such as a more effective pot design, an affordable and environmentally sound source of bait, and a larger supply of pots. Given conditions at the time, TNC decided to discontinue the pilot and re￾start it under another project if funding became available. TNC reported that it perceives the lobster pot pilot on Mayreau to have achieved its objectives. Many fishers did express gratitude at receiving boat equipment (e.g. life jackets, radios, coolers etc.), combined with the safety-at-sea training, and have been using the equipment provided. However, the scale of any true effort shift in the fishery from nearshore to pelagic waters—a key goal of the effort—remains unmeasured and thus unclear. Some interviewees pointed to fishers using new equipment to fish offshore fish aggregating devices (FADs) that have been approved by other programs, and to giving up spear and lobster fishing after reef fisheries collapsed. At the same time, several fishers and other interviewees also noted that many local fishers have small boats that cannot operate safely on the open ocean. It is therefore possible that equipment aimed at creating an effort shift to pelagic waters may in fact be creating greater pressure on already stressed resources in the nearshore environment. Linking fisherfolk organizations and MPA authorities Interviews with all fisher association leaders—in Carriacou, Union Island and Mayreau—reveal community leaders who understand sustainability concepts well and recognize the need for a shift to more sustainable practices across the Grenadine Bank. Field-based observations suggest that fishers on Carriacou have had more success at organizing themselves than other nearby islands, and have also shown a greater willingness and emerging capacity to move towards a more sustainable fisheries sector. Leaders in the fishing community on Union and Mayreau islands, in contrast, noted that it is still difficult to get fishers organized. In Carriacou the Evaluation Team observed mutual respect between fisher leaders in the community of L’Esterre and SIOBMPA staff, SUGREN and TNC. Wardens described how communication with fishers that was 12 challenging in the past has improved significantly under CMBP, partly as a result trainings and regular interaction that occurs during patrolling. Moreover, high level government officials in Carriacou remain very interested in continuing to align fisheries with national development plans and regional conservation initiatives (e.g. Caribbean Challenge Initiative, CMBP), particularly in relation to building effectively managed MPAs from which the community benefits. Finally, some interviewees suggested that future trainings should focus on international maritime industries and standards of practice, so as to improve the possibility of Carriacou residents securing alternative livelihoods beyond local fishing. Assessing changes in marine biodiversity and human wellbeing, and the potential to scale up In general, interviewees, both fishers and other stakeholders, showed limited knowledge of any activities to measure marine biodiversity. Some mentioned measurements undertaken by an implementing partner for AGRRA studies. Monitoring activities undertaken by a previous partner, CARIBSAVE, may have been discontinued with their departure from CMBP. No interviewees provided detailed responses regarding the impact of CARIBSAVE’s sudden closure. Prospects for expansion—described as continuing to build fisher association capacity to transition to more sustainable fishing practices while simultaneously protecting livelihoods—are perceived by leaders of these associations as good. This perception is not necessarily rooted in the progress, or lack thereof, that these associations alone have made. Rather, leaders in the fishing community, particularly in Carriacou but also Union island and Mayreau, recognize and acknowledge that CMBP is helping take the management and conservation of marine biodiversity in the right direction. In this sense, fishers and others welcome a continuation of the program. Many feels that a foundation for improved resource management and greater collaboration across the Grenadine Bank has been laid, and there is now the need to meet the expectation that MPAs will produce benefits and enhance livelihoods for fishers. Funding often proves to be the challenge for successful expansion, and it is also important to fit these projects into overall national plans and strategic development goals in both countries. Seamoss farming was spoken of as a real livelihood alternative on Mayreau. The provision of an ice cream maker for the sea moss production facility was also mentioned as a good incentive to discontinue unsustainable practices. Since there is no fisheries processing center in the islands, several fishers and others noted that the ice boxes supplied for fishers under CMBP improved handling and management of catch. Communication and education Some government officials and MPA authorities noted that, at the outset, they did not have the right approach to engage fishers. Initially, wardens just said, “this is a no fish zone”, and people felt disenfranchised and were not willing to comply with rules or support the effort to protect and conserve marine biodiversity, including locally valuable fish species. Education, interviewees noted, is key, associated with proper leadership within fisher associations. Of note, CMBP supported the hiring of an outreach officer at SIOMPA to increase community awareness of conservation issues. Stakeholders must be engaged in a meaningful way in order to achieve buy￾in to marine conservation goals and targets. Some pointed out that greater efforts to reach the public and tourists more broadly, including through the use of signage, are still needed and would be helpful. 13 Observations and Recommendations Based on findings, conclusions and progress towards targets described above, table 23 summarizes observations (improvement opportunities) and recommendations (potential future actions) for Expected Result 3 in the Grenadine Bank seascape. Table 23. Observations and Recommendations - Expected Result 3 Grenadine Bank Observations Recommendations  Disproportionate number of male fishers participating in training programs  Prioritize gender mainstreaming for remaining CMBP training courses  Unclear results of desired fishery effort shift from nearshore to pelagic waters and attempts at changing gear failed (e.g. lobster pots)  Design better methods for measuring changes in fishing pressure at key sites, and make some effort to assess in remaining program time  Re-engage fishers on gear change design ideas, as interest remains high to try  Motivating fishers to adopt sustainable practices could be influenced by opportunities to learn from others who have realized greater success (e.g. effort shift and gear changes)  Support exchanges that allow fishers to learn sustainable technologies and approaches, and how to overcome challenges, from other regional or international sites  Despite significant training hours, some fishers still engage in unsustainable practices with little incentive to change behavior  Apply lessons learned to better understand failures and develop new incentives for behavior change (e.g. how to influence market demand)  Fishers on Carriacou appear to have had more success than fishers on other islands at organizing/building capacity  Bring regional fisher association leaders to L’Esterre for a strategic planning and networking opportunity for fishers  Success in cultivating and out planting mangrove seedlings in the Ashton Lagoon area, Union island  Document successful approach (e.g. switch from PVC to bamboo) and lessons learned for application, as needed, in other CMBP seascapes North Coast of Hispaniola Seascape (Haiti) Summary analysis of findings The evaluation team concludes that Expected Result 3 related to more sustainable fisheries sector is partially achieved in the marine and coastal areas of the North Coast of Hispaniola seascape. Key achievements of in the North Coast of Hispaniola seascape include:  Performance indicator 6 (number of people with tangible and/or perceived increases in economic benefits derived from improved natural resource management and conservation) has met its target.  Performance indicator 8 (number of person hours of training in natural resources management and/or biodiversity conservation) has exceeded its target. 14  A rapid assessment of the legal framework for an enforcement unit for the Protected Area of Managed Natural Resources of the Three Bays (PA3B) was carried out, and discussions were conducted with the Ministry of Environment, the National Agency for Protected Areas (ANAP), and local mayors, which led to an agreement on supporting un-armed park wardens.  Many of the people involved in interviews and focus group discussions (FGDs) declared themselves willing to lend a strong hand for the defense of sustainable fishing practices, and to the enhancement of the marine and coastal ecosystem.  ANAP considers the PA3B management plan as a good reference to structure the development of the marine protection for other needed areas. Key challenges that have limited progress towards program targets include:  The FY17 target for CMBP indicator 7 (number of fishers and others who have applied new technologies or management practices) was set to 0, a target that seems inappropriate for the third year of the program implementation.  A pilot study designed to reduce juvenile catch using alternative fishing gear that was originally planned for year 3 has not been developed. Observations and recommendations listed at the end of this section reflect improvement opportunities and potential future actions that may enhance CMBP performance in PA3B under Expected Result 3. About half of the planned activities for the North Coast of Hispaniola Seascape listed in the Annex B consolidated CMBP revised Workplan, have been completed, including conducting a feasibility assessment for the implementation of a pilot fishing gear swap program, and a rapid assessment of the legal framework for an enforcement unit for PA3B. According to CMBP’s revised FY3 plan, activities pending include wide distribution of the Fisheries Sector Action Plan (FSAP), development of a feasibility study for the implementation of a pilot oyster cultivation project, a pilot study to test reduction of juvenile catch using alternative fishing gear, and drafting a conservation agreement for the designation of the first coral and fish reserve in Haiti. Partially completed activities include expansion of the apiculture project to other communities, and the establishment of a community enterprise for ecotourism services. CMBP is unique in Haiti in that it is the only comprehensive marine biodiversity conservation program with focus on MPA management, METT tools, supporting ANAP in the implementation of the MPA, and the possibility for exchanges and cooperation with other Caribbean countries. This has created a lot of interest from both communities and authorities, at the local and central level, and the National Agency for Protected Areas considers the PA3B management plan a good reference to structure development of marine protection for other needed areas. It is important to continue the consolidation of the program. Progress in accomplishing CMBP performance indicators targets CMBP’s Year 3 Monitoring Report indicates that the target for indicator 6 (number of people with tangible and/or perceived increases in economic benefits derived from improved natural resource management and conservation), is nearly achieved, with 49 people indicating receiving economic benefits, versus a target of 50 people. The target for indicator 7 (number of fishers and others who 15 have applied new technologies or management practices), for Haiti in FY17 is 0. A target of 0 seems inappropriate, since by the third year of the implementation the program should be able to demonstrate that at least some progress towards targets and results for this indicator. Furthermore, since the number of individuals that apply new technology is a key indicator to show how effective the numerous awareness and training activities delivered have been, the Evaluation Team recommends that, as much as possible, the data for this indicator should be disaggregated between regions with intensive and low intervention. This disaggregation will allow CMBP to evaluate how successful the training and awareness activities are in comparison with areas where intervention did not take place. It is further recommended that the pilot fishing gear swap program is implemented as soon as possible, to allow for learning and improving of CMBP’s implementation effectiveness and impact. CMBP has exceeded its targets for indicator 8 (number of person hours of training in natural resources management and/or biodiversity conservation) in the North Coast of Hispaniola seascape. Nevertheless, as it was pointed out before, the format in which these results are reported in CMBP’s Performance Monitoring Plan (PMP) and year-end progress reports are not conducive to understanding how trainings are strategically designed and what their respective expected outcomes have been. The annual report does not provide enough intelligible information on either the training program or awareness activities conducted, and there is no indication on whether and how the outcomes of these activities were regularly measured. It is therefore recommended to develop a clear and systematic plan for North Coast of Hispaniola seascape awareness activities for the wide public, as well as for the training program targeting the fisher associations. To be able to assess the effectiveness of training sessions provided, TNC should document its training strategic plan. The training plan should explain clearly the objectives, targets and expectations of the training activities to be implemented. Moreover, TNC should produce training evaluation reports, documenting what has been completed or not, and the reasons why. It is advisable that for each training activity TNC should have plans and reports. Progress in adopting sustainable fishing practices According to TNC’s ’Annex B consolidated CMBP revised workplan, a feasibility analysis for the implementation of a fishing gear swap pilot program was conducted by TNC. The feasibility analysis proposed a pilot, aimed at testing the reduction of juvenile catch using alternative fishing gear. That pilot was planned for year 3 of the project; however, the pilot activities including buying out small-mesh beach seine nets and providing fishers with larger-mesh gill nets, had not started by the time the evaluation team collected this information. According to TNC’s year 3 prioritization document, this pilot test should include 1) the identification of a small number of fishers willing to participate; 2) buying out their small-mesh beach seines and providing them with larger-mesh gill nets; 3) identification of the species to be targeted; 4) documentation of the catch per unit effort and the proportion of juveniles caught; and, 5) provision of a financial guarantee to compensate fishers for loss income. It is crucial that this sustainable fishing practices pilot starts without any delay, to provide lessons learned and eventually lead to the implementation of a full￾scale program. In relation to the increase in coverage and support for no-take fish sanctuaries, the Evaluation Team could not find evidence of the drafting and signing of a marine conservation agreement between fisher associations and local municipalities that was planned for year 3 of CMBP. 16 Nevertheless, TNC carried out a rapid assessment of the legal framework for an enforcement unit for PA3B and discussions were conducted with the Ministry of Environment, ANAP and local mayors to agree on supporting un-armed park wardens. This rapid assessment provided guidance on how current laws allow for effective park governance and surveillance, indicating the need to secure an authorization from ANAP and the Ministry of Environment to hire park wardens, and include local mayors and administrative councils in the management and surveillance of the park. ANAP indicated it is open to hiring park wardens under the Ministry’s payroll when CMBP ends, provided a proper authorization process is in place. Knowledge of sustainable and unsustainable fishing practices Creating awareness of sustainable and unsustainable fishing practices has been a major task of CMBP. All interviewed groups in the North Coast of Hispaniola Seascape—which included government officials, the sub-grantee, TNC, and fishers and other community stakeholders— identify sustainable fishing practices as those in which the communities have access to a large quantity and quality of fish in the long-term. Furthermore, all groups indicate that a sustainable fishery is one that is regulated by identifying fishing and non-fishing seasons, fishing practices that only allow catches of large fish, using a large mesh net, and a fishery that considers the protection of mangroves and coral reefs. By contrast, these four groups identified unsustainable fishing practices as those using non-selective fishing gear that catches both large fish and small fish with small mesh net, fishing with the use of compressors, and fishing during closed seasons for different species. Divergent opinions were observed among the fishers as to whether the use of long lines and spear guns, or fishing with light should be considered as unsustainable practices, although most of them tend to consider these methods as being more negative than positive. It is notable that many of the people that participated in interviews and FGDs declared themselves willing to lend a strong hand for the defense of sustainable fishing practices, and to the enhancement of the marine and coastal ecosystem. It would be desirable that TNC identifies at least two specific areas in the seascape to concentrate its efforts, to be able to measure the differential impact reached as per the intensity and the quality of the interventions. Fisher association capacity strengthening and the promotion of sustainable practices CMBP has contributed to strengthen local fisher associations’ capacity as expressed by the sub￾grantee, fishers and stakeholders in interviews. Participants identified CMBP’s support to fisher associations capacity via training sessions on environmental education, conflict management, and leadership, and the support for legal registration of fisher associations. In terms of measuring changes in local fisher associations’ capacity over time, FOPROBIM measures the change as the number of initiatives taken by the fisher associations. The interviews and FGDs held with PA3B fishers showed an increasing motivation towards supporting sustainable fisheries, thanks to the awareness raising sessions conducted by FOPROBIM under CMBP. These sessions covered thematic discussions about the value of and the importance of sustainable use of coral reefs, seagrasses, mangroves and fisheries. It is recommended that TNC devotes more resources in its remaining time to awareness sessions on those topics to reach more fisher associations and other protected area stakeholders. TNC also trained eight environmental facilitators who were later hired as park wardens. 17 To mitigate dependency on fisheries resources, TNC is promoting apiculture activities and is working with fishers to help them produce honey, and to market for their products. ANAP on his part has recently assigned a representative in the park, to harmonize interventions and identify coercive measures for prohibiting inappropriate fishing practices. TNC is working on a municipal bylaw with the mayor of Caracol to discourage unsustainable fishing practices, and is also working on an agreement with fisher associations to provide them new engines for the boats they received through the “Food for the Poor” program. Assessing changes in marine biodiversity & human well-being TNC used the AGRRA methods, through funding from the Inter-American Development Bank (IDB), to develop an ecological baseline and to measure changes in marine biodiversity as a result of CMBP interventions. It is expected that this same method will be used to measure changes in marine biodiversity later. TNC conducted an assessment on the changes in mangrove cover in the park, and reported positive and negative changes in different sectors of the intervention area. Reductions in mangrove cover are mostly due to factors out of control of the program, such as people cutting mangroves for charcoal and free ranging livestock. When asked, government officials, stakeholders and most of the fishers interviewed indicated not being aware of how changes in marine biodiversity are measured by CMBP. Some fishers mentioned having observed recent measurements done by CMBP on the size of fish captured and the nets used by the fishers. FOPROBIM indicated that the study mentioned by these fishers would be used as baseline for comparing fish measurements after the gear swap pilot activities in the northeast Haiti area of Madras. It is too early to demonstrate the impact of CMBP in achieving its overall objective of reducing threats to marine-coastal biodiversity in North Coast of Hispaniola seascape. In fact, it is not yet clearly defined how CMBP will measure the reduction in threats to marine coastal diversity, how frequently it should be measured, and through which indicator. According to TNC, a reduction in threats to marine and coastal biodiversity is not directly measured by the program. Despite the lack of quantitative information, fishers from Bord de Mer de Limonade mentioned, during focus group discussions, having observed a reduction in mangrove cutting as an outcome of CMBP’s awareness activities. More and more local authorities, such as Three Bays area mayors, have expressed support for the protection of coastal marine biodiversity, and the mayor of Caracol is working on producing a municipal decree to regulate fishing practices in the area. As for changes in human well-being, FOPROBIM mentioned a socio-economic survey conducted by TNC as part of CMBP, particularly related to apiculture activities. According to TNC, the survey was used to develop the socio-economic baseline of the potential program beneficiaries. Corrective measures and technologies Among the corrective measures or technologies to mitigate or discontinue unsustainable practices, interviewees from TNC, local authorities and fisher associations mention that ANAP recently put in place a committee of local stakeholders for the PA3B (Comité d’Appui), and FOPROBIM has implemented environmental education activities. The fisher organization OPPBL also indicated that CMBP has provided support in planting of mangroves and has helped them with organizational strengthening. TNC mentioned having regularly provided recommendations to 18 local authorities to stress corrective measures, and CMBP has hired wardens to collect information and report on illegal practices observed in the park. Obstacles to progress ANAP acknowledges the possibility that some fishers may be reluctant to discontinue unsustainable fisheries practices because they fail to see immediate benefits of adopting new techniques. According to ANAP, fishers would like to see fast results and quick changes in revenues, but these changes are hindered by complex bureaucratic procedures, especially with government entities. For fisher associations, the gap is related to the limited economic alternatives, and the lack of adequate fishing equipment, especially large mesh nets, to eliminate the use of small mesh nets. TNC identified the low capacity of government agencies as a primary obstacle to scale up CMBP pilot activities. According to the sub-grantee, lack of government funding is one of the main limiting factors. Other stakeholders mentioned that government officials often take part in meetings, but the lack of resources prevent them from following-up on critical decisions. TNC also recognized that, besides government limitation, it may be premature to gauge government support to scale up pilot activities that have recently started. Observations and Recommendations Based on findings, conclusions and progress towards targets described above, table 24 summarizes observations (improvement opportunities) and recommendations (potential future actions) for Expected Result 3 in the North Coast of Hispaniola seascape. Table 24. Observations and Recommendations - Expected Result 3 Three Bays Observations Recommendations  A completed feasibility analysis for a fishing gear swap pilot program proposed a small, six-month pilot  The pilot test for fishing gear swap should be implemented as soon as possible, to allow for learning and improving implementation effectiveness and impact  Training activities lack strategy and the way impact is reported does not allow for learning and impact evaluation  TNC and local partners should develop a clear and systematic plan for awareness and training activities to be able to evaluate effectiveness beyond the number of person hours  Park wardens are currently being paid with CMBP funding  TNC should make efforts to ensure ANAP hires park wardens under the Ministry’s payroll when the CMBP ends Pedro Bank and Southwest Coast Seascape (Jamaica) Summary Analysis of Findings The Evaluation Team concludes that CMBP in Jamaica has only partially achieved Expected Result 3, sustainable fisheries and sustainable livelihood for fishers. Program implementation has seen mixed results, with a clear lack of performance under indicator 6 (Number of people with tangible and/or perceived increases in economic benefits derived from improved natural resource management and conservation as a result of USG assistance), and indicator 7 (Number of fishers 19 and others who have applied new technologies or management practices as a result of USG assistance). That said, numerous trainings have been well received, and the emerging partnership between C-CAM and the Jamaica Defense Force shows promise and may afford lessons learned for other fish sanctuary managers. Key achievements in the Jamaica seascape include:  Rapid Fishery Sector Assessment (RFSA) and subsequent development of a Fishery Sector Action Plan (FSAP), which helped guide and inform CMBP activities.  Exceeded performance indicator 8 (Number of person hours of training in natural resources management and/or biodiversity conservation supported by USG assistance) targets. A combination of trainings, most notably the Campaign for Conservation, helped build organizational capacity of the Caribbean Coastal Area Management Foundation (C-CAM), Bluefields Bay Fishermen’s Friendly Society (BBFFS) and other conservation NGOs in Jamaica.  Trainings and innovative outreach events led by C-CAM educated Pedro Bank fishers on sustainability concepts and provided tools to put new knowledge into practice.  CMBP funds supported the hiring of C-CAM staff and purchasing a patrol boat, gear for wardens and computers. These investments increased patrolling and outreach efforts in and near the South West Cay Special Fishery Conservation Area (no-take zone).  CMBP funds supported the hiring of BBFFS staff. These investments increased patrolling in the Bluefields Bay Special Fishery Conservation Area (no-take zone) and local outreach.  An emerging partnership between C-CAM and the Jamaica Defense Force Marine Police have helped advance programs in the remote Pedro Cays, and show promise for further capacity building and conservation action. Key challenges that have limited progress towards program targets include:  Unclear what level of impact training and outreach programs at the Pedro Cays have had on getting fishers to adopt sustainable practices and thereby realize a reduction in threats to key species and overall marine biodiversity (i.e. trainings do not necessarily equate to measurable changes in behavior).  Limited organizational and technical capacity of the BBFFS to administer anything other than small, fixed-price grants. Limited capacity of BREDS – Treasure Beach Foundation to receive and administer even a small grant.  Needed support from key government agencies is often slow moving, even when high level officials in these agencies express interest in CMBP activities and progressive fisheries legislation and policy ideas.  Communication about CMBP from the implementing partner and the sub-grantee is perceived by many in government as ad hoc, and needs improvement. Observations and recommendations listed at the end of this section reflect improvement opportunities and potential future actions that may enhance CMBP performance in the Pedro Cays and the southwest coast under Expected Results 3. Strengthening effective fisheries policy and building a network of no-take fish sanctuaries CMBP facilitated a RFSA and subsequent development of the Fisheries Strategic Action Plan for Jamaican Seascape: Pedro Bank and South West Coast (FSAP) as a first step to achieving a more 20 sustainable fisheries sector. The FSAP was developed in a highly participatory manner, with visits to numerous fishing communities in Pedro Cays and along the southwest coast of Jamaica. The FSAP provided a roadmap to help CMBP partners and beneficiaries identify and advance priority actions, build local marine resource management capacity, and work with fishers to increase adoption of sustainable practices. The strategic plan also aims to foster compliance with recently established no-take zones, legally designated as “Special Fishery Conservation Areas” or “SFCAs” but also commonly known as fish sanctuaries. The scope of the FSAP included the Pedro Cays and the southwest coast (from Whitehouse in Savanna-la-Mar to Great Bay, St. Elizabeth). This was an important consideration for CMBP given that the Bluefields Bay SFCA is managed by a local fisher association, and fishers, in many respects, are also de facto wardens at Galleon St. Elizabeth SFCA. As noted in the Expected Results 4 section above, C-CAM has taken on responsibility for management of the South West Cay SFCA, located in the Pedro Cays. No formal fisher association exists at the Pedro Cays, as fishers consistently turn over, come from many areas along the south coast of Jamaica and represent different local associations. C-CAM also manages three other fish sanctuaries, including Galleon Harbor, Three Bays and Salt Harbor. Given that most CMBP support went to C-CAM, and to a lesser degree to fisher associations like BBFFS, activities under Expected Result 3 in the Jamaica seascape largely focused on organizational capacity building (operational and institutional), trainings, and undertaking outreach activities to various stakeholders (mostly fishers) to raise awareness and promote sustainable fishing practices. Following a sanctuary wide capacity assessment and management effectiveness evaluation using an adaptation of the World Bank’s marine version of the METT (supported by CMBP and briefly described below), representatives from each fish sanctuary management organization gathered for a weeklong professional development workshop. Participants collaborated closely to develop minimum performance standards for four priority management activities—human resources, enforcement operations, biophysical monitoring and financial management. In addition, each organization closely reviewed its evaluation results, identified needed performance improvements and began to map out action steps forward that would build individual and organizational capacity and, over time, improve management at individual sites which, collectively, make up the Jamaica Fish Sanctuary Network. Training and capacity building of C-CAM, BBFFS and fishers A combination of trainings, described below, illustrate how CMBP has already exceeded its life of project target under indicator 8 (number of person hours of training in natural resources management and/or biodiversity conservation), with 1,812 person hours completed against a target of 1,064 (1,175 for men and 637 for women). Of note, more than a third of the total number of persons trained were women, demonstrating that CMBP interventions are contributing to an increase in the participation of women in the fisheries sector. More specifically, CMBP funds have contributed to the employment of women as patrol wardens, mid-level staff persons, and managers at both C-CAM and BBFFS. As noted in the Jamaica Expected Result 4 section, C-CAM staff and some fishers received trainings in biophysical monitoring and a variety of safety oriented issues (CPR, first aid etc.), and C-CAM has facilitated creative and innovative outreach events to engage and educate fishers in 21 sustainability concepts. TNC and C-CAM jointly led a training on the lobster life cycle, combined with distribution of calipers to measure lobster size upon catch, thus encouraging fishers to throw back under-sized catch. And like in other seascapes, fishers, as well as some C-CAM staff, received important safety-at-sea training, which, interviews revealed, is appreciated and perceived by many as very helpful. C-CAM incentivized fisher participation in these trainings by providing participants with coolers, GPS units and solar flash lights. Several interviewees noted that important and well received trainings took place out in the Pedro Cays in 2017, including a training in how to harvest lionfish. Of note, interviewed fishers expressed interest in selling lionfish. TNC plans a renewed campaign to create a lionfish market in the remaining CMBP time period, but needs to study the supply chain first. Finally, TNC held a FishPath workshop, but the Fisheries Division has yet to commit to using this kind of decision-support tool for Jamaica’s lobster fishery. The focus group discussion with the BBFFS, which manages the Bluefields Bay SFCA, revealed that trainings have been well received by this group, but have not yet led to the development of pilot programs or the creation of alternative livelihoods, with the notable exception of people hired as staff at BBFFS. Specifically, CMBP support improved staffing capacity, patrolling and outreach efforts to the local community. Interviewees highlighted key benefits of the Campaign for Conservation workshop, particularly a better understanding of how to disseminate information to the community, as well as the relevance of, and interest in, ecotourism development and apiculture trainings. TNC may explore ecotourism as a pilot, and plans to conduct apiculture training in Bluefields. BBFFS staff have developed a marketing plan for the area, however, implementation of this plan will require funding. The Evaluation Team did not observe any apiculture pilot projects in the area; however, many individuals in the BBFFS focus group discussion expressed interest in this activity. Finally, interviewees suggested that application of the marine METT helped increase understanding of how to prioritize and conduct critical management activities in the fish sanctuary. Perceptions of sustainability Fishers described sustainable fishing practices as those that allow longevity of fishery stocks for their grandchildren. These same interviewees further described the concept of sustainable fishing as good/using permissible equipment and practices versus unsustainable fishing as bad/using illegal practices etc. Other interviewees noted the implications of climate change and land based pollution as contributing factors to declining fish stocks, and overfishing as negatively affecting fish getting to maturity. TNC and C-CAM were less confident in the knowledge of fishers to respond to such concepts of “sustainable” or “unsustainable”. BBFFS is conducting outreach in schools and the wider community to promote/transfer knowledge of sustainable fishing practices. Distribution of useful tools with the aim of improving sustainable practices Regarding indicator 7 (number of fishers and others who have applied new technologies or management practices), CMBP reports 27 people (25 men and 2 women) applying new technologies. With a life of project target of 40 persons, this appears to be a somewhat successful intervention. This is largely due to, as noted above, C-CAM’s distribution of lobster calipers to Pedro Cays fishers who attended a workshop focused on sustainable harvest and management of lobster. However, the Evaluation Team did not receive any additional feedback or evidence during interviews to demonstrate that fishers are indeed using the calipers to determine if lobster is mature enough to keep, or should be returned, and if the proportion of undersized lobster in the catch has 22 changed due to use of the calipers. As such, reporting on indicator 7 is limited in actually demonstrating 1) application of new technologies and, perhaps more importantly, 2) change in behavior to more sustainable practice as a result of having the technology available. At this stage it is therefore unclear as to whether this technology has fostered more sustainable practices and, if so, has made a significant contribution to reducing threats to marine biodiversity. No application of new technologies was reported in Bluefields Bay or anywhere else along the southwest coast. Despite significant achievements in the number and type of trainings and the number of persons who attended, TNC reporting indicates that CMBP failed to meet indicator 6 targets (number of persons with tangible and/or perceived increases in economic benefits derived from improved natural resource management and/or biodiversity conservation). The target is premised on the notion that when people receive tangible economic benefits they are more likely to value and support these activities into the future, well after the project ends, creating both long-term societal benefits and durable conservation impact. It was targeted that 54 persons (32 men and 22 women) should indicate having tangible or perceived benefits. However, only 36 people (30 men and 6 women) indicated economic benefits from improved management of natural resources. Beneficiaries of the aforementioned trainings made significant mention of the ongoing need for improved and alternative economic opportunities in order to maintain motivation to support sustainable fishing practices. Of the 30 men and 6 women who indicated tangible and/or perceived increases in economic benefits derived from improved natural resource management and/or biodiversity conservation, the majority were male fishers who benefitted from initiatives carried out in the Pedro Cays for sustainable lobster and lionfish harvesting practices. This suggests additional gender mainstreaming, particularly in relation to the availability of training opportunities, is warranted. Further, as highlighted above, surveys indicate a mixed response to adoption of new technologies or management practices. Despite training and promotion of more sustainable practices, many interviewees noted that fishers still engage in unsustainable practices within the target seascapes. In fact, some noted that fishers recognize some of their behavior is unsustainable, but that awareness alone does not translate into changes in actual behavior. Moreover, there is a disconnect between how fishers’ individual actions affect the bigger picture. As such, there remains a need to pay close attention to the identification and selection of strategic training programs and outreach activities which promote skills development opportunities for fishers, reinforce sustainability concepts and practices, and advance gender mainstreaming to create capacity building opportunities for both men and women. Capacity development and ongoing needs CMBP has facilitated hiring of additional wardens. This allows for three persons on any shift, thereby making the surveillance operation more effective. Financial support for regular patrolling at South West Cay and Bluefields Bay remains a priority. It was also noted that funding is required to maintain additional wardens as a stipend from the Fisheries Division is only able to support two workers per shift. Current needs include uniforms; training in business planning/development training; GIS support to demarcate boundaries; development of a lionfish marketing and sales plan; and support for a communications campaign to bring about social and consumer change. Regarding monitoring, most interviewees in Bluefields Bay made general reference to work done by CARIBSAVE, but were unfamiliar with measures under CMBP. TNC requested that C-CAM 23 staff develop a tool/protocol, which could be used by wardens, for monitoring sea turtles and birds. However, TNC did not receive a work product from for more than 18 months. Nascent signs of progress Notwithstanding the challenges described above, a few fishers and others expressed support for increasing coverage of the South West Cay SFCA. It was reported during some interviews that NEPA has not yet fully demarcated the official boundaries, and still needs financial and technical support to accomplish this task. A number of fishers expressed appreciation for the lobster workshops, and stated verbally that some fishers are indeed using the calipers to measure their catch, although no other evidence of such use was obtained. Moreover, interviewed fishers talking about workshops, safety-at-sea trainings, and attempts to “keep the fish around for the future”, spoke positively of CMBP, understand the support comes from the United States, and expressed hope that it would continue. The BBFFS manager and wardens demonstrated broad understanding of fundamental fisheries sustainability concepts. Wardens reported that fishers say they are catching more fish just outside the fish sanctuary, and fishers who swim through the sanctuary are seeing bigger fish. This success is attributed directly to the increase in patrols that have been enabled by CMBP. At the same time, wardens noted that patrolling will only become truly effective when BBFFS has the capacity to conduct operations 24 hours a day, seven days a week. Other BBFFS staff suggested that an increase in seabird sightings in the last few years provides further evidence of slowly recovering fisheries. Although BBFFS staff thinks a “spillover effect” is already occurring, during interviews staff acknowledged that biophysical monitoring is still needed to demonstrate if this is indeed true. Building partnerships, improving communication and ramping up Finally, it is important to note that C-CAM, building on its participation in Rare’s Campaign for Conservation training, designed and has led a multi-faceted program at South West Cay and nearby waters to patrol and enforce a no-take zone, educate and train fishers in sustainable practices, and build partnerships that help increase resource management capacity. Moreover, many of the training activities C-CAM offers have been implemented with the support of various government partners, such as the Jamaica Defense Force (JDF)-Coast Guard and the Fisheries Division. It is noteworthy, as described in the Jamaica Expected Result 4 section above, that collaborative partnerships forged between C-CAM and national agencies (e.g. Marine Police, Coast Guard and Fisheries Division) have helped facilitate trainings for fishers and an increase in marine patrols. Notwithstanding C-CAM’s evolving relationship with JDF, it is also a notable finding that many government agencies say communication about CMBP has been ad-hoc at best. Conversely, TNC and others note that communication and coordination with government is often a major impediment to efficient and timely implementation of the program’s activities. Thus, it appears there is significant room for improved communication and coordination between conservation leaders and government in Jamaica. If not addressed, this will likely have negative implications relative to achieving CMBP’s overall objective of reducing threats to marine-coastal biodiversity. Advancing campaigns that target consumers Some interviewees noted that the public consumer must be targeted more to complement efforts to discourage unsustainable practices, as fishers will supply whatever demand is present. Better crafted environmental campaigns are also needed, including targeting consumer demand through 24 innovative marketing campaigns. Advocacy groups need to improve communication and leverage relationships to get more done faster, as no single entity can do it alone. Observations and Recommendations Based on findings, conclusions and progress towards targets described above, table 25 summarizes observations (improvement opportunities) and recommendations (potential future actions) for Expected Result 3 in the Jamaica seascape. Table 25. Observations and Recommendations - Expected Result 3 Pedro Cays/Southwest Coast Observations Recommendations  Lack of clarity on whether distribution of calipers to lobster fishers has led to changes in fishing practices, and in the proportion of undersize lobster in the catch.  Revisit how best to measure changes in fisher behavior, relative to this specific pilot, in remaining CMBP time, and measure the proportion of undersize lobster in the catch of both fishers that received or did not receive calipers  Outreach and anecdotal evidence suggests it remains difficult for fishers to take up sustainable practices if there is not an economic incentive  As planned, TNC should study whether or not an adequate supply of lionfish can be harvested at the Pedro Cays and work to create a market share on the mainland  Limited capacity for BBFFS to administer large grants, yet strong interest in leveraging grant resources to develop self-sustaining enterprises (e.g. ecotourism, apiculture)  Consider specialized financial management training for 1-2 BBFFS staff persons in remaining CMBP time.  As planned, facilitate pilot ecotourism activities in Bluefields and apiculture training in remaining CMBP time  Integrate the marketing plan that BBFFS developed into any ecotourism pilot  Most CMBP training attendees to date are men  Revisit and prioritize gender mainstreaming efforts  Patrolling is simultaneously facilitating enforcement and outreach in both the Pedro Cays and Bluefields Bay, but programs have outstanding gaps  Conduct a strategic planning exercise (e.g. SWOT) to identify strengths and weaknesses, and then facilitate action planning to improve enforcement  Many government agencies consider communication about program activities as ad hoc and needing improvement  Revisit and determine the most effective communication methods relative to the supporting agencies and the roles these agencies needs to play in CMBP Sub-grantee Organizational Capacity Assessment This section presents findings, conclusions and recommendations based on CMBP mid-term performance evaluation question 3: What is the capacity (financial, managerial, and programmatic/technical) of CMBP’s local implementing partners (i.e. sub-grantees)? What implications does sub-grantees’ overall capacity have on CMBP’s ability to achieve expected 25 results? And, what implications does sub-grantees’ capacity have for the sustainability of the activities implemented and results achieved by these organizations? As mentioned in the evaluation design, methods and limitations section, the Evaluation Team conducted an assessment of the organizational capacity of each CMBP sub-grantee. USAID’s Organizational Capacity Assessment (OCA) tool was applied and complemented with additional information obtained during interviews and focus group discussions with TNC, government agencies, stakeholders and the sub-grantees. TNC implements CMBP in four different seascapes through four local sub-grantees, the Center for Conservation and Eco-Development of Samana Bay and its surroundings (CEBSE) in the Dominican Republic, the Sustainable Grenadines (SUSGREN) in Grenada and St. Vincent & the Grenadines, the Foundation for Protection of Marine Biodiversity (FOPROBIM) in Haiti, and the Caribbean Coastal Areas Management Foundation (C-CAM) in Jamaica. This section the report provides insight on the level of organizational capacity possessed by CMBP’s sub-grantees, specifically in terms of financial, managerial, and programmatic/technical capacities and capabilities. The evaluation further includes an analysis of the effectiveness of the implementing partner in convening, engaging and committing stakeholders to support CMBP’s goals and objectives, and assesses the capacity of sub-grantees to promote stakeholder engagement, participation and commitment to MMAs/MPAs. The applied scoring system equated areas of Low Capacity with a score of 1; areas of Basic Capacity with a score of 2; areas of Moderate Capacity with score of 3; and areas of Strong Capacity with a score of 4. Finally, information about the sub-grantees monitoring capacity was obtained from TNC and was included at the end of each organization’s section. Although CMBP’s sub-grantees support monitoring efforts with boats and supplies, TNC does the data collection. None of the sub-grantees have full staff that can conduct CMBP monitoring activities on their own, and usually outsource or hire a contractor to conduct monitoring efforts. Center for the Conservation and Eco-Development of Samana Bay and its Surroundings The Center for the Conservation of Samana Bay and its Surroundings (CEBSE) is a Dominican nonprofit organization, whose mission is "The conservation and sustainable use of the natural and cultural resources of Samana Bay and the natural areas which surround it, with the active participation of its communities." Results of CEBSE’s OCA guided self-assessment for CEBSE are presented in Table 26. A narrative summary of additional findings is presented below. Table 26. CEBSE organizational capacity assessment score results. Section Sub-Section Score 1. Governance and legal structure 1.1 Vision and mission 4 1.5 Succession planning 3 Average section 1 score 3.5 2.1 Budgeting 4 26 Section Sub-Section Score 2. Financial management and internal control systems 2.8 Cost sharing 1 Average section 2 score 2.5 3. Administration and procurement systems 3.1 Operating policies, procedures, and systems 4 3.2 Information technology 3 3.5 Fixed assets management 4 3.6 Branding and marking 4 Average section 3 score 3.75 4. Human resources systems 4.1 Adequacy of staffing and job descriptions 4 4.2 Recruitment and retention 3 4.7 Staff and contractor supervision and work planning 4 4.8 Volunteers and interns 4 Average section 4 score 3.75 5. Program management 5.1 Donor compliance requirements 3 5.4 Stakeholder involvement 3 5.5 Culture and gender issues 4 Average section 5 score 3.33 6. Project performance management 6.2 Project and program evaluation 4 6.3 Service delivery standards 2 6.4 Field support, operations, and oversight 4 6.5 Project performance 4 Average section 6 score 3.5 7. Organizational management and sustainability 7.1 Strategic (business) planning 2 7.2 Annual workplans 3 7.3 Change management 4 7.4 Knowledge management and external linkages 4 7.5 Fundraising and new business development 3 7.6 Internal communications and decision making 4 7.7 External communications 2 Average section 7 score 3.14 Average OCA score (average of all section scores) 3.35 The Evaluation Team concludes that, with an overall average score of 3.35, CEBSE is at a Moderate Level of organizational capacity. Specific areas of Moderate Capacity include financial management and internal control, program management, and organizational management and sustainability. All other areas are considered at a Strong Capacity level. Under financial management and control, CEBSE could increase its capacity level for cost-sharing, which is currently at Low Capacity level. Similarly, areas at the Basic Capacity level that CEBSE could improve include service delivery standards (under project performance management), strategic 27 business planning, and external communications (both under organizational management and sustainability). Due to its 35-year presence in the Samana Bay region, CEBSE is one of the most widely recognized and respected non-governmental organizations in the Dominican Republic. Interviewees from MARENA, CODOPESCA, and fishers in Los Cacaos identified CEBSE as an organization with an adequate level of managerial, financial and programmatic capacity to support CMBP’s efforts to achieve expected results. CEBSE’s grass-roots style activism has lent the organization credibility with local communities, and with local and national government institutions. CEBSE is perceived as one of the most honest NGOs in the Dominican Republic for its professional and hardworking methods, per interviewee feedback. Local governments indicate that some of the most critical environmental work conducted in the region is done with CEBSE’s participation in the processes of decision-making, planning, and execution of activities. CEBSE has adequate organizational recognition in both the Caribbean environmental community and, as noted, the local community. Most of its recognition stems from active stakeholder engagement, a strong network of volunteers the support program work, community feedback sessions, and basic word-of-mouth exchange. CEBSE has had some basic branding success through its logo and some printed outreach materials. However, the organization’s webpage scarcely highlights any of the important programmatic work the organization conducts. Increased online presence and information sharing could directly lead to an increase in organizational recognition, securing of additional donor funds, and ultimately expansion of programs. CEBSE does receive some funding from government agencies; however, it is unclear how much unrestricted funds CEBSE obtains annually through membership fees or basic donor contributions, each of which could be enhanced by a greater online presence. CEBSE struggles to secure regular funding. Because CEBSE’s area of focus is limited to the Samana Bay region, it is difficult for the organization to secure funding proposals at the national level (CEBSE has not expanded its geographic scope in the last several decades). One key related negative impact of this challenge has been the organization’s inability to retain marine biologists or ecologists as permanent staff. Of note, CEBSE considers one of its strengths the capacity for collaboration with adept scientists, both local and international, as one of its strengths. So as an interim solution to deal with funding challenges, the organization hires scientists when needed. Finally, CEBSE staff also indicated lack of vehicles, equipment and technical assistance as particular challenges to achieving certain programmatic results associated with gaps in funding. CEBSE’s current staff of eight remains passionate and dedicated to the organization’s mission and vision, and programmatic staff retention appears high. Still, CEBSE recognizes its challenges in being competitive in the labor market in order to be able to attract and retain adequate personnel. Furthermore, CEBSE’s small-town location can at times impede the interest of prospective contractors or collaborators. CEBSE staff occasionally received professional development training in pertinent topics such as management, administration, planning, evaluation, and environmental management. Staff demonstrate ambition and capacity for growth as program managers, community organizers and project implementers. Much institutional knowledge is retained with current staff. This essential 28 history could be lost should any member(s) of staff choose to leave the organization. In interviews with CEBSE, staff acknowledged this particular issue, and indicated they are currently undergoing a process to develop both a strategic plan and annual work plan in order to capture existing institutional knowledge, and ensure it is memorialized in strategic documents that guide future work of the organization. Staff noted that strategic planning will include, among other things, identification of measurable programmatic objectives and targets, and work plans will be linked to program budgets. As alluded to previously, CEBSE has limited capacity for financial management and internal control systems. Staff indicated one of its greatest challenges is the organization’s inadequate ability to consolidate funding from different donors in order to manage administrative expenses. The organization also lacks a full-time accountant, and has at times in the past had some issues meeting donor cost-share requirements. However, it is important to recognize that CEBSE does produce an annual project budget that reflects individual project budgets, includes multi-year projections of the organization’s revenues and expenditures, indirect cost rates for projects, and necessary line items to meet donor requirements. CEBSE’s persistent work in Samana Bay has retained its credibility as an organization that achieves successes related to important community resource management efforts, sustainable development, biodiversity conservation and collaborative engagement with governmental entities. CEBSE has been engaged in this work for decades before the onset CMBP, and has necessary staying-power, primarily due to the organization’s impressive, and extensive, network of collaborators and volunteers. TNC reported being impressed with CEBSE’s overall capacity to achieve expected results under CMBP, and MARENA and CODOPESCA officers, as well as fishers in Los Cacaos also expressed further offered the recognition that CEBSE has the capacity to be self-reliant (e.g. operating outside of the sponsorship of TNC) if sufficient/consistent funding is obtained from other sources. The government sustainability fund, Fondo MARENA, which received support from CMBP in operationalizing funds from the Caribbean Biodiversity Fund, can serve as a source of local funding and will soon call for project proposals on an invitational basis. There is indication that CEBSE will be invited to submit a project proposal. In the interim, the presence of an implementing partner, such as TNC, helps to alleviate some of capacity limitations faced by the organization as it seeks to expand its programmatic work and make progress towards financial stability. In summary, CEBSE has the organizational staying power, staff capacity, and appropriate level of programmatic organization to achieve present and future expected results under CMBP. That said, the organization still requires oversight and financial support of an implementing partner, such as TNC, to achieve successes, and is not yet ready to operate without that support. CEBSE could strongly benefit from increased administrative support, primarily cost-sharing capacities, in order for the organization to move towards independence, and sustain activities that have been implemented under CMBP. According to TNC, CEBSE’s capacity to conduct regular monitoring and reporting is limited; however, TNC is currently working with CEBSE to improve their reporting and interpretation skills. CEBSE has four technical staff that have been trained by CMBP in SCUBA and AGRRA techniques, that conduct coral nursery monitoring at La Farola Cay, and work with a local dive 29 shop in Samana Bay that provides diving gear and supervision. CEBSE does not have the required experience to lead surveys or to interpret coral monitoring data and generate the technical reports required for CMBP. Although CEBSE does not have staff trained in mangrove monitoring, TNC has provided them with a data collection and reporting template that is being refined with CEBSE. Finally, CEBSE has links to potential contractors that could carry out socio-economic monitoring, and also has a set of young students who help monitor whale watching and marine mammal activities. Sustainable Grenadines, Inc. Sustainable Grenadines Inc. (SUSGREN) is an integrated development and biodiversity conservation NGO operating the Grenadines Islands. SUSGREN’s mission is “To empower community groups in the Grenadines for self-governance and the protection and development of resources.” SUSGREN evolved from “The Sustainable Grenadines Project,” an eight-year initiative led by the University of the West Indies (UWI). This inception project emphasized conservation of biological diversity, primarily of the coastal and marine environment, through education and strengthening responsibility for the protection of natural resources throughout local communities in the Grenadines Islands. Over the course of this initiative, hundreds of community members participated in trainings and stakeholder workshops, began implementing small conservation projects, and developed a strong communications network. As such, SUSGREN has grown into a trans-boundary NGO for people living in the Grenadine Bank islands of both Grenada and St. Vincent and the Grenadines. Results of SUSGREN’s OCA guided self-assessment are presented in Table 27. A narrative summary of additional findings is presented below. Table 27. SUSGREN organizational capacity assessment score results. Section Sub-Section Score 1. Governance and legal structure 1.1 Vision and mission 4 1.5 Succession planning 3 Average section 1 score 3.5 2. Financial management and internal control systems 2.1 Budgeting 4 2.8 Cost sharing 2 Average section 2 score 3 3. Administration and procurement systems 3.1 Operating policies, procedures, and systems 4 3.2 Information technology 3 3.5 Fixed assets management 4 3.6 Branding and marking 4 Average section 3 score 3.75 4. Human resources systems 4.1 Adequacy of staffing and job descriptions 4 4.2 Recruitment and retention 4 4.7 Staff and contractor supervision and work planning 4 4.8 Volunteers and interns 4 Average section 4 score 4 5. Program management 5.1 Donor compliance requirements 3 30 Section Sub-Section Score 5.4 Stakeholder involvement 4 5.5 Culture and gender issues 4 Average section 5 score 3.66 6. Project performance management 6.2 Project and program evaluation 4 6.3 Service delivery standards 4 6.4 Field support, operations, and oversight 4 6.5 Project performance 3 Average section 6 score 3.75 7. Organizational management and sustainability 7.1 Strategic (business) planning 4 7.2 Annual workplans 4 7.3 Change management 4 7.4 Knowledge management and external linkages 4 7.5 Fundraising and new business development 3 7.6 Internal communications and decision making 4 7.7 External communications 3 Average section 7 score 3.71 Average OCA score (average of all section scores) 3.62 The Evaluation Team concludes that, with an overall average score of 3.62, SUSGREN is at a Strong Level of organizational capacity. The only capacity area where SUSGREN is at Moderate Capacity is financial management and internal control, particularly cost-sharing. All other organizational capacity areas are at the Strong Capacity level. SUSGREN has a focused vision and mission, and demonstrates strong organizational management skills, management systems, and have a well-established governing body through a 13-member Board of Directors. They have a history of achievement in the region, an online presence, and recognition throughout the community and among local governments and NGOs. SUSGREN has the capacity to plan for and implement a variety of programmatic work efforts and training programs, including regular support for MPA authorities, as well as engagement of community members, fishers and schools. The organization has experience in biophysical monitoring and management effectiveness, nature trail development, seamoss farming, fisheries sustainability, water taxi operations, coral reef conservation, and lagoon restoration, among others. Though the organization has considerable experience, it has struggled to build a robust full-time staff to accomplish new and/or continuing project work. CMBP enabled SUSGREN to expand its staff capacity significantly, including the addition of a Fisheries Coordinator, administrative support, and interns who assist with monitoring efforts. The Evaluation Team directly witnessed how these additional staff are essential for overall capacity to achieve CMBP expected results, especially for targeted outreach to fisher associations, MPAs and others as conservation programs are growing and expanding. 31 Historically, SUSGREN hires staff on a per-project basis, which makes staff retention nearly impossible. Once a project or grant contract is complete, that staff position disappears. This does not allow for continuity of staff, building of staff skill sets, or consistent organizational presence in the community. With CMBP funds, SUSGREN has been providing project support in both areas MPA capacity building and management and sustainable livelihoods. SUSGREN recognizes the critical need to secure more permanent funding sources for staff positions, specifically identifying the need for a development director. Staff are developing a five-year strategic plan that is expected to be complete in spring 2018. SUSGREN and TNC both agree the sub-grantee has potential for significant growth and independent sustainability. Each concludes that SUSGREN’s work in the region will continue after CMBP is complete. That said, SUSGREN requires training in fundraising and business development efforts. The strategic plan can and perhaps will include comprehensive annual budgets, in contrast to how SUSGREN currently reviews finances on an individual project basis. SUSGREN also requires capacity improvements in succession planning and management, and improved IT systems for information sharing, additional issues that could be addressed in the strategic plan. Of note, SUSGREN’s relationship with TNC is both strong and beneficial. According to TNC, SusGren has limited monitoring capacity and most of the coral reef and mangrove monitoring, as well as for socio-economic monitoring, must be conducted with contractors. On the other hand, SusGren, as a member of the board of the Grenadine Network of MPAs and thanks to its links to the Centre for Resource Management and Environmental Studies at the University of West Indies, may be able to have access to divers working in MPA management in Saint Vincent and Grenada. There is also a staff member at SusGren with the capacity to review technical biological reports. According to TNC, socio-economic monitoring of CMBP’s beneficiaries in the Grenadine Bank has proven to be difficult, since informant fatigue appears to exist due to the substantial number of projects and aid work happening in the area. TNC has found that it is better to take time to first know the beneficiaries and let them know the person conducting the socio-economic assessments beforehand. TNC has found that beneficiaries are more welcoming and open to answering questions using that approach. Foundation for the Protection of Marine Biodiversity The Foundation for the Protection of Marine Biodiversity (FOPROBIM) is well known as one of Haiti’s oldest environmental organizations working in the areas of sustainable development, education, research, monitoring and advocacy. Officially recognized by the Haitian government in 1995, FOPROBIM’s mission6 includes: (i) raising awareness concerning needed changes in regulations and attitudes; (ii) increasing knowledge and capabilities at the government, community and individual levels in order to make sustainable improvements in the environment and the lives of people, and; (iii) conducting educational programs as well as scientific research, including environmental monitoring, in order to promote improved management of the environment and the protection of biodiversity through a better understanding of the need to sustainably use environmental resources. Results of FOPROBIM’s OCA guided self-assessment are presented in Table 28. A narrative summary of additional findings is presented below. 6 http://FOPROBIM.org/index.html 32 Table 28. FOPROBIM organizational capacity assessment score results. Section Sub-Section Score 1. Governance and legal structure 1.1 Vision and mission 4 1.5 Succession planning 4 Average section 1 score 4 2. Financial management and internal control systems 2.1 Budgeting 4 2.8 Cost sharing 2 Average section 2 score 3 3. Administration and procurement systems 3.1 Operating policies, procedures, and systems 4 3.2 Information technology 3 3.5 Fixed assets management 4 3.6 Branding and marking 4 Average section 3 score 3.75 4. Human resources systems 4.1 Adequacy of staffing and job descriptions 4 4.2 Recruitment and retention 4 4.7 Staff and contractor supervision and work planning 3 4.8 Volunteers and interns 3 Average section 4 score 3.5 5. Program management 5.1 Donor compliance requirements 3 5.4 Stakeholder involvement 4 5.5 Culture and gender issues 4 Average section 5 score 3.66 6. Project performance management 6.2 Project and program evaluation 4 6.3 Service delivery standards 4 6.4 Field support, operations, and oversight 4 6.5 Project performance 4 Average section 6 score 4 7. Organizational management and sustainability 7.1 Strategic (business) planning 4 7.2 Annual workplans 4 7.3 Change management 4 7.4 Knowledge management and external linkages 4 7.5 Fundraising and new business development 3 7.6 Internal communications and decision making 4 7.7 External communications 3 Average section 7 score 3.71 Average OCA score (average of all section scores) 3.65 33 The Evaluation Team concludes that, with an overall average score of 3.65, FOPROBIM is at a Strong Level of organizational capacity. Similar to SUSGREN, the only capacity area where FOPROBIM is at a moderate level is financial management and internal control. All other organizational capacity areas are at the Strong Capacity level. Also similar to SUSGREN, the main capacity sub-section which needs strengthening is cost-sharing (under financial management and control). All other sub-sections are at the Moderate to Strong capacity level. In its more than twenty-year history, FOPROBIM has conducted extensive work in the area, including the Protected Area of Managed Natural Resources of the Three Bays (PA3B). They work with myriad community groups, government agencies, and collaborate with local and international NGOs to conduct monitoring efforts, trainings, environmental advocacy, research efforts, and various other programmatic work. FOPROBIM’s organizational recognition, extensive experience working within the Haitian community, and history of successful project completion demonstrates capacity to achieve results. However, like other sub-grantees, FOPROBIM requires more staff and technical equipment in order to increase its capacity to conduct, or sustain, programmatic work. Interviewees indicated the organization could benefit specifically from SCUBA certified staff, additional staff trained in fisheries and marine biodiversity, and additional monitoring/field equipment, including transportation vehicles. Occasionally, FOPROBIM’s lack of funding and technical capacity has caused them to miss project deadlines or fail to meet project deliverables. In recent instances, TNC has endeavored to support the organization by contracting with other entities or individuals to conduct specific activities related to a grant task. FOPROBIM has a solid foundation, a long-term vision, developed operating procedures, and is very efficient at mobilizing local communities. The organization has a developed, if somewhat outdated online presence, and communicates with stakeholders frequently using social media accounts. However, the organization has no master budget aligned with a strategic plan. Instead, budgets are organized on a project-by-project basis and do not consistently bill for administrative fees or overhead in grant proposals. The organization receives only a very limited amount of general funds through small online donations. The lack of consistent and ample funding is a common critical challenge among NGOs in the Caribbean region, and FOPROBIM is no exception. Improvements in fundraising and donor relations are needed. Similar to other sub-grantees participating in this collaboration, one of FOPROBIM’s limitations is its administrative capacity to autonomously run a program such as CMBP. In an effort to help overcome this limitation, for the last several years TNC sponsored an annual audit of the organization, sharing results and ways to improve in staff contract management or in financial management and archiving. The presence of TNC as implementing agency remains necessary at this time. Regarding FOPROBIM’s capacity and ability to assist CMBP in achieving expected results, TNC and ANAP believe FOPROBIM is vital for development program activities in PA3B. FOPROBIM is the only major marine conservation partner for TNC in Haiti, and it remains the only Haitian￾operated, and Haiti-based NGO dedicated to the protection and management of the country’s coastal and marine ecosystems and adjacent watersheds. The organization’s long-time executive 34 director is the winner of several prestigious environmental prizes. Moreover, many fishers openly express in FGD their satisfaction for the good work that FOPROBIM has always done in the community, and hope that CMBP will be able to receive more funds to expand and intensify CMBP activities in the communities. Fishers also appreciate the fact that FOPROBIM have young talented professionals close to the community and very committed to their work, and they believe FOPROBIM can be a reliable partner for them because of their long experience working in the communities, particularly in marine biodiversity. To sustain CMBP activities in the future, both ANAP and TNC indicate that more staff support is needed to develop the organization and implement activities. Of note, CMBP funding nearly doubled FOPROBIM staff capacity, and the organization hopes it will retain the ability to implement activities, perhaps at a lower level, once CMBP funding expires. TNC has reservations about FOPROBIM’s monitoring capacity, particularly since the wardens have not yet been properly trained in biological monitoring methods. The wardens are new and are still in the initial stages of developing knowledge about the importance of protecting the marine environment and building their capacity to carry out their roles in MPA management. Except for the CEO, FOPROBIM has no staff with diving training, and FOPROBIM’s capacity for socio￾economic monitoring is also limited. On the other hand, FOPROBIM has a staff member with capacity to conduct GIS analyses. FOPROBIM will have to hire contractors to conduct monitoring and to produce reports beyond CMBP. Caribbean Coastal Area Management Foundation C-CAM Established in 1997, the Caribbean Coastal Area Management Foundation (C-CAM) promotes coastal conservation in Jamaica. C-CAM supports co-management of coastal and marine resources through organized community involvement. The organization has established four co￾management councils which enable coordination and collaboration with other NGOs, businesses, government entities, and community groups. These co-management councils are Portland Bight Fisheries Management Council (PBFMC), Portland Bight Tourism Council (PBTC), Portland Bight Industrial Council (PBIC) and Portland Bight Citizens Council (PBCC). C-CAM goals include developing sustainable livelihoods, encouraging wise use of natural resources, promoting community involvement in decision-making, and protecting cultural heritage and biodiversity. Results of C-CAM’s OCA guided self-assessment are presented in Table 29. A narrative summary of additional findings is presented below. Table 29. C-CAM organizational capacity assessment score results. Section Sub-Section Score 1. Governance and legal structure 1.1 Vision and mission 3 1.5 Succession planning 4 Average section 1 score 3.5 2. Financial management and internal control systems 2.1 Budgeting 4 2.8 Cost sharing 2 Average section 2 score 3 3.1 Operating policies, procedures, and systems 4 35 Section Sub-Section Score 3. Administration and procurement systems 3.2 Information technology 4 3.5 Fixed assets management 4 3.6 Branding and marking 4 Average section 3 score 4 4. Human resources systems 4.1 Adequacy of staffing and job descriptions 4 4.2 Recruitment and retention 4 4.7 Staff and contractor supervision and work planning 4 4.8 Volunteers and interns 4 Average section 4 score 4 5. Program management 5.1 Donor compliance requirements 4 5.4 Stakeholder involvement 4 5.5 Culture and gender issues 4 Average section 5 score 4 6. Project performance management 6.2 Project and program evaluation 4 6.3 Service delivery standards 4 6.4 Field support, operations, and oversight 4 6.5 Project performance 3 Average section 6 score 3.75 7. Organizational management and sustainability 7.1 Strategic (business) planning 4 7.2 Annual workplans 4 7.3 Change management 4 7.4 Knowledge management and external linkages 4 7.5 Fundraising and new business development 3 7.6 Internal communications and decision making 4 7.7 External communications 3 Average section 7 score 3.71 Average OCA score (average of all section scores) 3.71 The Evaluation Team concludes that, with an overall average score of 3.71, C-CAM is at a Strong Level of organizational capacity. Similar to SUSGREN and FOPROBIM, C-CAM only capacity area at a Moderate Capacity level is financial management and internal control. All other capacity areas are at the Strong Capacity level. Also similar to SUSGREN and FOPROBIM, the main organizational capacity sub-section which needs strengthening is cost-sharing (under financial management and control). All other sub-sections are at the Moderate to Strong capacity level. C-CAM maintains field offices in Pedro, Salt River, and Portland Bite. Each field office is guided by a master plan, and each office maintains annual work plans, budgets, and periodic progress reports with the organization’s headquarters in Lionel Town. C-CAM has six major funding sources, and has maintained a positive cash flow over the last two years. C-CAM staff indicated that while the organization has very little unrestricted income, it does have sufficient financial 36 resources to continue existing programs for the upcoming year without new donor funding. C￾CAM’s annual master budget includes individual project costs and core operating/administrative costs. That said, multi-year projections of the organization’s revenues and expenditures are not completed on an annual basis. C-CAM scored strongly in administration and procurement systems, human resources systems, organizational management and sustainability, and program management. However, TNC indicated during interviews that C-CAM has periodic challenges in meeting project deadlines and submitting deliverables. C-CAM’s work plans and performance objectives are typically set on a project-level basis, rather than at an organizational level. C-CAM’s strategic plan expired in 2017. Furthermore, the organization could benefit from increased training on personnel and administration management. The Pedro Cays is a large area, requiring collaborative management between governmental and non-governmental project partners. TNC is building C-CAM’s capacity for effective management at their Pedro field office. This support has included implementing staffing structure changes, hiring a sanctuary manager for South West Cay SFCA, and increasing the organization’s financial resources through additional grants. Capacity or some other challenges remain however. For example, TNC requested C-CAM staff develop a tool/protocol, which could be used by wardens, for monitoring sea turtles and birds. TNC did not receive a work product from C-CAM for more than 18 months. C-CAM indicates that capacity challenges remains in terms of staff limitations, uniforms, and dive gear, some of which needs to be replaced. C-CAM’s efforts at the Pedro Cays are intended to function as a systems improvement demonstration project, and are expected to help inform capacity development in other C-CAM field stations. Notwithstanding the challenges outlined above, both TNC and C-CAM staff expressed confidence that, with continued financial support and oversight by an implementing partner, C-CAM can address its capacity challenges and continue to both improve and sustain the many activities implemented under CMBP. According to TNC, C-CAM’s monitoring capacity includes two divers, and TNC is currently helping C-CAM build their biological monitoring capacity. One of the divers has been trained in AGRRA and may require a refresher in underwater data collection methods, and the second diver has just recently completed scuba training. C-CAM has limited capacity to perform basic socio￾economic monitoring, and adequate guidance and supervision will be necessary to ensure proper methods are being used. Regarding monitoring equipment, C-CAM has some dive gear available, but much of the equipment has been observed to need servicing or replacement. Sub-grantee Capacity Evaluations: Summary Conclusion and Recommendation The Evaluation Team concludes that all CMBP sub-grantee organizations have an adequate level of managerial, financial and programmatic capacity to support CMBP’s efforts to achieve expected results. Moreover, sub-grantees have the organizational capacity needed to sustain activities into the future. This conclusion is informed by OCA self-assessments and the opinion of stakeholders, government agencies and TNC. Each organization is unique and faces its own challenges, but collectively these NGOs will continue leading important coastal and marine conservation work and community outreach in the Caribbean. 37 CMBP and its partners may wish to put together a plan for strengthening sub-grantee cost sharing capacities, as this is a common weakness for each organization. The cost sharing sub-section assesses whether the organization has systems to track, report, and document cost sharing and meet the cost sharing requirement in agreement with various donor regulations. From interviews, it is apparent that sub-grantees have either not had any donor cost-sharing requirements or had prior donor cost-sharing requirements that were not met, have no written policies and procedures for recording and reporting on cost-sharing, nor written policies and procedures on cost sharing that are inadequate and require substantial changes. More generally, each sub-grantee may benefit by focusing on diversifying funding, strengthening project design and management capacity, and becoming self-sufficient organizations. Regional Findings, Conclusions and Recommendations The bullets listed below present a summary of the main findings and conclusions of this performance evaluation. Regional-level conclusions and recommendations follow that are based on these findings, and include strategic direction and specific actions that USAID and CMBP partners may consider in order to build on progress to date, address challenges, and inform programmatic priorities and focus under the remaining CMBP timeframe. Moreover, lessons learned may inform future planning, as needed, for CMBP or similar regional initiatives. While it is clear that CMBP fell short on some of its original goals, important conservation work has taken place across the region which can and should be built upon. Summary of Performance Towards Program Targets Expected Result 4: Effective Management and Governance of Marine Protected Areas Performance indicators  Indicator 3 (new laws, policies and plans) has produced mixed results: exceeded in Jamaica and fully achieved in Haiti, but only 50% towards the performance target in the Grenadine Bank and Samana Bay.  Indicator 5 (hectares under improved management) is showing substantial progress, with full target achievement in Samana Bay, Haiti and Grenadine Bank, and 60% of target achieved in Jamaica.  Indicator 8 (number of person hours of training), which applies to both Expected Results 3 and 4, has exceeded its targets in all four seascapes. MPA management plans and action plans  The action plans for coral and marine mammal conservation in Samana Bay are of sub-standard quality and require significant improvements.  The Protected Area of Managed Natural Resources of the Three Bays (PA3B) Management Plan is the most comprehensive and well written of all the plans reviewed. Haiti’s Agency for Protected Areas, considers the management plan a good reference to structure the development of marine protection for other needed areas. 38  Good quality rating for the updated Pedro Cays and Surrounding Waters Management Plan, Jamaica, with some need for improvement. The process of management plan development for the South West Cay Special Fishery Conservation Area has begun.  Good interagency and stakeholder collaboration for Pedro Cays and Surrounding Waters Management Plan development process and advancing towards government approval Jamaica. However, mixed responses as to whether lead implementing agency is clear to all parties.  Good quality rating for adaptive mooring management plans in the Grenadine Bank, and implementation occurring in both TCMP and SIOBMPA. MPA governance and management, including patrolling and outreach  Successful creation of six new no-take zones in Samana Bay, declared through a resolution of the Ministry of Environment.  Increased capacity for patrolling, outreach and education, monitoring and general management reported at South West Cay and Bluefields Bay in Jamaica. However, limited patrolling capacity (i.e. less than 24 hr/7 day a week presence) inhibits anti-poaching enforcement efforts at both sites.  Ongoing implementation of a user fee system at TCMP and SIOBMPA (mooring fees) helps secure a sustainable revenue stream. That said, some stakeholders perceive too much focus on collection of user fees to the detriment of other priority management actions at TCMP. Management effectiveness tracking scores (METT)  In the Samana Bay MPA, only Bancos de la Plata y de la Navidad Marine Mammals Sanctuary has shown an increase in METT scores; all other MPA scores have largely stayed the same.  Substantial improvements seen in METT scores in the Grenadine Bank MPAs and modest improvements in Three Bays Park in Haiti.  METT performance scores have not changed substantially at the three assessed Jamaican SFCAs: South West Cay, Bluefields Bay and Galleon St. Elizabeth.  Most organizations are applying WWF’s 2007 version of the METT, or a modified version. It does not appear that organizations have consulted or are aware of the 2004 “marine” version of the METT developed by the World Bank. Expected Result 3: Sustainable Fisheries and Sustainable Livelihoods for Fishers Performance indicators  Performance indicator 6 (number of people with tangible and/or perceived increases in economic benefits derived from improved resource management) has met its target in Haiti for CMBP Year 3. Results are mixed in other seascapes, with great results among Jamaican men (94% completion of the target), but limited success among Samana Bay men (22% completion of the target)  Performance indicator 7 (number of fishers that have applied new technologies or management practices) has had poor results. The percentage achievement of this indicator’s targets was the lowest in Samana Bay and Haiti (0% of the target completion for both men and women) with a slightly better result in the Grenadine Bank (32% completion of the target for men and 4% for women). Results are better in Jamaica, with 71% completion of the target for men and 40% for women. 39  Indicator 8 (number of person hours of training), which applies to both Expected Results 3 and 4, has exceeded its targets in all four seascapes. Fisheries sector action plans  Successful development of a Fisheries Strategic Action Plan for Pedro Bank and the Southwest Coast of Jamaica seascape provided guidance for capacity building of fish sanctuaries.  The Fisheries Sector Action Plan developed for Samana Bay was received positively by the Ministry of Environment and CODOPESCA.  Successful development of a Fishery Sector Action Plan for Grenadine Bank, which helped guide and inform program activities. Capacity of fisher associations  Administrative capacity of fisher associations in Samana Bay has increased and will facilitate sustainable management of cooperative common funds.  In Jamaica the organizational and technical capacities of BBFFS limit the type of financial support they can receive and administer to small, fixed-price grants. No fisher association for the Pedro Cays has been organized as yet.  Community leaders on some islands across the Grenadine Bank report that fisher associations are experiencing difficulties getting organized, and thus most associations are in an early stage of development. Building fisher associations into truly organized and active groups is challenging, particularly on Union Island and Mayreau. Fisher support for sustainable management, fisher association capacity building and rising environmental awareness  Samana Bay fishers and Dominican government officials are optimistic the cooperation network created by CMBP will last far beyond the project timeline, resulting in tangible, long￾term benefits for ecosystems and local communities. CMBP implementation has led to more formal participation of local organizations in the co-management of Bajo Yuna’s natural resources in Samana Bay.  A wide range of trainings conducted in Jamaica were well received and generally had balanced participation of men and women. Nevertheless, outreach and education activities face the challenge of limited literacy among fishers and some wardens, and high turnover of fishers in Pedro Cays.  In the Grenadine Bank increasing SUSGREN and MPA staff engagement has led to improving relationships with fishers, tour operators, and local communities, but key user groups, especially fishers in communities near both TCMP and SIOBMPA, show limited support and some poaching in no-take zones may still be occurring. Notably, leaders of fisher associations in Carriacou, Union Island and Mayreau are well aware of the need for fishers to embrace more sustainable practices. Fisher support for sustainable fishing and increased compliance due to better patrolling  Awareness of the need for more sustainable fishing practices has increased in Samana Bay and in Haiti; however, raising awareness has so far not proved to be enough to change the way local communities use marine resources. 40  HACCP training and equipment for cold storage of catches provided to over 50 fisherfolk in the Grenadine Bank. 70% of participants reported tangible economic benefits as a result of the activity, which responded to stakeholder needs and the Fisheries Sector Action Plan.  In Jamaica, trainings and innovative outreach events led by C-CAM have helped educate Pedro Bank fishers on sustainability concepts, and provided tools to enable fishers to put concepts into practice; however, it is unclear what level of impact the training and outreach programs have had on getting fishers to adopt sustainable practices and thereby reduce threats to key species and marine biodiversity. CMBP funding for staff and patrolling activities provided to C-CAM and BBFFS has led to increased patrolling and outreach efforts in the Special Fishery Conservation Area no-take zones at South West Cay and Bluefields Bay.  Emerging partnerships between C-CAM and the Marine Police, as well as the Jamaica Defense Force Coast Guard, have helped advance programs in the remote Pedro Cays, and show promise for further capacity building, conservation action and lessons learned for the full network of 17 fish sanctuaries in Jamaica.  Safety-at-sea and product handling trainings were broadly received and appreciated by fishers in the Grenadine Bank. However, small boats owned by most fishers cannot fish in pelagic waters, thus it is as yet unmeasured and unclear whether an effort shift in the fishery has occurred, and if this has reduced fishing pressure in nearshore waters. Some stakeholders noted that unsustainable fishing practices are still common amongst Union Island fishers. Overarching Conclusions The original program goals were overly ambitious, yet important foundational work has taken place across the region. Several interviewees—including many who are closely linked to program design, implementation and evaluation—candidly noted during interviews that, after early delays and then a few years of work, it became clear the original program goals were too ambitious or would at minimum a take longer time to achieve. At the same time, many of these same interviewees feel strongly that important, foundational work related to building MPAs and engaging fishers has occurred throughout CMBP sites and needs to continue evolving. This sentiment was validated repeatedly amongst a range of local stakeholders who feel they have received direct benefits from the program, especially though training, new skills development, or the attempts at creating incentives to fish differently or find alternative livelihoods. Some interviewees expressed frustration at challenges often beyond their ability to control, such as delays in needed government support or procurement of equipment and supplies. Finally, it is also important to note that the overambitious reach of the original program goals was also described in the meeting which the Evaluation Team held with CMBP COP at the beginning of the mid-term performance evaluation back in October 2017. This was further confirmed by the prioritization exercise, conducted by CMBP the first months of 2017, which was designed to increase the focus and associated impact of project activities. CMBP has yet to fully evolve into a regional initiative that connects all seascapes. Although originally designed as a regional program, the mid-term performance evaluation revealed that CMBP has to date evolved more like four somewhat interconnected seascape projects, driven by similar goals and program activities. While TNC serves as an important regional program implementer, most activities are implemented at the individual seascape level, and there is little interaction among sub-grantees and other beneficiaries from different countries, with the exception 41 of the Grenadine Bank. Language barriers present another challenge that may limit communication and interaction between seascapes, as three official languages are spoken in the region, including English, French and Spanish, plus other local creole dialects. CMBP participants from one sub-grantee organization expressed during an interview that they would like to have the opportunity to interact closely with sub-grantees in the other seascapes, since the only chance they currently have for face-to-face interaction is once a year, during CMBP’s yearly planning meetings. TNC’s level of participation and guidance also manifests differently in different seascapes, with staff in Jamaica and the Grenadine Bank having more direct involvement with sub￾grantees and other CMBP beneficiaries than in the Dominican Republic and Haiti. All this said, it is important to note that TNC’s monitoring efforts have secured important biophysical and socio￾economic data which creates a regional scale baseline data set. Workshops, trainings and associated equipment distribution are considered one of the most valuable elements of the program to date. Stakeholders across all seascapes commonly cited the value of trainings and workshops, though also often noted that much more training is needed. People are learning and applying new skills or approaches to their work or livelihood, but capacity limitations remain with sub-grantees, local partners and fishers. Some positive examples included: Campaign for Conservation (Jamaica and Dominican Republic); safety-at-sea training (Jamaica and Grenadines Bank); sustainable lobster harvest trainings (Jamaica); sub-grantee capacity building (e.g. administration, monitoring, business development etc.); SCUBA training to maintain mooring buoys (Grenadine Bank); AGRRA training (Grenadine Bank and C-CAM in Jamaica); and apiculture and kayak training (Haiti). MPAs are steadily building capacity, advancing management activities and increasing protections at CMBP sites across the region. Although many sites show limited improvements based on METT scores alone, field visits by evaluators, combined with a close review of management plans, show that most if not all MPAs are engaged in thoughtful planning, and are steadily building capacity and putting important management activities such as patrolling and outreach in place. Key activities that have advanced under CMBP—described in the seascape sections above—include management plan/action plan development, on-water enforcement, construction and maintenance of mooring buoys, and outreach and education among others. Moreover, a number of fishers across the region acknowledged the importance of no-take zones, and a willingness to continue working with the conservation community. This change in attitude alone, where fishers are slowly coming to support MPA management, is a significant improvement in and of itself. MPA authorities and local sub-grantees may still tailor METT tools, and could realize greater benefits by analyzing and addressing evaluation results. Review of METT self￾evaluation scores from each seascape demonstrates that CMBP partners are using WWF’s 2007 version of the METT, or a modified version of this document (e.g. ECMANN modified version in the Grenadine Bank). It is unclear if CMBP partners have consulted the “marine” version of the METT developed in 2004 by the World Bank to ensure key indicators linked to MPA management are not missed in the evaluation process. That said, the Evaluation Team recognizes that, given the history of METT application across the region—at times at the behest of other funders—that local practitioners need to ensure data sets are comparable over time. It is therefore understood that practitioners may wish to make only minor modifications, or none at all, to past versions of 42 the tool which have been applied. Perhaps more importantly, interviews with sub-grantees and other MPA management authorities revealed few examples of how organizations are analyzing results of their evaluations and subsequently incorporating new knowledge into management planning processes. Such analysis and renewed planning may be occurring, just to a limited degree. Finally, some sites would still benefit by integrating government and other stakeholder perspectives into the METT process. The impact of capacity building and training efforts targeting fishers and fisher associations is limited at this stage. CMBP has made commendable strides in training fishers and advancing projects designed to help fishers move towards more sustainable practices. Interviews across the region revealed that many fishers, most notably the leaders of fisher associations, understand well the difference between sustainable and unsustainable practices, even if that understanding does not yet always translate in how the common fisher still practices his or her trade. CMBP launched activities aimed at changing destructive gear or generating alternative livelihoods, but to date these programs have had limited success or failed upon first attempt. Moreover, progress towards the goal, in some seascapes, of promoting offshore fishing to rest nearshore resources remains unmeasured and unclear. Ultimately, it appears that overfishing and unsustainable fishing remain key threats across the region, and thus working with fishers should remain a central thrust of the program. Many fishers throughout the region, if the right incentive can be put forward, remain interested in trying to shift from unsustainable to sustainable practices. CMBP has aligned well with national and local level conservation planning. Many high-level government officials and local partners across the region noted the CMBP has done a good job at listening to the direction local people want to take the work, and then aligning program goals and associated projects with national and community-based conservation plans and stakeholder interests. One interviewee, describing this sentiment, noted that, “CMBP came in and listened, adapted, and in many ways let our priorities and interests shape the direction of the program. They did not come in as an outsider and tell us what to do. We very much appreciated this and think it is the right approach.” Several government officials from across the region also noted that they are pleased to have TNC as a partner. Regional-level Recommendations Continue supporting established CMBP local partners and targeting persistent challenges. Given the limited capacities of many local NGOs and fishing cooperatives, support for development of alternative livelihoods (e.g. ecotourism, apiculture) and fishing gear swaps is still needed until pilot projects become operational. That said, a new strategy for promoting new gear and sustainable fishing practices needs to be developed to avoid following the same path that has led to limited or no adoption. Drawing on human behavior adoption science and using social marketing tools and techniques may help create a deeper understanding of both barriers and needed incentives for fishers to adopt new gear or sustainable practices. At the same time, CMBP should continue working with established local partners that in each seascape, like the sub-grantees, are recognized and respected by local and national actors. The fact that many local NGOs have a small number of permanent staff should not be seen as overly limiting. It is a widespread practice across CMBP countries that experts are brought in based on project priorities and needs. The 43 crucial factor is that NGOs cultivate and develop staff with the necessary range of competencies, build and sustain organizational capacity, and maintain a trusted network of collaborators. Revisit and retrain local partners in management effectiveness evaluation concepts, tools and practices. Across the globe, diverse types of protected area management effectiveness evaluation methodologies and tools have proved beneficial in helping protected area authorities better understand management challenges and opportunities, build capacity, and create a continuous feedback loop of new information that improves management effectiveness. Protected area managers and staff must do much more than complete a scorecard, discuss the findings and submit a deliverable to a funder. Any time an organization completes the METT or any other management effectiveness tool, the organization should closely review and attempt to further validate results to better understand threats, challenges and capacity development opportunities. Evaluation results should also be used to identify needed improvements and associated project concepts which may then be integrated into annual workplans, future grant proposals or other organizational planning efforts. CMBP partners may wish to consult the “marine” version of the METT developed in 2004 by the World Bank to ensure key indicators linked to MPA management are not missed in the evaluation process. Alternatively, CMBP partner may consider transitioning to a progressive tool such as the IUCN Green List.7 Learn from and build upon evaluation of current management plans and action plans and start/continue the implementation of the actions therein. The numerous MPA management plans and action plans that now exist throughout CMBP seascapes represent a key element of the foundation of success that has been built with support from USAID. A close review revealed that most of the plans submitted for review possess an array of strengths. Furthermore, field interviews and site visits demonstrated that many of the activities described in these plans are steadily advancing across the region. At the same time, the evaluation of these plans revealed that each one has room for improvement. The Evaluation Team recommends that USAID share the evaluation findings of each management plan or action plans with the local partners who are implementing the plans. Specifically, the MS Excel worksheets that were designed and utilized to conduct the desktop evaluation should be disaggregated by plan and shared with the relevant management authority. It is understood that these partners have the most informed and intimate knowledge of local circumstances surrounding and influencing the success or failure of MPAs across the region. Viewed in this context, the plan evaluations can serve as a benchmark to identify needed areas of improvement, both in the plan itself and in its implementation. Continue to prioritize stakeholder engagement and effective communication. The need for effective engagement and communication with key stakeholders at all levels of the program cannot be stressed enough. Inadequate communications and interaction with CMBP at the strategic level was cited as a limitation by government officials and some stakeholders. TNC and its local partners must ensure that key government officials and stakeholders are provided clear information on program goals, expected outcomes and needed levels of support from individuals and their associated institutions. At the same time, outreach to locally based resource users and other stakeholders remains equally important and may continue to enable the program to turn past critics of MPAs, and marine conservation more generally, into supporters and even, at times, direct partners in the work ahead. Many local partners, such as MPA authorities and local sub-grantee 7 https://www.iucn.org/theme/protected-areas/our-work/iucn-green-list. 44 staff, may benefit, during the remaining CMBP time period, from training in facilitative leadership, consensus building and conflict management. Develop learning, information sharing and peer exchange networks. As noted above, CMBP consists of a set of relatively distinct project sites in the different seascapes. That said, the program shows potential to develop into a regional initiative that would allow for regional scale learning. If CMBP evolves into another phase (assuming the first years of the program as the initial phase), USAID and TNC should jointly consider development of an information and peer exchange program among seascapes that facilitates knowledge transfer, skills development and dissemination of lessons learned among local sub-grantees, MPA managers and fishers working to advance marine conservation across the region. Local partners may also benefit by participating in exchange programs to other communities around the wider Caribbean that have tackled similar challenges and offer useful lessons learned which can be transferred to the rest of the Caribbean. Finally, an exchange program may also provide an opportunity to standardize training modules and materials around the region, and produce CMBP-branded documents and videos. Learn lessons from others in the Caribbean who are facing similar conservation challenges. Related directly to the aforementioned exchange program, it is important to acknowledge that other resource managers and conservation practitioners in the Caribbean have faced similar challenges and are, with varying degrees of success, advancing progressive, community-based MPA strategies and fishery management models. Lessons on MPA management can be drawn from networks like the Caribbean MPA Managers Network and Forum (CaMPAM) and places like the Bonaire Marine Park (Netherlands Antilles); Roatan Marine Park (Honduras); Hol Chan and Port Honduras Marine Reserves (Belize); Sian Ka’an and Banco Chinchorro (Mexico); and many MPAs in The Bahamas. In relation to fisheries management, one notable example is the managed access program in Belize, where non-governmental organizations have worked with fishers over several years to devise simple indicators which provide scientific guidance and help inform data limited fishery management within an adaptive management framework. The use of Casitas Cubanas for live lobster fishing in Cuba, and community-based lobster management in Sian Ka’an, Mexico, are other notable examples that may provide lessons for fishers in CMBP seascapes. 45 Annex I: Evaluation Matrix Evaluation Question Type of Evidence Required Data Collection Methods Data Sources Data Analysis Methods 1.To what extent has CMBP met its targets under Expected Result 4: Effective management and governance of MPAs/MMAs? What implications does CMBP performance on Expected Result 4 have for achieving CMBP’s overall objective of reducing threats to marine-coastal biodiversity? 1a(i) What is the status (e.g., draft, approved, implemented, etc.) of existing and/or new MPA/MMA management plans in the four priority seascapes? Direct evidence of formally adopted management plans and/or plans under development. Official documents, policies or policy statements, minutes from planning meetings. Interviews with TNC and sub-partner staff, government officials and/or designated MPA/MMA management authorities. Interviews with TNC and sub￾partner staff, activity beneficiaries, and stakeholders. Acquisition of draft and/or final documents. Documents: Draft plan; approved and/or adopted Plan; Annual Report with activities implemented. Related policy documents and/or enabling legislation (local or national). Direct observation during field visits. CMBP progress reports. Desktop. Draft developed YES/NO Approved YES/NO Implemented YES/NO Responses for each of the above validated with illustrative description of process and outputs. 1a(ii) What is the level of quality of these MMA management plans? Existence of goals oriented, broadly supported, and results driven management plans. Comprehensive in nature, demonstrated science integration (social and natural) and ambitious yet practical in illustrated goals, management activities and monitoring and adaptive management protocols. Interviews with government officials and/or designated management authorities, and acquisition of documents and description of document preparation process. Interviews with USAID, TNC and sub￾partner staff, activity Draft and final management plan documents. Aggregate data from all interviews (quantitative, qualitative and anecdotal). Enabling legislation, policies or policy statements. Direct observation during field visits. Review of management plans, METT scores and/or other planning tools. Analysis of document content (breadth, informational inputs, writing style, practical applicability, embedded monitoring protocols). For METT evaluations, review of responses related to management planning, and, 46 Evaluation Question Type of Evidence Required Data Collection Methods Data Sources Data Analysis Methods participants, and affected or interested stakeholders. CMBP progress reports. if applicable, comparative analysis with existing baseline data and changes over time. 1a(iii) How are the MMA management plans being used, and by whom? Demonstrated actions linked to management plan goals, management activities, monitoring protocols etc. Identification of person or persons responsible for leading, reporting or overseeing actions. Interviews with government officials and/or designated management authorities. Acquisition of documents, interview-based data and other evidence that illustrate/demonstrate personnel responsibilities and actions linked to management plan implementation, and/or a supportive implementation role from stakeholders and/or other management partners. Interviews with USAID, TNC and sub￾partner staff, activity Interviews from a range of sources, and document acquisition. Direct observation during field visits. CMBP progress reports. Analysis of links between collected data and management approach/content embedded in management plans (e.g. goals, management activities, monitoring protocols). Analysis of roles and responsibilities of designated management authorities and affected stakeholders and/or supportive management partners. 47 Evaluation Question Type of Evidence Required Data Collection Methods Data Sources Data Analysis Methods participants, and affected stakeholders. 1b(i) In the four priority seascapes, which MMAs are using the Management Effectiveness Tracking Tool (METT) or similar tools? Demonstrated completion (partial or full) and documentation of METT or similar management effectiveness tracking tools. Interviews government officials and/or designated MPA/MMA management authorities. Interviews with USAID, TNC and sub-partner staff, activity participants, and stakeholders. Interviews with other government authorities or funders who have supported application of the METT or similar tools. Interviews from a range of sources. Acquisition of scores and data associated with completion of METT evaluations. Direct observation during field visits. CMBP progress reports. Comparative analysis of which sites are using METT or similar tools versus which sites are not. Analysis of which sites have used METT or similar tools to simply gather baseline information (i.e. one application), as well as sites that have gathered baseline and later information which has enabled tracking of progress, or lack thereof, over time (i.e. multiple applications). 1b(ii) How are the results of the METT being used? Demonstrated and/or document actions, and associated results, that link management activities to lessons learned from the application of METT or similar tools. Interviews government officials and/or designated MPA/MMA management authorities. Acquisition of documents or other direct evidence that link management activities guidance or recommendations from METT studies. Interviews with USAID, TNC and sub￾Interviews from a range of sources. Acquisition of scores and data associated with completion of METT evaluations. Direct observation during field visits. CMBP progress reports. Analysis of demonstrated evidence, or lack thereof, of the application of lessons learned from completion of the METT process by management authorities, affected stakeholders and other supporting partners. 48 Evaluation Question Type of Evidence Required Data Collection Methods Data Sources Data Analysis Methods partner staff, activity participants, and stakeholders. Interviews with other government authorities or funders who have supported application of the METT or similar tools. 1c(i) What data or evidence does the METT provide regarding changes in MMA management effectiveness over time? METT reports from different time periods showing change in scores, with supporting data and evidence demonstrating how and why scores changed. Collection of MPA/MMA management effectiveness data generated by METT or similar tools. Interviews government officials and/or designated MPA/MMA management authorities, as well as affected stakeholders and management partners. Interviews from a range of sources. Acquisition of scores and data associated with completion of METT evaluations. Direct observation during field visits. CMBP progress reports. Identification and description of data and evidence collected during METT exercises that illustrates changes, or lack thereof, in management over time. 1c(ii) Using this tool, has MMA management effectiveness changed (i.e., increased, decreased, or stayed the same) since CMBP’s inception? METT reports from different time periods showing change in scores, with supporting data and evidence demonstrating how and why scores changed. Consolidation of MPA/MMA management effectiveness data generated by METT or similar tools Interviews from a range of sources. Acquisition of scores and data associated with completion of METT evaluations. Direct observation during field visits. Analysis and demonstrated evidence of how and why METT scores, and associated management performance, has changed over time. 49 Evaluation Question Type of Evidence Required Data Collection Methods Data Sources Data Analysis Methods CMBP progress reports. 1c(iii) What factors or conditions (e.g., enabling environment, presence or absence of financing, local capacity, etc.) help explain why or how MMA management effectiveness has changed? Direct evidence of changes in management performance and links to causality (i.e. what factors or conditions led to changes) Interviews with government officials and/or designated MPA/MMA management authorities. Interviews with TNC and sub￾partner staff, activity beneficiaries, and stakeholders. Acquisition of draft and/or final documents that inform or otherwise help advance management activities. Specific interview questions that explore why and how changes occur, and what factors or conditions informed or helped advance change. Interviews from a range of sources. Acquisition of documents, as well as qualitative data and anecdotal information, that explain factors and conditions that inform or otherwise help advance change. Direct observation during field visits. CMBP progress reports. Analysis of links between changes in management and surrounding social, political and environmental factors and conditions (i.e. what factors and conditions enable change and why) 1d(i) What types of management infrastructure has CMBP provided? Direct evidence of new of improved infrastructure. Interviews with government officials and/or designated MPA/MMA management Interviews and direct observation during field visits. CMBP progress reports. Review of interview data and direct observations. 50 Evaluation Question Type of Evidence Required Data Collection Methods Data Sources Data Analysis Methods authorities, as well as affected stakeholders and management partners. Direct observation during field visits. 1d(ii) What plans has CMBP put in place to ensure that the infrastructure provided will continue to be used beyond the life of CMBP? CMBP agreements with recipients of management infrastructure. Evidence of infrastructure maintenance or management plans. Interviews with government officials and/or designated MPA/MMA management authorities, as well as affected stakeholders and management partners. Direct observation during field visits. Acquisition of infrastructure maintenance or management plans. Interviews, direct observation and infrastructure maintenance or management plans. CMBP progress reports. Analysis of how infrastructure has been put in place, how it is used, and the degree of importance it holds for future MPA/MMA management. Analysis of effort that management authorities are putting in to maintain and sustain infrastructure over time. 1d(iii) Are the management actions prioritized in the MMA/MPA plans being effectively implemented with CMBP support? Observation and direct evidence of management plan implementation, or lack thereof. Interviews with government officials and/or designated MPA/MMA management authorities, as well as affected stakeholders and management partners. Interviews with USAID and TNC. Documentation or other Interviews, documentation or other anecdotal information. Direct observation during field visits. CMBP progress reports. Comparative analysis of all collected data from different sources and subsequent determination of what management activities, and their level of effectiveness, can be validated versus those which require more documentation or direct evidence before being considered valid. 51 Evaluation Question Type of Evidence Required Data Collection Methods Data Sources Data Analysis Methods evidence that demonstrate implementation, or lack thereof, of management activities. 2.To what extent has CMBP met its targets under Expected Result 3: More sustainable fisheries sector? What implications does CMBP performance on Expected Result 3 have for achieving CMBP’s overall objective of reducing threats to marine￾coastal biodiversity? 2a(i) What do fishers and relevant stakeholders, such as local authorities and local fisher associations, identify as sustainable or unsustainable fishing practices? Description and demonstrated level of understanding of sustainable versus unsustainable fishing practices by fishers and fisher associated. Interviews and/or focus groups with fishers and fisher associations. Interviews with government officials and/or designated MPA/MMA management authorities, as well as affected non-fisher stakeholders or other resource users affected by fishing impacts. Collection of educational documents that describe sustainable or unsustainable fishing practices. Interviews, educational documents and direct observation during field visits. CMBP progress reports. Comparative analysis of all collected data from different sources, identification of sustainable versus unsustainable fishing practices and subsequent determination of the level of understanding among fishers and fisher association, but also management authorities and stakeholders affected, directly or indirectly, by fishing impacts to coastal and marine resources. 2a(ii) How is CMBP strengthening and measuring changes in Demonstrated evidence of capacity development linked to CMBP goals. Interviews and/or focus groups with fishers and fisher associations. Interviews with Interviews with fishers and fisher associations. Documentation, Comparative analysis of all data sources to determine evidence of capacity improvements, or lack 52 Evaluation Question Type of Evidence Required Data Collection Methods Data Sources Data Analysis Methods local fisher associations’ capacity over time? Examples of how changes in capacity are measured. government officials and/or designated MPA/MMA management authorities who support or otherwise work with fishers and fisher associations. Documentation or other direct evidence of changes in capacity as well as methods for how changes in capacity are measured. direct or indirect evidence, or other anecdotal information that demonstrates changes in capacity. Documentation that illustrates how changes are measured over time. Direct observation during field visits. CMBP progress reports. thereof, and the factors and conditions that enable these improvements. Conversely, analyze all data sources to determine lack of capacity improvements and reasons why. Finally, analyze the level of effectiveness of how changes in capacity are measured and determined as a means to validate any supposed improvements and inform future changes to how this type of analysis is conducted. 2b(i) What are CMBP, local authorities, and/or fishery associations doing to reinforce appropriate fishing practices and discourage inappropriate practices in the area? Demonstrated evidence that illustrate education, enforcement or other efforts to promote sustainable fishing practices and discourage unsustainable fishing practices. Interviews and/or focus groups with fishers and fisher associations. Interviews with government officials and/or designated MPA/MMA management authorities, as well as affected non-fisher stakeholders or other resource users affected by fishing impacts. Acquisition of documentation that illustrate how Interviews with fishers and fisher associations. Documentation, direct or indirect evidence, or other anecdotal information that illustrate how sustainable fishing is being promoted and unsustainable fishing discouraged. Direct observation during field visits. Comparative analysis of all data sources to identify steps (programs, projects, actions etc.) being taken to promote sustainable fishing practices and discourage unsustainable fishing practices, as well as validate the effectiveness, efficacy and reach of these steps. Identify what steps are being led by who, what groups are being targeted and where, and how efforts are being received by target groups. 53 Evaluation Question Type of Evidence Required Data Collection Methods Data Sources Data Analysis Methods sustainable fishing is being promoted and unsustainable fishing discouraged. CMBP progress reports. 2b(ii) How is CMBP measuring changes in marine biodiversity and human well-being as a result of these pilots? Methods and outputs associated with any biophysical and socio￾economic monitoring surveys and/or studies. Anecdotal evidence. Interviews with government officials and/or designated MPA/MMA management authorities, as well as affected stakeholders and management partners. Interviews with USAID and TNC. Acquisition of surveys, studies and/or reports. Biophysical and socio-economic surveys, studies and reports etc. Interviews with government officials and/or designated MPA/MMA management authorities, as well as affected stakeholders and management partners. Interviews with USAID and TNC. Analysis of how monitoring of marine biodiversity and human well-being is conducted, what results have been produced, and how analysis of results inform ongoing management planning. 2b(iii) Based on this evidence, what are the prospects for successful expansion? Ways in which biophysical and socio-economic studies have informed and improvement management planning and subsequent implementation of management activities. Interviews with government officials and/or designated MPA/MMA management authorities, as well as affected stakeholders Biophysical and socio-economic surveys, studies and reports etc. Interviews with government officials and/or designated Building on the above, analysis of how the results of biophysical monitoring and socio-economic monitoring demonstrates progress towards management goals at the 54 Evaluation Question Type of Evidence Required Data Collection Methods Data Sources Data Analysis Methods and management partners. Interviews with USAID and TNC. Acquisition of surveys, studies and/or reports. MPA/MMA management authorities, as well as affected stakeholders and management partners. Interviews with USAID and TNC. local level, as well as achievements linked to CMBP goals. 2c(i) What corrective measures and/or technologies are CMBP, local authorities, and/or fishery associations promoting to mitigate or discontinue unsustainable practices? CMBP progress report indicating activities implemented to corrective measures / technologies to promote mitigation of unsustainable practices. Anecdotal statements from fishers indicating promotion discontinuation of unsustainable practices Interviews with government officials and/or designated MPA/MMA management authorities, fishers and fisher associations, as well as affected stakeholders and management partners. Interviews with USAID and TNC. Interviews. Documentation or other direct evidence that illustrates capacity gaps or management challenges. Comparative analysis of all data sources mapped against local management plan goals, as well as CMBP goals, with an eye toward identifying key gaps and obstacles in relation to enabling conditions (social, political, environmental) and resources, or lack thereof (financial, human, technical, infrastructure), 2c(ii) What are the obstacles and/or gaps? Documentation or other direct as well as anecdotal evidence of obstacles and gaps. TNC progress reports. Semi-structured interviews with USAID, TNC and implementing partners and fishers that inquire for the obstacles and gaps in switching from unsustainable to sustainable practices TNC progress reports. Semi￾structured interviews with USAID, TNC and implementing partners and fishers Characterization of obstacles and gaps identified. 55 Evaluation Question Type of Evidence Required Data Collection Methods Data Sources Data Analysis Methods 2d To what extent are governments scaling up pilot activities implemented by CMBP that promote sustainable fishing practices? Documentation or other direct as well as anecdotal evidence of government-led programs, projects or other capacity development efforts that promote sustainable fishing practices and are in the process of expanding or being scaled up. Interviews with government officials and/or designated MPA/MMA management authorities, fishers and fisher associations, as well as affected stakeholders and management partners. Interviews with USAID and TNC. Interviews. Documentation or other direct evidence that illustrates government led programs that are expanding or scaling up. Comparative analysis of all data sources (different interviewee responses) to determine existing programs and validate whether or not they are being scaled up as a result of CMBP investments. 3.What is the capacity (financial, managerial, and programmatic/technical) of CMBP’s local implementing partners (i.e. sub￾grantees)? 3a What implications does sub-grantees’ overall capacity have on CMBP’s ability to achieve expected results? Identification of sub￾grantee existing capacity and gaps, and relationship of each to current and potential future management performance, specifically as it relates to achieving local management goals as well as CMBP goals and expected results. Identification or adaptation of existing organizational capacity assessment tool. Interviews with USAID, TNC and sub￾partner staff, activity participants, and stakeholders. Also, descriptions of views and opinions expressed by USAID, TNC and sub-partner staff, activity participants, and stakeholders if relevant. Integration of overall data collected during desk top research and field visits, particular as the data illustrate current organizational capacity, gaps, and potential implications for achieving local management goals and overall CMBP goals. In depth analysis of overall data collected which relates to existing capacity development, challenges and organizational performance. Identification of gaps and obstacles, and associated implications for the organization's respective ability to achieve local management goals and CMBP goals. 56 Evaluation Question Type of Evidence Required Data Collection Methods Data Sources Data Analysis Methods 3b What implications does sub-grantees’ capacity have for the sustainability of the activities implemented and results achieved by these organizations? Related to the above, identification of sub￾grantee existing capacity and gaps, and relationship of each to current and potential future management performance, with a view towards how gaps will limit the organization's ability to sustain activities and associated achievements under the current phase of the CMBP. Identification or adaptation of existing organizational capacity assessment tool. Interviews with USAID, TNC and sub￾partner staff, activity participants, and stakeholders. Also, descriptions of views and opinions expressed by USAID, TNC and sub-partner staff, activity participants, and stakeholders if relevant. Integration of overall data collected during desk top research and field visits, particular as the data illustrate current organizational capacity, gaps, and potential implications for future ability of the organization to sustain implementation of management activities, and associated achievements, which began and/or developed under the CMBP. Related to the above, in depth analysis of overall data collected which relates to existing capacity development, challenges and organizational performance, and associated implications for the organization's respective ability to build on and sustain actions and achievements that occurred under the CMBP. 57 Annex II: MPA Management Plan Evaluation Tool Category: Management Goals and Objectives Sub-categories (Indicators) Evaluation Criteria Ranking (Hi/Med/Low) Scoring Criteria Plan Status Supporting Evidence 1. Vision and Mission Priority level: need to have Vision and mission are clearly defined High 3 Only vision or mission are clearly defined, the other category is somewhat defined Medium 2 Vision and mission are somewhat defined Low 1 Vision and mission are not defined at all No score 0 2. Goals Priority level: need to have Clear and overarching management goals have been defined High 3 Management goals have been defined, but are not clear or overarching Medium 2 Management goals are somewhat defined Low 1 Management goals have not been defined No score 0 3. Objectives Priority level: need to have Clear and concise management objectives across a range of management topics have been defined and are SMART (Specific, Measurable, Achievable, Relevant, and Time bound) High 3 Management objectives have been defined across a range of management topics, but are not clear or concise and not SMART Medium 2 Management objectives have been defined across a limited number or just a few management topics, and are not clear, concise and SMART Low 1 Management objectives have not been defined No score 0 Category: Legal and Institutional Framework Sub-categories (Indicators) Evaluation Criteria Ranking (Hi/Med/Low) Scoring Criteria Plan Status Supporting Evidence 58 4. Integration within national development plans Priority level: good to have Plan clearly demonstrates how site management is integrated within national development plans and institutional frameworks High 3 Plan somewhat demonstrates how site management is integrated within national development plans and institutional frameworks Medium 2 Plan only makes limited reference to how site management is integrated within the national development plans and institutional frameworks Low 1 Plan does not demonstrate that site management is integrated within national development plans and institutional frameworks No score 0 5. Integration of natural resource management laws Priority level: need to have Plan clearly demonstrates that national laws which enable management of natural resources are integrated and help guide management actions High 3 Plan somewhat demonstrates that national laws which enable management of natural resources are integrated and help guide management actions Medium 2 Plan only makes limited reference to national laws which enable management of natural resources and it is unclear if these laws help guide management actions Low 1 Plan does not reference national laws which enable management of natural resources No score 0 6. Integration of international agreements Priority level: need to have Plan clearly demonstrates that international agreements are integrated and help guide management actions High 3 Plan somewhat demonstrates that international agreements are integrated and help guide management actions Medium 2 59 Plan only makes limited reference to international agreements and it is unclear if these agreements help guide management actions Low 1 Plan does not reference international agreements No score 0 7. Rules and regulations Priority level: need to have Plan clearly describes rules and regulations that govern resource use with the MPA/MMA boundaries High 3 Plan somewhat describes rules and regulations that govern resource use with the MPA/MMA boundaries Medium 2 Plan only makes limited reference to rules and regulations that govern resource use with the MPA/MMA boundaries Low 1 Plan does not make reference to rules and regulations that govern resource use with the MPA/MMA boundaries No score 0 Category: Management Planning and Activities Sub-categories (Indicators) Evaluation Criteria Ranking (Hi/Med/Low) Scoring Criteria Plan Status Supporting Evidence 8. Budget analysis Priority level: need to have The Plan includes a robust analysis of budgetary requirements, and has identified, described and prioritized these requirements to efficiently conduct all listed management activities High 3 The Plan includes a somewhat robust analysis of budgetary requirements, and has only identified, but not described and prioritized these requirements to efficiently conduct all listed management activities Medium 2 The Plan includes a general analysis of budgetary requirements, and has not identified, described and prioritized these requirements to efficiently conduct all listed management activities Low 1 60 The Plan does not have an analysis of budgetary requirements for conducting all listed management activities No score 0 9. Analysis of limitations and opportunities Priority level: good to have A strategic planning process that identified limitations and opportunities for management was conducted, its findings incorporated into the plan, and a similar process is carried out at least every two years High 3 A strategic planning process that identified limitations and opportunities for management was conducted, but did not inform the plan, and is not carried out at least every two years Medium 2 A strategic planning process that identified limitations and opportunities for management was conducted, but did not inform the plan nor has it been revisited since it was first carried out Low 1 A strategic planning process that identified limitations and opportunities for management has never been carried out No score 0 10. Context in which the plan operates Priority level: need to have The context in which the plan must operate (local, political, environmental, relevant laws) has been clearly and completely defined (by addressing all four areas of context) High 3 The context in which the plan must operate (local, political, environmental, relevant laws) has been somewhat defined (by addressing three of four areas of context) Medium 2 The context in which the plan must operate (local, political, environmental, relevant laws) has been somewhat defined (by addressing two of four areas of context) Low 1 61 The context in which the plan must operate (local, political, environmental, relevant laws) has not been defined No score 0 11. Periodic reporting Priority level: need to have A mechanism for periodic reporting has been created, is embedded in the plan, and is revisited every time the plan is updated High 3 A mechanism for periodic reporting has been created, and is embedded in the plan Medium 2 A mechanism for periodic reporting has been created, but is not embedded in the plan Low 1 A mechanism for periodic reporting has not been created No score 0 12. Administration Priority level: need to have The plan has clearly identified, described and prioritized administrative roles and procedures High 3 The plan has somewhat identified and described administrative roles and procedures, but has not prioritized these procedures Medium 2 The plan has somewhat identified administrative roles and procedures, but has not described or prioritized these procedures Low 1 The plan has not identified, described and prioritized administrative roles and procedures No score 0 13. Internal organization (decision-making) Priority level: need to have The plan has a clearly defined decision-making structure and describes the decision-making process High 3 The plan has a somewhat defined decision￾making structure and the decision-making process is also somewhat described Medium 2 The plan has a somewhat defined decision￾making structure, but the decision-making process has not been described Low 1 The plan has not defined the decision-making structure or process No score 0 62 14. Income generation Priority level: need to have The plan demonstrates that the management authority has identified a diverse potential income portfolio, and is receiving funding from several sources High 3 The plan demonstrates that the management authority has identified a diverse potential income portfolio, and is receiving funding from a few sources Medium 2 The plan demonstrates that the management authority has identified a diverse potential income portfolio, and is receiving funding from 1-2 sources Low 1 The plan does not demonstrate that the management authority has identified a diverse potential income portfolio No score 0 15. Training Priority level: good to have The plan identifies training as a high priority, states training needs for the staff or resource users to carry out their roles and responsibilities, and includes a calendar of training sessions that are needed to ensure staff knowledge and skills are up-to-date High 3 The plan identifies training as an intermediate priority, partially explains the training the staff or resource users needs to carry out their roles and responsibilities, but does not include a calendar of training sessions that are needed to ensure staff knowledge and skills are up-to-date Medium 2 Training is a low priority in the plan, and the plan does not explain staff or resource users training needs to carry out their roles and responsibilities Low 1 63 The plan does not include or mention any training needs or priorities for staff or resource users No score 0 16. Threats analysis Priority level: need to have The plan includes a comprehensive threats analysis that is used to prioritize and direct management actions High 3 The plan contains a somewhat comprehensive threats analysis that is used to direct management actions Medium 2 The plan contains a somewhat comprehensive threats analysis but it is not clear that this analysis is used to direct management actions Low 1 The plan does not contain a threats analysis No score 0 17. Zoning Priority level: need to have The plan includes a clearly defined zoning plan that covers the entire MPA/MMA and there are differentiated activities allowed in the different zones High 3 The plan has a somewhat defined zoning plan that covers more than half of the entire MPA/MMA, and there are differentiated activities somewhat defined that are allowed in the different zones Medium 2 The plan has a somewhat defined zoning plan that covers less than half of the entire MPA/MMA, but there are no differentiated activities defined that are allowed in the different zones Low 1 A zoning plan has not been created No score 0 18. Enforcement Priority level: need to have The plan includes a clearly defined enforcement plan High 3 The plan includes a somewhat defined enforcement plan Medium 2 64 The plan includes a vaguely defined enforcement plan Low 1 The plan does not include an enforcement plan No score 0 19. Infrastructure plan Priority level: need to have The plan includes a clearly defined infrastructure plan High 3 The plan includes a somewhat defined infrastructure plan Medium 2 The plan includes a vaguely defined infrastructure plan Low 1 The plan does not include an infrastructure plan No score 0 20. Carrying capacity plan Priority level: good to have The plan demonstrates that carrying capacity studies have been conducted, and the results of these studies inform management activities and the amount of people that can use and/or enjoy the MPA/MMA High 3 The plan has somewhat defined MPA/MMA carrying capacity, but it is unclear if this informs management activities Medium 2 The plan has only vaguely defined MPA/MMA carrying capacity Low 1 The plan does not include an assessment of MPA/MMA carrying capacity No score 0 21. Climate change adaptation or risk mitigation plans Priority level: need to have The plan clearly highlights how resiliency is incorporated into management actions that aim to avoid or minimize climate related impacts, through either adaptation or mitigation actions High 3 The plan somewhat mentions how resiliency is incorporated into management actions that aim to avoid or minimize climate related impacts, through either adaptation or mitigation actions Medium 2 The plan only vaguely mentions how resiliency to climate change is incorporated into Low 1 65 management actions, through either adaptation or mitigation actions The plan does not mention planning for resilience to climate change No score 0 Category: Stakeholder Analysis and Engagement Sub-categories (Indicators) Evaluation Criteria Ranking (Hi/Med/Low) Scoring Criteria Plan Status Supporting Evidence 22. Community participation Priority level: need to have The plan demonstrates that community participation has occurred at all three stages of protected area development: design, implementation and evaluation High 3 The plan demonstrates that community participation has occurred at two stages of protected area development: design, implementation or evaluation Medium 2 The plan demonstrates that community participation has occurred at one stage of protected area development: design, implementation or evaluation Low 1 The plan does not demonstrate that community participation has occurred No score 0 23. Advisory committee Priority level: need to have The plan describes how an Advisory Committee was established, was a key actor during the establishment of the MPA/MMA, and continues to inform the planning process High 3 The plan describes how an Advisory Committee was established, but played or currently plays only a limited role in MPA/MMA planning Medium 2 The describes how Advisory Committees may be utilized to capture stakeholder input and inform management, but it is not clear that an Advisory Committee has yet been formed Low 1 66 The plan does not have provisions for the creation of an Advisory Committee and such a committee has not been established No score 0 24. Consideration of social issues Priority level: need to have The plan clearly demonstrates how early or current planning processes attempt to prioritize and achieve a balance of respecting traditional resource use and cultural protocols with conservation goals and sustainable resource use High 3 The plan somewhat demonstrates how early or current planning process may attempt to prioritize and achieve some balance of respecting traditional resource use and cultural protocols with conservation goals and sustainable resource use Medium 2 The plan only vaguely demonstrates how early or current planning process may attempt to prioritize and achieve some balance of respecting traditional resource use and cultural protocols with conservation goals and sustainable resource use Low 1 The plan does not demonstrate a process to balance of respecting cultural/social traditions with the MPA/MMA conservation goals No score 0 25. Consensus building and conflict management Priority level: need to have The plan highlights the importance of stakeholder engagement, consensus building and conflict management, and describes processes or mechanisms which staff and stakeholders have used in the past or may apply in the future High 3 The plan somewhat mentions the importance of stakeholder engagement, consensus building and conflict management, but does not describe processes or mechanisms which could be used Medium 2 67 The plan somewhat mentions the importance of stakeholder engagement, consensus building and conflict management, but does not describe processes or mechanisms which could be used Low 1 The plan does not mention the importance of stakeholder engagement, consensus building or conflict management No score 0 26. Education Priority level: need to have The plan highlights the importance of education and describes a multifaceted program, mentioning at least four of these five methods: visual, active, fieldwork, radio, entertainment High 3 The plan somewhat highlights the importance of education and describes a multifaceted program, mentioning at least three of these five methods: visual, active, fieldwork, radio, entertainment Medium 2 The plan somewhat mentions the importance of education and describes a multifaceted program, mentioning at least one of these five methods: visual, active, fieldwork, radio, entertainment Low 1 The plan does not mention the importance of education nor does it describe an education plan No score 0 27. Communications plan Priority level: good to have The plan includes a clearly defined and complete outreach and communications plan High 3 The plan includes a somewhat defined outreach and communications plan Medium 2 The plan includes only a few ideas regarding outreach and communication Low 1 The plan does not have an outreach and communications plan No score 0 Category: Monitoring, Evaluation and Adaptive Management Sub-categories (Indicators) Evaluation Criteria Ranking (Hi/Med/Low) Scoring Criteria Plan Status Supporting Evidence 68 28. Understanding of biological resources (baseline data) Priority level: need to have The plan clearly demonstrates that a scientifically-sound baseline data set has been collected, analyzed and used to inform development of the MPA/MMA, and is used in active and adaptive management High 3 The plan somewhat demonstrates that a scientifically-sound baseline data set has been collected, analyzed and used to inform development of the MPA, and is somewhat used in active and adaptive management Medium 2 The plan demonstrates that a baseline data set has been collected, but it is not clear if this data set was analyzed or used to inform development of the MPA/MMA Low 1 The plan does not demonstrate that a baseline data set was collected No score 0 29. Biophysical monitoring Priority level: need to have The plan clearly defines biophysical monitoring needs, describing the protocols to be used and the periodicity of monitoring campaigns High 3 The plan defines biophysical monitoring needs, and mentions the monitoring protocols to be used and the periodicity of monitoring campaigns Medium 2 The plan somewhat defines biophysical monitoring needs, and mentions monitoring protocols that currently exist, yet does not define which protocols are to be used Low 1 The plan mentions biophysical monitoring needs, but does not mention monitoring protocols to be used No score 0 30. Socio-economic monitoring The plan clearly defines socio-economic monitoring needs, describing the monitoring High 3 69 Priority level: need to have protocol to be used and the periodicity of monitoring The plan somewhat defines socio-economic monitoring needs, and mentions the monitoring protocol to be used and the periodicity of these monitoring campaigns Medium 2 The plan somewhat defines socio-economic monitoring needs, and mentions monitoring protocols that currently exist, yet does not define which protocols are to be used Low 1 The plan loosely defines socio-economic needs, but does not mention monitoring protocols to be used No score 0 31. Governance monitoring Priority level: need to have The plan clearly defines governance monitoring needs, describing the monitoring protocol to be used and the periodicity of monitoring High 3 The plan somewhat defines governance monitoring needs, and mentions the monitoring protocol to be used and the periodicity of these monitoring campaigns Medium 2 The plan somewhat defines governance monitoring needs, and mentions monitoring protocols that currently exist, yet does not define which protocols are to be used Low 1 The plan loosely defines governance needs, but does not mention monitoring protocols to be used No score 0 32. Sustainable tourism Priority level: need to have The plan clearly prioritizes sustainable tourism and provides a list of "best management practices" that recreation providers should adhere to. This list includes: six or more best practices for recreational scuba-diving services; six or more best practices for recreational High 3 70 snorkeling services; and six or more best practices for recreational boat operations The plan somewhat prioritizes sustainable tourism and provides a list of "best management practices" that recreation providers should adhere to. This list must include: three to five best practices for recreational scuba-diving services; three to five best practices for recreational snorkeling services; and three to five best practices for recreational boat operations Medium 2 The plan somewhat prioritizes sustainable tourism and but does not provide a list of "best management practices" for recreation providers Low 1 The plan does not prioritize sustainable tourism and does not provide a list of "best management practices" for recreation providers No score 0 33. Performance evaluation plan Priority level: need to have Application of management effectiveness evaluation tools (e.g. METT) is described as a core element of monitoring and evaluation, and is regularly used to inform and update the overall plan and needed management activities High 3 Application of management effectiveness evaluation tools (e.g. METT) is described as an element of monitoring and evaluation, and is sometimes used to inform and update the overall plan and needed management activities Medium 2 Application of management effectiveness evaluation tools (e.g. METT) is somewhat described as an element of monitoring and evaluation Low 1 The plan does not describe management effectiveness evaluation as part of monitoring and evaluation No score 0 71 34. Local fisheries monitoring Priority level: need to have The plan's monitoring program includes regular monitoring of locally important fisheries species, and involves volunteer support in the monitoring process from fishers or other groups High 3 The plan's monitoring program mentions the need for regular monitoring of locally important fisheries species, but does not appear to involve volunteer support from fishers or other groups Medium 2 The plan's monitoring program is not robust, does not mention the need for regular monitoring of locally important fisheries species, and does not appear to involve volunteer support from fishers or other groups Low 1 The plan does not mention a plan for monitoring locally important fisheries species No score 0 Category: Style and Presentation of Content Sub-categories (Indicators) Evaluation Criteria Ranking (Hi/Med/Low) Scoring Criteria Plan Status Supporting Evidence 35. Maps, tables and graphics Priority level: need to have Maps, tables and graphics are used consistently throughout all components of the plan, and clearly illustrate key features of the MPA/MMA or other useful information High 3 Maps, tables and graphics are used consistently throughout all components of the plan, and somewhat illustrate key features of the MPA/MMA or other useful information Medium 2 Maps, tables and graphics are not consistently used throughout the components of the plan, and poorly illustrate key features of the MPA/MMA Low 1 Maps, tables and graphics are not used throughout the plan No score 0 36. Style and content presentation The plan meets at least 10 or more of the following criteria: clear and accessible; concise High 3 72 Priority level: need to have and comprehensive; accurate and objective; systematic and logical; acceptable and motivating; precise and practical; focused and effective The plan meets seven to nine of the following criteria: clear and accessible; concise and comprehensive; accurate and objective; systematic and logical; acceptable and motivating; precise and practical; focused and effective Medium 2 The plan meets fewer than six of the following criteria: clear and accessible; concise and comprehensive; accurate and objective; systematic and logical; acceptable and motivating; precise and practical; focused and effective Low 1 The plan does not meet any of the following criteria: clear and accessible; concise and comprehensive; accurate and objective; systematic and logical; acceptable and motivating; precise and practical; focused and effective No score 0 73 Annex III: Synthesis of TNC Annual Progress Reports As the regional implementing organization of the Caribbean Marine Biodiversity Program (CMBP), The Nature Conservancy (TNC), has submitted annual projects reports to the United States Agency for International Development (USAID), summarizing the activities implemented across the four priority seascapes in five Caribbean countries: (i) North Coast of Hispaniola (focus on Haiti); (ii) Samaná Bay (Dominican Republic); (iii) Pedro Bank and Southwest Coast of Jamaica (Jamaica); and (iv) Grenadine Bank (Grenada and St. Vincent and the Grenadines). This document synthesizes information included in these reports, focusing primarily on achievements to date and challenges encountered across the region. Preparation of this synthesis focused on Expected Results 3 (Sustainable Fisheries and Sustainable Livelihoods for Fishers) and 4 (Effective Management and Governance of MPAs), as required by the Mid-Term Evaluation solicitation. The Evaluation Team reviewed past TNC reports and prepared this synthesis in advance of field-based interviews with CMBP partners and beneficiaries. Achievements and challenges described below cover work led by TNC, in collaboration with local and regional partners, from the launch of the CMBP in September 2014, up to the end of the third year of the initiative in September 2017. This synthesis does not attempt to exhaustively report on all issues and topics addressed in TNC’s annual reports. Rather, the focus is on key achievements and challenges under CMBP and, as described in TNC’s reports, attribution is made to those who led specific activities at the national or local level. Perhaps not surprisingly, many of the listed achievements become much more descriptive by year 3 of the regional initiative. Achievements to Date In order to increase and build local capacity TNC initially focused its effort on signing sub-award agreements with local partner organizations, including: the Foundation for the Protection of Marine Biodiversity (FOPROBIM) in Haiti; the Center for Conservation and Eco-development of Samaná Bay and its Surroundings (CEBSE) in the Dominican Republic; the Caribbean Coastal Area Management Foundation (C-CAM) in Jamaica; the Sustainable Grenadines, Inc. in Grenada; and CARIBSAVE, a conservation non-profit (NGO) with a regional focus in the Eastern Caribbean. Other start up activities included completion of the required planning documents, recruitment of personnel, inception meetings and development of the monitoring and evaluation program. Over the ensuing three-year period covered by TNC’s annual reports, the following achievements were realized across the region: Sustainable Fisheries and Sustainable Livelihoods for Fishers Regional level activities Year 1  Customized terms of reference for the development of Fishery Sector Assessments and associated Fishery Sector Action Plans, and critical stakeholder consultation processes were solidified and agreed.  Established Fisheries Working Groups in each seascape. 74  Completion of five required USAID start-up deliverables: The Year 1 Work Plan; Performance Management Plan (PMP); Branding and Marking Plan; Standard templates for contracts and sub-awards; Environmental Monitoring and Mitigation Plan (EMMP). Year 2  Fishery Sector Action Plans developed for each seascape.  Engaged National Oceanic and Atmospheric Administration (NOAA) ecologist to determine the most practical approach for developing and effectively managing a market for lionfish, and supported repeated lionfish control efforts in multiple sites. Terms of reference were developed to support a feasibility study (see challenges associated with this effort listed below). Year 3  CMBP developed a regional parrotfish conservation outreach campaign, to promote the non￾harvesting, sale and consumption of parrotfish, stressing the vital role this species plays in maintaining healthy coral reefs. Dominican Republic Activities Year 1  Initiated desktop research and field work with local partners.  CEBSE conducted organizational assessments and provided training to two local fisher associations.  CEBSE initiated a mangrove restoration pilot program in Los Corozos Municipal District, under a previous USAID funded project and a subsequent grant from the GEF. CEBSE provided ongoing technical support under CMBP.  Established a formal relationship with ReefCheck Dominican Republic to educate, promote awareness and conduct trainings on lionfish control.  TNC and local partners initiated efforts, including multiple planning sessions, to propose the establishment of a new, locally co-managed (with fisher associations) marine protected area around La Farola Cay in Samaná Bay. Year 2  Completed the Rapid Fisheries Sector Assessment and subsequent Fishery Sector Action Plan for Samaná Bay.  Recommendations developed, in consultation with stakeholders in Samaná and Sabana de la Mar, based on the findings in the Fishery Sector Action Plan.  Carried out multiple outreach activities, including six trainings on the importance of parrotfish for reef habitat, support for the annual “Marisco Ripiao” Festival in Sanchez, and six awareness raising presentations to schools and fisher organizations in Samaná and Sanchez.  Supported the institutional and administrative strengthening of three fisher organizations from the north and south coast of Samaná Bay: San Lorenzo and Espiritu Santo fisher cooperatives, from Sanchez and Los Cacaos respectively, and the Fish Workers Association of Sabana de la Mar.  Piloted sustainable fisheries in Samaná Bay (oyster mariculture/mud crab fisheries and trawl net exchange to minimize bycatch).  Engaged in multiple activities to promote sustainable alternative livelihoods: o Assisted Espiritu Santo Cooperative in the maintenance and care of coral nurseries near La Farola Cay, and in establishing an ecological trail around the cay. 75 o CEBSE developed a business plan for a San Lorenzo kayak ecological tour inside a national park. o CMBP/CEBSE provided Dios es Amor (Women Artisans) Association with training in administration and basic accounting to help them manage a new gift shop in the community. Year 3  CMBP organized fisherfolk roundtable meetings in Los Cacaos, Sanchez and Sabana de la Mar in July to facilitate discussions between fishers and the government authorities on a new resolution which prohibits the catch and marketing of all species of shark and rays (all Elasmobranches), parrot fish, doctor fish and black sea urchins. The meetings helped improve fisher understanding of the new regulations.  Through CMBP, action plan proposals for conservation of corals and marine mammals in Samaná Bay and marine mammals in the Santuario de Mamíferos Marinos Estero Hondo and Santuario Marino del Norte (Estero Hondo Marine Mammals Sanctuary and Northern Marine Sanctuary) were drafted for discussion and approval by the Ministry of Environment.  CMBP initiated a pilot gear exchange activity to replace unsustainable bottom trawl nets, popularly known as licuadoras, with a more sustainable fishing gear called suripera nets (casts net adapted for shrimp fishing) in Sabana de la Mar and Sanchez. CMBP subsequently conducted a technical and economic feasibility study of the suripera nets in the Dominican Republic. Based on the results of the study, CMBP designed a fishing gear exchange pilot program.  CMBP supported the San Lorenzo Fishermen Cooperative to develop a kayak ecotourism business, including support in business plan development, accounting, first aid, and guiding. CMBP also assisted the group to obtain preliminary approval from the Environmental Ministry in June for a kayak roof rack in Los Haitises National Park.  CMBP also continued to provide support to the Espiritu Santo Fishers Cooperative in developing ecotourism activities in La Farola Cay. With the assistance and follow up of CMBP, the Cooperative signed a concession contract with the Environment Ministry for ecotourism operations in La Farola Key.  CMBP continued to assist the Women’s Association Artesanas de los Los Corozos to develop alternative livelihoods through the production and marketing of handicraft products. The 21 members of the women’s group are now ready to sell their products in Samaná on market days and to participate in fairs and artisan’s markets around the country, and potentially soon on the cruise ship board walk.  CMBP supported the sixth humpback whale festival was organized and celebrated in Samaná.  CMBP, in close coordination with CODOPESCA, designed and published a calendar of banned seasons for fishing.  CMBP began developing a communication campaign to promote use of suripera nets in Sanchez and Sabana de la Mar among fish shops, boat and fishing gear owners, as well as cooperatives and fisher associations. In addition, CEBSE has negotiated a public-private partnership agreement with a local radio station in Sabana de la Mar to support this campaign by offering free airing of a radio spot promoting the use of suripera nets. A similar partnership is being negotiated with another local radio station in Sanchez, with the same purpose.  A second campaign for parrotfish conservation began in Samaná, Los Cacaos and Las Galeras, communities with the largest number of divers and fishers.  CMBP organized the annual Seafood Festival (Marisco Ripiao Festival) in Sánchez, Samaná. 76 Haiti activities Year 1  Initiated desktop research and field work with fisheries specialist Dr. James Miller; draft Fisheries Assessment and Action Plan was complete in November 2015.  Held preliminary discussions on strengthening fisheries policy with the Haiti Fisheries Department and the Inter-American Development Bank (IDB).  Progress made in establishing and operationalizing new local fisher and related associations, including: CEProBaC (Caracol Bay), UGeBFo (Fort-Liberté/ Lagon aux Boeufs), ASME (Fort-Liberté Bay). Year 2  Completed development of the Rapid Fisheries Sector Assessment for the Three Bays National Park; shared draft findings and solicited feedback from fishers, with majority expressing support recommendations included in the action plan.  Made steady progress in strengthening fishers and local fisher associations, to support sustainable fisheries, including training in administrative processes to local fisher associations, and the development of mangrove nurseries and rehabilitation sites.  Conceptualized options for a small pilot “juvenile catch reduction” monitoring program.  Conducted feasibility assessments to assess the market viability and sustainability of three additional alternative livelihoods: honey production, kayaking (ecotourism) services in the park area, and collection and recycling of plastic.  Completed apiculture training in Caracol, and revised apiculture program developed. Year 3  CMBP completed the final draft Fisheries Sector Action Plan for Three Bays National Park and presented it to the public at the University of Limonade, along with the management plan.  A CMBP consultant prepared an assessment of the legal framework for enforcement at Three Bays National Park, which provided clear guidance how current laws allow for effective park governance and surveillance. Meetings were also held with ANAP, the Ministry of Environment, local mayors, and the local police, to obtain their support for the activities carried out by the “environmental animators” trained by FOPROBIM.  CMBP completed a gear exchange feasibility assessment and organized a meeting with mayors and local stakeholders to discuss the preliminary findings, as well as collect additional input for preparation of the final report. Following discussions with different stakeholders, a tripartite cooperation system was proposed for the pilot gear exchange program with key groups: Fishing associations in Madras, Local government authorities with the mayor’s office as the focal point, and FoProPiM. CMBP nearly finalized the Terms of Reference for the recruitment of a contractor to launch a pilot gear exchange program in Madras at the beginning of Year 4.  CMBP completed feasibility studies and mini-business plans for three potential livelihood areas - plastic collection/recycling, kayaking/canoeing, and honey packaging and selling. CMBP engaged a local consultant group, Initiative for Environment and Integrated Development in Haiti, to support development of pilot community-based enterprises for both kayaking ecotourism and apiculture/honey marketing within the Three Bays area. The consultant prepared a workplan and designed customized training to help launch new enterprises. 77  CMBP selected APF (Association des Pecheurs de Fort-Liberté) in Fort Liberté, to lead the first pilot kayaking ecotourism activities through a community-based enterprise. Multiple meetings were held with key stakeholders, including government officials, to help launch the effort.  CMBP contracted an expert apiculturist who began extensive training activities in March for 80 people, 20 each from Bord de Mer Limonade, Jackzyl, Phaeton and Derac. In addition, CMBP installed 20 hives and bee colonies at each of the communities to allow the participants to begin managing the apiary.  CMBP implemented one-day activities to raise awareness about the value and sustainable use of coral reefs, seagrass meadows, mangroves, and fisheries in nine communities - Derac, Phaeton, Chabanon, Fort-Liberté, Jaquesyl, Bord-de-Mer Limonade, Madras and Garde￾Saline.  CMBP contacted local businesses and started plans to organize a meeting to raise awareness about impacts of overfishing and other destructive practices at the Three Bays National Park. Initial work was completed on pamphlet design, messaging, and planning logistics. The meeting was delayed due to hurricanes.  CMBP led two- day training sessions on association strengthening for 30-40 participants from eight fisheries associations - Asosiyasyon Pechè Fayeton, Cooperative des Pecheurs de Derac, Organisation des Citoyens pour le Progrès Avisé de Chabanon, Rasanbleman Peyizan pou Devlopman Jakzi, Brigade Maritimes en Action, Oganizasyon BaSalin pou Devlopman Karakol, Asosiyasyon Pechè Karakol pou Devlopman laPèch and Association des Pecheurs de Garde-Saline. Jamaica Activities Year 1  Initiated desk research and field work in September 2015, in collaboration with Marine Spatial Information Solutions, Inc., and the first draft of the Fishery Assessment and Action Plan was completed in November 2015.  Engaged fishers in wide-ranging discussions on marine conservation and sustainable fisheries issues (e.g. fish sanctuaries, capacity building needs, fishing practices, alternative livelihoods)  Completed a study of pelagic fisheries in Jamaica, entailing stakeholder consultations and a literature review, and produced a draft Pelagic Fisheries project document.  Completed a needs assessment and training to promote local craftsmanship and ecotourism.  At the request of the Fisheries Division, coordinated on-the-ground activities with other donors.  Presented recommendation to establish seven new fish sanctuaries within conservation zones in Pedro Bank.  Participated in public consultations and submitted comments to the Fisheries Division on the National Fisheries and Aquaculture policy paper.  Held discussions with the Fisheries Division on updates to fisheries legislation. Year 2  Completed development of the Rapid Fisheries Sector Assessment and Action Plan in consultation with stakeholders, and developed recommendations.  Promoted ecotourism by supporting a pilot snorkel tour and a series of beautification projects in Bluefields Bay. 78  Supported the hosting of the annual Fisherman’s Day conference where fishers and other fisheries stakeholders gathered to discuss relevant issues with representatives from the ministries responsible for fisheries (Ministry of Industry Commerce, Agriculture and Fisheries – MICAF), including the Minister, the Permanent Secretary and the Director of Fisheries (DOF).  CMBP offered to provide technical assistance to the Ministry of Industry, Agriculture and Fisheries (MICAF) for preparing regulations to accompany the Fisheries Bill. The Ministry has not yet provided feedback on the draft TOR for consultant led support for this effort. That said, August, the Ministry wrote TNC that it wanted technical assistance for Year 4.  CMBP completed the report, “Literature Review of the Length at First Maturity of Commercially Important Reef Fish in Jamaica.” The review document makes recommendations to reduce juvenile fish catch based on the best available scientific evidence, and was shared with the Ministries responsible for fisheries and the environment.  Due to a new management regime instituted by the Fisheries Division in 2017, CMBP decided to focus its interventions in Jamaica on the Spiny lobster fishery. Main activities and achievements included: o CMBP organized a three-day FishPath workshop in February 2017 for the Jamaica Fisheries Division to improve management of the lobster fishery. The Fisheries Division was provided access to the FishPath tool which they can now apply in examining various aspects of the fishery (e.g. artisanal or commercial) and in determining which strategies would be more effective under different scenarios. CMBP offered further assistance to the Division to conduct a management strategy evaluation. While the Fisheries Division was receptive, they were not able to provide the necessary data to move forward. o CMBP designed and implemented a social marketing campaign focusing on the sustainable Spiny lobster fishery called “Size Matters”. Major components included:  C-CAM conducted a rapid lobster fishery awareness survey among thirty-four stakeholders on Pedro Cay to measure awareness and the practice and management of the lobster fishery.  DJ competition with prizes for the best songs about sustainable lobster harvest, as well as preparation of message boards and signs, which were then placed in target locations.  C-CAM distributed 47 lobster gauges, to be used by fishers to measure lobster size to reduce undersized lobster landings.  A survey was administered July through September 2017 to assess the impact of the campaign on stakeholders on the Cays. Year 3  CMBP trained 68 fishers on Top Cay and Middle Cay and 11 members of the Jamaica Defense Force on Middle Cay in lobster regulations, highlighting the legal minimum size, best handling and harvesting techniques and the closed season.  CMBP experimented with one strategy to reduce the lionfish population - increase both the catch and consumption of the fish. Major interventions included 3 lionfish handling and awareness training sessions; a rapid survey of seven restaurants on the South coast of Jamaica to assess the potential of a lionfish market within the Pedro landing sites; and outreach to build 79 a relationship with Food for the Poor, a local NGO that assists community fisheries and people in poverty.  C-CAM organized a Safety at Sea awareness session for 37 persons which featured presentations from the Jamaica Defense Force (JDF) Coast Guard. Subsequently a curriculum was created with the JDF Coast Guard to guide Safety at Sea training sessions with the Pedro Bank fisherfolk. The Coast Guard agreed to facilitate more of these training sessions.  CMBP continued to develop ecotourism and crafts as two areas for potential alternative livelihoods in Jamaica: ecotourism and community and crafts production and marketing. o Ecotourism: CMBP engaged a contractor to work with the community of Bluefields to assess existing community-based/nature-based tourism activities and build upon and/or develop at least one new activity. o Crafts production and marketing: The CMBP contractor is also working in Bluefields, Treasure Beach and Crawford (a community associated with the Galleon sanctuary) to assist craft makers to further develop their products, improve their business processes (e.g. costing, sourcing of materials, etc.) and market their products.  Public awareness sessions were held with stakeholders on Middle Cay (the most populated cay on the Pedro Bank). In addition, teaching aids were developed for wardens and supervisors to guide them in public awareness sessions with fisherfolk.  Following on the Campaigning for Conservation (C4C) training conducted in Year 1, CMBP awarded sub-grants of $6000 each to four local NGOs to fund social marketing campaigns that raise awareness of the value of marine protected areas. Grants were awarded in 2017 to fish sanctuary co-management organizations, Alligator Head Foundation, Montego Bay Marine Park Trust, Negril Environmental Protected Trust, and Oracabessa Bay Foundation.  C-CAM, in collaboration with stakeholders in the fishing industry, organized and hosted the International Fisherman’s Day (IFD) Conference on June 29, 2017. Grenadine Bank Activities Year 1  TNC held preliminary internal meetings in Year 1 to develop the terms of reference for the Fishery Assessment and Action Plan, and to identify data sources.  TNC and local partners held several consultations which led to the establishment of three fisher associations: Carriacou Fisher Organization, Petite Martinique Fishing Cooperative, and Mayreau Explorers Cooperative.  SUSGREN and the Tobago Cays Marine Park Authority jointly carried out a lionfish training, an annual Lionfish Derby event, and, subsequently, a Culinary Expo  TNC and local partners developed a strategy to secure agreements with hotels/restaurants in Grenada and St. Vincent and the Grenadines to purchase lionfish from local fishers. In addition, SUSGREN was contacted by the SVG Ministry of Agriculture to collaborate on lionfish control.  Based on local interest in pelagic fishing, a decision was made that CMBP resources will be used specifically to support safety-at-sea training in the use of GPS and VHF equipment―an essential requirement for all offshore fishing. Year 2  Completed the Fisheries Sector Assessment and Action Plan, and produced recommendations on multiple topics, including: governance, marine resource management, and sustainable fisheries-based livelihoods. 80  Carried out an assessment of the institutional capacity of the fisher associations in Carriacou, Union Island, Mayreau and Petite Martinique, followed by a capacity needs assessment. Subsequently held two workshops focused on strengthening governance of the four target fisher associations in the Grenadines (organized by CARIBSAVE and SUSGREN).  Facilitated an exchange visit for the president of the Union Island Fisher Folk Cooperative with the Toledo Institute for Development and Environment in Belize. In addition, SUSGREN assessed the Cooperative’s needs in effective management and operations.  After preliminary consultations with fishers in Mayreau, SUSGREN developed a training program in Y2 on switching to lobster pots. SUSGREN also identified ways in which CMBP could support livelihoods development in these fishing communities, including economic empowerment of women. Year 3  CMBP facilitated two meetings with 25 fishers on Mayreau, equipment suppliers and lobster buyers to obtain a consensus on changing current fishing methods. With training provided by fishers from Bequia (another Grenadine island), the participants built fifty lobster pots and tested them to determine if this more sustainable gear was economically feasible for the fishers. After three months of use, the fishers provided feedback regarding the effectiveness of the lobster pots, the challenges faced and the next steps for CMBP in supporting more sustainable fisheries in Mayreau. Although fishers supported the initiative, due to a variety of challenges associated with its implementation, the team decided to discontinue the pilot at this juncture and continue it under another project if funding became available.  CMBP provided training and safety supplies to fishers in Carriacou to enable them to perform as licensed water taxi operators and earn additional income without putting pressure on the fishery resources within Sandy Island/Oyster Bed Marine Protected Area. The training was considered a success by participants, and the supplies are expected to boost business and improve the economic situation in Carriacou.  CMBP trained 51 fisherfolk from Mayreau, Union Island, Carriacou and Petite Martinique in post-harvest handling of seafood, based on the international accredited standards for Hazard Analysis and Critical Control Point. In addition, seventeen fishers each received an ice chest to store their catch.  CMBP started apiculture as a supplemental livelihood option for fishing communities in Union Island and conducted an assessment in Carriacou to determine suitability for beekeeping there. Major achievements included: o Trained twelve members of the Union Island Fisher Folks Cooperative Society on the basic theory and practice of apiculture. CMBP also provided the necessary equipment and supplies, including 30 complete hives, 15 bee suites, 15 bee gloves, 8 smokers, 8 hive tools and 2 honey extractors. o CMBP conducted a feasibility assessment to determine the suitability for apiculture on crown lands at Lauriston Carriacou (areas in and around the Lauriston mangrove ecosystem), in collaboration with the Ministry of Agriculture, Lands, Forestry and Fisheries and the Ministry of Carriacou and Petite Martinique Affairs.  CMBP provided equipment for a fisher/farmer market that will be held each month in Carriacou, starting in October 2017.  CMBP also identified the manufacture and sale of sea moss ice-cream as a viable sustainable livelihood for women and men in Mayreau, which is a cruise ship stop and has many visitors. 81  CMBP trained twelve individuals from Union Island communities in August 2017 in eco￾tourism/tour guiding.  CMBP facilitated dialogue and networking between fisherfolk from different islands and between fisherfolk and fisheries divisions in Grenada and SVG. Specifically, CMBP convened a meeting to discuss the findings of the Fisheries Sector Assessment and Action Plan for the seascape attended by fisher organization leaders and senior fisheries officers. CMBP also facilitated dialogue between the executive board and the members of the Union Island Fisherfolk Cooperative. Effective Management and Governance of MPAs Regional Year 2  Began applying Management Effectiveness Tracking Tool (METT) to assess progress towards improved management effectiveness in core CMBP sites. Year 3  CMBP made progress in building the essential “infrastructure” for successful MMAs: solid MMA management plans that guide action; effective MMA governance mechanisms; and sufficient human capacity to carry out community outreach, enforcement, monitoring, restoration, and other day-to-day management tasks. Haiti Year 1  TNC led an ecological baseline assessment process to characterize, inventory and analyze marine and coastal biological resources in Three Bays National Park. An AGRRA assessment was also conduct in the park.  TNC supported FOPROBIM’s efforts to build local operational capacity at the Three Bays National Park.  TNC supported a stakeholder workshop and the subsequent establishment of a technical advisory group for the Three Bays National Park.  FOPROBIM held environmental education events in Caracol and Ft. Liberté; also developed educational materials on conch and lobster.  FOPROBIM continued planning for coral restoration  FOPROBIM identified potential park wardens, discussed appropriate park management regime with stakeholders.  TNC drafted a memorandum of understanding, focused on marine protected areas generally, and the Three Bays National Park specifically, to be signed with the Ministry of Environment. Year 2  CMBP started developing the first management plan for the Three Bays National Park, including active stakeholder engagement in the process.  CMBP initiated community outreach and training activities to strengthen local capacity for implementing various aspects of the Three Bays National Park management plan: o FOPROBIM began training eight individuals from different communities within the Three Bays National Park to possibly work as park wardens. o FOPROBIM advanced its mangrove restoration work in Caracol 82 o FOPROBIM held environmental education sessions for seven communities: Phaeton, Derac, Jaquezy, Madras, Fort-Liberté, Bord-de-Mer Limonade and Caracol.  CMBP organized a workshop in July 2016 for environmental and fisheries representatives of the Dominican and Haitian Governments, as well as members of NGOs carrying out projects related to the coastal-marine environment on both sides of the border. Dominican Republic Year 2  CEBSE produced the annual humpback whale catalog to support whale monitoring and awareness raising during the whale season.  CMBP launched the Dominican Coral Network, a joint initiative of the Propagas Foundation, The Nature Conservancy, Punta Cana Group Foundation and Reef Check Dominican Republic, to promote effective management of the coral reefs across the Dominican Republic; CMBP started the contracting of a coordinator who will prepare the network’s the action plan and start its operationalization. A Memorandum of Understanding was signed by the four founding organizations and discussions advance on how to collaborate with the government  CEBSE trained members of Women’s Association Artesanas de los Corozos, an association of artisans, to monitor the mangrove restoration activity in Samaná Bay; artisans subsequently organized a clean-up of plastic waste. 640 red mangrove seedlings were planted in places where reforestation had not been successful for different reasons and seedling recollection also took place in some areas.  CMBP provided training in coral reef monitoring using the Atlantic and Gulf Rapid Reef Assessment method in Samaná Bay. Year 3  CMBP continued to support Fisherfolk Roundtable meetings as an effective platform to increase fishers’ participation in policies issues and engage in dialogue with the government. A total of ten meetings were held in Sanchez, Samaná, Los Cacaos and Sabana de la Mar. 120 fishers received a basic orientation on Dominican fisheries legislation, which improved their understanding of allowed and un-allowed fishing practices. With CMBP support, CEBSE participated actively in the Evaluation Commission, which is the governance mechanism related to the license granting and monitoring of the whale watching season. CEBSE trained eight 8 persons in the use of the monitoring template, basic GPS and cameras. During the whale watching season in Samaná Bay, which ran from January to March, CEBSE conducted 91 monitoring trips at sea to identify and describe anthropogenic impacts affecting the humpback whales.  CMBP consolidated the coral nursery at La Farola Key and expanded the coral frames from five to ten frames, with an approximate length of 15 meters of Acropora cervicornis coral tissue. CMBP also supported several training activities in coral nursery management. In addition, CMBP supported Open Water Dive training and an exchange visit to Bayahibe for several members of the Cooperative Espíritu Santos to further build their capacity for coral nursery management. Subsequently, CMBP assisted the Cooperative to negotiate agreements with two local dive centers in Samaná - Cayo Levantado Water Sport Center and Scubaquatic - for the maintenance of the nursery in La Farola Key.  With the support of CMBP, an action plan for the management of coral reefs in the Samaná Bay was prepared under a consultancy with FUNDEMAR. 83  CMBP expanded the mangrove nursery at the Arroyo Barril site in the Samaná area, in coordination with the Environmental Ministry. CMBP also provided training on mangrove plantules reproduction for seventeen members of the reforestation brigades of Arroyo Barril.  CMBP organized a workshop at Sánchez City Hall to identify sources of pressure depleting natural resources and discuss conservation strategies and implementation of sustainable actions at Manglares del Bajo Yuna National Park. A similar workshop was also organized with the RAMSAR committee in the Rincón de Molenillo community to identify possible synergies among government authorities and civil society organizations to enhance conservation actions.  During the Year 3 CMBP/TNC facilitated a series of meetings to facilitate government participation in the Dominican Reef Action Network (RAD). As a result, CODOPESCA and the Ministry of Environment will be joining the network next year. In addition, the bylaws were approved in the 4th quarter of Year 3, and it is expected that RAD will be legally registered in the next quarter. Jamaica Year 1  Revised the management plan for the Pedro Cays and surrounding waters  CARIBSAVE provided training and technical support, as well as financial support for staff of the Bluefields Bay and Galleon fish sanctuaries.  Developed a draft business plan for Bluefields Bay fish sanctuary.  Laid groundwork for improved management capacity at Southwest Cay, including building and training staff, purchasing equipment, developing new patrolling data collection methods, and engaging fishers. Year 2  CMBP provided training in coral reef monitoring using the Atlantic and Gulf Rapid Reef Assessment method.  CMBP also completed activities to strengthen the management of Bluefields Bay and Galleon MMAs and Southwest Cay Fish Sanctuary (Pedro Bank). o Provided funding for critically needed wardens (Bluefields Bay). o Provided for ecotourism development, including supplies to carry out snorkel tours (Bluefields Bay). o Supported the continuation of coral restoration efforts (Bluefields Bay). o Prepared a draft Special Fisheries Conservation Area management plan (Bluefields Bay). o Provided outreach and scuba-related equipment to the Treasure Beach Foundation (Galleon). o TNC/CMBP provided support to the drafting of a Marine Spatial Plan (Pedro Bank). o Supported development of a draft management plan for South West Cay Special Fisheries Conservation Area (supported primarily by UNEP). CMBP updated the Management Plan at NEPA’s request to incorporate the feedback from four public consultations (Pedro Bank). o Drafted a memorandum of understanding (MOU) for the overall management of the Southwest Cay Special Fisheries Conservation Area in Pedro Bank; C-CAM and the Fisheries Division later signed the MOU (Pedro Bank). 84 o A MOU has been drafted that will allow for greater operational collaboration between the Marine Police and C-CAM’s Conservation Officers and Wardens who enforce the Southwest Cay Special Fisheries Conservation Area (Pedro Bank). o Provided funding to C-CAM for a patrolling boat and for four new supervisors who oversee the work of wardens (Pedro Bank). o C-CAM provide a wide range of trainings, including but not limited to dive training, standard operating procedures for enforcement, sea safety and vessel handling, and conservation and general warden training (Pedro Bank). o Provided staff with the necessary equipment and supplies to carry out their functions. o Installed three mooring buoys at both South West Cay and Middle Cay to allow for safer anchorage of the patrol vessel.  CMBP signed protocols with eight local associations to affirm their support in protecting the natural resources of Three Bays National Park to the best of their abilities.  Discussions continued with ANAP for the development of a co- management agreement for Three Bays National Park with FOPROBIM as well as suitable enforcement measures for the implementation of the new management plan within the Three Bays National Park.  CMBP completed the draft Three Bays National Park management plan through an extensive participatory process involving numerous meetings with Haiti’s central and local governments and local stakeholders in the main coastal communities, all of whom provided useful feedback and guidance. The final management plan was presented to municipal and central government officials in two separate meetings, and subsequently approved. Afterwards the CMBP consultants (Global Parks) prepared a corresponding action plan, which was presented to ANAP at the end of August.  CMBP provided training in environmental science and ecology during the first and second quarters to eight community members who were identified as potential wardens for ANAP. CMBP subsequently signed contracts with the eight individuals, hiring them as surveillance agents for Three Bays National Park under FOPROBIM’s supervision.  CMBP reached agreement with the Gulf and Caribbean Fisheries Institute (GCFI) to jointly plan and implement an enforcement training workshop for wardens that work within in Three Bays National Park.  CMBP met with members of a local fishers’ association, Brigade Maritime en Action (BMA), as well as the mayor’s office in Caracol to discuss potential collaboration with FOPROBIM and ANAP to begin patrolling the Bord de Mer Limonade and Caracol area waters.  CMBP facilitated the second assessment of management effectiveness in 3BNP using the Management Effectiveness Tracking Tool (METT) developed by TNC. The assessment found that 3BNP’s METT score increased by 8 points, from 31 to 39. A meeting was held to discuss the assessment in July with representatives from the ministries of the environment and agriculture, ANAP (including the newly appointed Three Bays National Park director), CEPROBAC and UGEBFO, UNDP's EBA project and several mayors.  During Year 3 CMBP outplanted 60,000 R. mangle mangroves seedlings in nurseries around Three Bays National Park. In addition, CMBP completed mangrove ground truthing at Three Bays National Park in August, using a combination of remote sensing technology and field surveys to provide in-depth information on the spatial extent and biophysical conditions of the forest. 85 Year 3  CMBP provided support to the Jamaica Fish Sanctuary Network (FSN), which is comprised of all the co-management entities in Jamaica which have arrangements with the government to manage fish sanctuaries. Support included: o CMBP supported a Fish Sanctuary Network meeting in Bluefields Bay. Twelve sanctuaries were represented, and updates were given by each organization describing the status of each sanctuary including staffing, resources, enforcement activities, funding and challenges being experienced. o CMBP facilitated a week-long workshop in St. Ann for the members of the network. This workshop facilitated the sharing of experiences among the members, and provided an opportunity for both the Fisheries Division and the National Environment and Planning Agency to meet with the members of the network, and discuss both existing and proposed legislation.  The National Environment and Planning Agency granted two-year Game Warden status to the Pedro staff wardens, supervisors and Project Assistant.  A range of training sessions were conducted by CMBP for C-CAM staff: 2 staff were SCUBA certified, 8 staff received fish and bird identification training, 6 staff received CPR and first aid training, and conflict and dispute resolution training. Two C-CAM staff members attended turtle monitoring training in August and then cross-trained the Pedro Wardens.  Teaching aids were developed to promote sustainable fishing practices among fisherfolk, specifically focused on lobster conservation, regulations and safety at sea.  Sanctuary staff and a contractor, with CMBP support, replaced eight lost marker buoys and maintained several others around the South West Cay sanctuary. A mooring was installed on Middle Cay and on South West Cay to allow the CMBP sanctuary patrol boat to be safely tied up. A new South West Cay fish sanctuary sign was also created, and it was erected at each of the three Pedro Cays.  Pedro South West Cay sanctuary wardens completed 269 patrols between October and August, on average every 24 days per month.  CMBP continued monitoring of adult turtle tracks and nesting birds. Bird surveys were conducted at Middle and SWC in November, December and February.  CMBP awarded a fixed price agreement to the Bluefields Bay Fishermen’s Friendly Society to support enforcement, training, conservation and public awareness for the Bluefields Bay Special Fishery Conservation Area, which they co-manage.  Enforcement and monitoring activities commenced in Bluefields Bay, with 140 patrols logged over the period May to July 2017.  CMBP started monitoring the sea turtle nesting sites in Bluefields Bay. 23 nests were recorded during the period, with eight nests confirmed and fifteen unconfirmed. CMBP also facilitated sea turtle tagging training in Bluefields for CMBP community groups as well as other groups carrying out turtle monitoring programs.  CMBP began supporting coral restoration work in Bluefields Bay Special Fishery Conservation Area.  CMBP also supported training in SCUBA diving and coral gardening to three members of the Bluefields Bay Fishermen’s Friendly Society staff to facilitate the continued maintenance of the coral nursery as well as to establish a second nursery. 86  CMBP awarded an in-kind grant to BREDS (The Treasure Beach Foundation), which is the management entity for the Galleon SFCA. The grant provides equipment and training assistance to BREDS.  Data measuring the health status of mangroves in targeted CMBP seascapes (St. Vincent and the Grenadines, Haiti and the DR) were collected and analyzed.  CMBP completed a gender assessment of the five target countries and developed an action plan to reduce potential gender inequities in CMBP activities. Grenadine Bank Year 1  Mostly completed reviews of existing management plans for two CMBP Core Sites (Tobago Cays Marine Park and Sandy Island Oyster Bed Marine Protected Area).  Advanced multiple seascape wide programs, including: o Water quality monitoring program o Junior ranger program o Training and information to support effective marine resource management o Regional awareness campaigns o Hiring management staff and providing support to wardens and manager o Technical and financial support for patrols and mooring buoy installation o Support for development of enforcement and communication plans, and sustainable finance for management Year 2  CMBP/SUSGREN drafted two management plans for mooring buoys, one for Tobago Cays Marine Park and one for Sandy Island/Oyster Bed Marine Protected Area.  SUSGREN continued the popular MPA Junior Ranger Program for the Grenadines. eleven modules were delivered to 14 local schoolchildren and activities included a field visit to TCMP, a site visit to Palm Island Resort to conduct bird monitoring, and an exchange visit to Grenada with children who were participating in the Fisheries Division summer camp. Year 3  CMBP created two adaptive mooring management plans for Tobago Cays Marine Park and Sandy Island/Oyster Bed Marine Protected Area in consultation with MPA boards, staff, yacht charter companies and dive shops. In addition, CMBP facilitated the creation of an operational plan for the management of Sandy Island/Oyster Bed Marine Protected Area by the co￾management board for the MPA at a special board meeting. CMBP also presented the plan to the Sandy Island/Oyster Bed Marine Protected Area Board. CMBP also trained wardens to improve moorings maintenance  A standard operating procedures manual for MPA enforcement in St. Vincent and the Grenadines was created with counterpart funding from the German government through the Eastern Caribbean Marine Managed Areas project.  CMBP hired two wardens and an outreach officer for Sandy Island/Oyster Bed Marine Protected Area, enabling the MPA to improve collection of fees, moorings maintenance, surveillance and enforcement, education and outreach and other MPA management functions.  CMBP trained six rangers at Tobago Cays Marine Park and one SUSGREN member of staff in SCUBA diving. Four rangers and the SUSGREN staff member attained the PADI Open Water 87 Diver certification and two TCMP rangers attained the PADI Advanced Open Water Diver certification.  CMBP trained 12 young people from Union Island through SUSGREN’s pilot Junior Ranger Program.  CMBP is began establishing several public-private sector partnerships to strengthen the management of SIOBMPA: o Sustainable Grenadines, Inc, established a public-private-partnership (PPP) with the Mustique Company Ltd. to improve management of the Sandy Island/Oyster Bed Marine Protected Area. o Memoranda of agreements have been drafted to establish public-private sector partnerships between the Sandy Island/Oyster Bed Marine Protected Area and three dive shops in Carriacou - Deefer Diving, Lumber Dive and Arawak Dive - pledging their support for MPA management activities, such as training wardens in SCUBA diving, biophysical monitoring and moorings installation and maintenance. o CMBP also facilitated the drafting of a Memorandum of Understanding for a public-private sector partnership between Deefer Diving Carriacou Ltd., and the Ministry of Carriacou and Petite Martinique Affairs, which is the government body that leads the local co- management board of the Sandy Island/Oyster Bed Marine Protected Area.  With counterpart funding through the Eastern Caribbean Marine Managed Areas project, architectural designs were created for a multi-purpose ranger station, visitor center and eco￾friendly washroom facility in Tobago Cays Marine Park.  With counterpart funding through the Eastern Caribbean Marine Managed Areas project, 3,000 MPA brochures were printed and distributed to visitors to Tobago Cays Marine Park.  CMBP provide extensive support towards the development of the Grenadines Network of MPAs, which has proven to be an effective platform for sharing knowledge and experiences among seven MPAs in Grenada and St. Vincent and the Grenadines, including CMBP’s two core MPAs. CMBP assistance resulted in the following achievements: o MPA practitioners were trained in natural resource management during the 7th Annual Meeting of the Grenadines Network of MPAs. o A monitoring calendar was prepared to coordinate biophysical monitoring among the MPAs within the Network. o Peer-to-peer training between member MPAs resulted in increased staff capacity, including water quality monitoring capacity at Tobago Cays Marine Park and moorings maintenance and visitor relations skills at Sandy Island/Oyster Bed Marine Protected Area. o Grande Anse MPA became the latest MPA to join the Grenadines Network of MPAs. o CMBP assembled a “Technical Guru Team”, a set of experts who are willing to provide pro bono consultancy services to the Network member MPAs.  TNC and SUSGREN facilitated an assessment of management effectiveness in Tobago Cays Marine Park using the Management Effectiveness Tracking Tool developed by TNC. The assessment found that the marine park’s METT score had increased significantly to 71% since the previous assessment in August 2014, when the score was 55%.  CMBP completed its planned reforestation of four hectares of degraded mangroves at Ashton Lagoon in Union Island. Major achievements included: 88 o TNC first conducted a field assessment of the mangrove ecosystem, in collaboration with SUSGREN, TCMP and key stakeholders from Union communities, to determine the cause of the mangrove dieback /degradation and formulate a feasible approach for reforesting the area. o CMBP successfully trained eight individuals from Union Island, mainly from the Ashton community, in mangrove seedling propagation and out-planting techniques, installed a temporary nursery using local materials, propagated 3000 red mangrove seedlings and replanted four hectares of degraded mangroves. o CMBP also mapped mangrove areas around Ashton Lagoon to analyze and compare mangrove extent cover and biomass over time (i.e. from 2015 to 2017). Challenges The challenges below are generally listed in the sequence in which they occur in TNC’s annual progress reports. That said, some challenges have been continual over the 3 year period, and thus the list below is not separated by project year.  Time consuming process for hiring new staff, establishing sub-award agreements with local partners, and identifying tax exempt processes in all countries.  Slow response time by government authorities, including limited availability of staff with necessary qualifications. Certain CMBP activities require government consultation or approval, and delayed responses from government agencies result in delayed project implementation.  Delays in engagement by government partners due to elections in Haiti and the Dominican Republic, as well as general challenges associated with political instability in Haiti.  Delays in advancing development of Fishery Sector Assessments and associated Fishery Sector Action Plans due to loss of in-house TNC Fisheries Specialist; additional delays based on local challenges in Haiti and the Grenadines.  Due to lack of funding from its partner NOAA, TNC decided to drop the lionfish market feasibility study from its workplan.  The sudden closure of CARIBSAVE temporarily disrupted activities in Jamaica and the Grenadine Bank, until TNC or other partners could take over its main activities.  Local NGOs and associations in the region have weak capacity and require institutional strengthening before they can implement project activities and manage their operations.  CMBP will spend the total amount of funds obligated in the cooperative agreement much sooner than originally planned (around Dec 2018) due to an accelerated average spending rate, thereby complicating program planning and communications with stakeholders.  Pedro Bank remains a particularly challenging site for project implementation in view of its remote, offshore location; the limited availability of transportation to and from Pedro Bank; the vagaries of offshore weather; and the general capacity limitations of partners.  The original targeted groups for the fishing gear exchange program in the DR show less demand for the suripera net than expected, requiring the target groups be expanded to include fish shop and fishing gear owners who have an important influence in fishing practices.  It has proven more difficult than expected to fund the constructions costs of the planned ranger station in TCMP in St Vincent. 89  While new young members are joining the Espiritu Santo Fishermen Cooperative for its ecotourism initiative at La Farola Key, the fishers’ participation in the organization is declining.  ASOTRAPESAMAR’s management and Board members do not appear entirely committed to fulfill the organization’s mission or to cooperate with CMBP’s support activities.  The approval process for VAT exemptions in the Dominican Republic has become longer with the Ministry of Foreign Affairs (MIREX) channeling requests to DGII, thereby delaying the implementation of activities, such as the contract for the Fishing Legislation workshops and the contract for the RAD coordinator. 90 Annex IV: Sources of Information Plans, Reports and Technical Documents Award and Modifications: AGENCY TRANSFER CN #15 CARIBBEAN DEVELOPMENT PROGRAM (Env_t-632b IAA with USFWS) AID-OAA-A-14-00064 - Mod 02 - TNC signedMay15 AID-OAA-A-14-00064 - Mod 02 - TNC AID-OAA-A-14-00064 - TNC AID-OAA-A-14-00064 (04) LS AID-OAA-A-14-00064 (04) AID-OAA-A-14-00064 (Mod. 06) AID-OAA-A-14-00064 Mod 1 AID-OAA-A-14-00064 Mod 1signedMay15 AID-OAA-A-15-00064-03signedMay15 AOR Designation - AID-OAA-A-14-00064 - TNC AOR designation letter 2015_01 CMBA TECHNICAL APPLICATION BY TNC: Revised Final August 21, 2014 USAID TNC CMBA Cost Application MPA/MMA Management Plans and Action Plans: Action Plan for Samana Bay MMA Samana Bay Fisheries Sector Action Plan Management Plan for Three Bays Protected Area English version Fisheries Sector Action Plan 3 Bays National Park -Haiti Fisheries sector action plan CMBP Jamaica Seascape CMBP Gender Assessment and Action Plan 9-30-2017 CMBP Grenadine Bank Seascape Fisheries Action Plan Final 07202017 SIOBMPA Adaptive Mooring Management Plan SIOBMPA Management Plan 2015-2025 TCMP Adaptive Mooring Management Plan Pedro Cays Management Plan Jul 2016 Samana Bay DR Rapid Fisheries Sector Evaluation METT Reports: AP3B_METT_Haiti_2017 Bluefields METT_2015, 2016 and 2017 DR METT Methodology May2015 DR METT Los Haitises National Park_2015 DR METT Manglares del Bajo Yuna National Park 2015 DR METT Marine Mammals Sanctuary 2015 Galleon METT 2015, 2016 and 2017 GB METT 2015, 2016 and 2017 Informe METT RD September 2017 JM_METT 2015, 2016 and 2017 91 METT Hoja de Datos Generales Parque Nacional Manglares del Bajo Yuna METT Hoja de Datos Generales Parque Nacional Los Haitises METT Hoja de Datos Generales Santuario de Mamiferos Marinos METT Participants List SIOBMPA Sept 11, 2017 METT Participants List TCMP Sep 12, 2017 METT score Tabulation TCMP and SIOBMPA Sep 2017 METT DR 2015, 2016 and 2017 SIOBMPA METT Evaluation September 2017 Final SIOBMPA METT 2015 and 2016 SWCay METT 2015, 2016 and 2017 Tablas METT Bajo Yuna Tablas METT Haitises Tablas METT Santuario Annual Plans and Reports: Annex A - Summary Table of Top Achievements Annex A Summary Table of Expected Top Achievements FINAL 3-2-2017 Annex B - CMBP consolidated revised WorkPlan Annex B - CMBP Equipment Inventory Sept 2016 Annex B CMBP consolidated revised WorkPlan FINAL 3-2-17 Annex B_ CMBP Equipment Inventory Sept 2017 Annex C Y3 Adjusted Activities Final CMBP Semi-annual Technical Report_Final_05-15-15 CMBP Y2 Annual Performance Report FINAL_12-30-16 CMBP Y3 Annual Performance Report FINAL 9-30-2017 v.2 CMBP Y3 Mid Term Performance Report Final v.2 CMBP Y3 Third Quarter Performance Report 07302017 CMBP Year 1 Final Technical Report_12-22-15 CMBP Year 2 work plan_revised_11-04-15 CMBP Year 3 Work Plan Submission to USAID_ 10142016 Studies: Analisis legal para el establecimiento de zonas de reservas pesqueras en Rep Dom Catalogo de las Ballenas Jorobadas Bahia de Samana - 2016 Centros y sitios de buceo contemplativo Bahia de Samana Estudio de situacion ostras y cangrejos en Sanchez, Samana Feasibility business plan apiculture in 3 Bays VF Feasibility business plan for kayaks in 3BAP VF Feasibility business plan plastic collection-recycling in Three Bays VF INFORME Viabiliadad redes suriperas en Samana Complete Jamaica Juvenile Fish Report Plan de Negocios Mercadeo y Operaciones Kayak San Lorenzo Haitises Plan de Negocios, Mercadeo y Operaciones Cayo La Farola Ecotours RV Monitoring Documents: AGRRA final report complete FY15-16 92 CMBP Mangrove Report - 11-30-2016 CMBP Monitoring plan excerpts CMBP PMP 14Sep2015 CMBP DQA forms FY15 VF May2015 Annex A_CMBP Year 3 Monitoring and Evaluation Report 10-3-2017 Annex A - CMBP Y2 ME Plan Final 12-30-2016 Annex D Revised Targets Summary FINAL 3-2-2017 INFORME FINAL Manglar de Las Garitas Bahia de Samana Informe Monitoreo Ballenas Jorobadas 2017 Informe Monitoreo de Ballenas Bahia de Samana 2016 CMBP consolidated socio-economic assessment CMBP Gender Assessment and Action Plan 9-30-2017 Three Bays Ecological Baseline Report USAID CMBA PMP FINAL 12192014 Interviews and Focus Group Discussions Table 30. Interviewed Stakeholder Groups Name of Organization Type of Organization Dominican Republic Fishers Cooperative Bahia de San Lorenzo Fishers association Women’s Association Artesanas de los Corozos Women’s association CEBSE NGO Partner Espiritu Santo Fishermen Cooperative Fishers association Consejo Dominicano de Pesca y Acuicultura (CODOPESCA) Government Fondo Nacional para el Medio Ambiente y Recursos Naturales - Fondo MARENA Government Vice-Ministry of Coastal and Marine Resources of the Environmental Ministry Government Jamaica Independent Fishers (coordinated with CCAM) Independent fishers Ministry of Industry, Commerce, Agriculture and Fisheries (MICAF), Fisheries Division Government National Environment and Planning Agency Government National Conservation Trust Fund of Jamaica NGO, trust fund Bluefields Bay Fishermen’s Friendly Society Fisher association Haiti Foundation for the Protection of Marine Biodiversity (FOPROBIM) NGO partner Organisation des Pécheurs pour la Protection de Bord de mer de Limonade (OPPBL) Fishers association (apiculture project) Association des Femmes Actives pour le Développement de Phaéton Women association (apiculture project) 93 Name of Organization Type of Organization Association des Pecheurs de Fort Liberte (APF) Fishers association National Protected Areas Agency (ANAP) Government (management plan, park management and governance) Fisheries and Aquaculture Division (DPAQ) Government Public University- Campus of Limonade Public University (key stakeholder in management plan process and feasibility studies) Initiative pour l’Environnement et le Développement Intègre en Haiti (IEDIH) Contractor (implementation of apiculture and kayaking activities) Grenadine Bank (Grenada and St. Vincent and the Grenadines) Ministry of Carriacou and Petite Martinique Affairs (MOCPMA) Government Management Board for Sandy Island Oyster Bed Marine Protected Area (SIOBMPA) Multi-stakeholder organization comprised of representatives from fisher organization, dive shop, Ports Authority, Police, and Ministry of Carriacou Ministry of Agriculture, Lands, Forestry and Fisheries Government ministry Carriacou Fishers Folks (CFF) Fishers association Carriacou and Petite Martinique Water Taxi Association (CPMWTA) Fishers livelihoods association St Vincent and the Grenadines Tobago Cays Marine Park (TCMP) Non-government organization National Parks, Rivers and Beaches Authority Quasi-governmental organization Ministry of Agriculture, Forestry, Fisheries and Rural Transformation Government ministry Mayreau Explorers Cooperative Society ltd. Fishers association Union Island Fishers Folks Cooperative Society Fishers association Mangrove Reforestation Contractor (Matthew Harvey) Contractor CMBP Sites Visited Dominican Republic  Place: Sánchez. Group visited: Fishers Cooperative Bahia de San Lorenzo.  Place: Los Corozos. Group visited: Women’s Association Artesanas de los Corozos (handicrafts).  Place: Santo Domingo. Groups visited: The Nature Conservancy DR (Carlos Garcia and Miguel Silva); Consejo Dominicano de Pesca y Acuicultura (CODOPESCA), Government 94 officials; Fondo Nacional para el Medio Ambiente y Recursos Naturales (Fondo MARENA), Government officials; Vice-Ministry of Coastal and Marine Resources of the Ministry of Environment and Natural Resources, Government officials.  Place: Samana. Groups visited: Centro para la Conservacion y Ecodesarrollo de la Bahia de Samaná y su Entorno (CEBSE), Local Implementing Partner.  Place: Los Cacaos. Groups visited: Cooperativa de Produccion, Trabajo y Servicios Múltiples Espíritu Santo, Association of fishers and others.  Place: Sabana de la Mar. Groups visited: Asociacion de Trabajadores de la Pesca de Sabana de la Mar (ASOTRAPESAMAR) , fisher association. Grenada  Place: Grenada. Group visited (two interviews) Ministry of Agriculture, Forestry and Fisheries.  Place: Carriacou. Group visited: Ministry of Carriacou and Petite Martinique Affairs (MOCPMA).  Place: Carriacou. Group visited: TNC.  Sandy Island Oyster Bed Marine Protected Area (SIOBMPA) Carriacou. Group visited (two individuals): Park rangers, one who also is part of local water taxi association.  Place: Lesterre, Carriacou. Group visited (individual): Carriacou Fisher Folks (CFF) Beneficiary of the CMBP Project – Safety-at-sea supplies. Haiti  Place: Bord de Mer Limonade. Group visited: Organisation des Pécheurs pour la Protection de Bord de mer de Limonade (OPPBL). Beneficiaries of formation, honey production and mangrove nursery.  Place: Fort Liberte. Groups visited: Association des Pecheurs de Fort Liberte (APF) .  Place: Jacquezyl. Group visited: Association des Femmes Actives pour le Développement de Phaéton (AFADFA). Formation + honey production beneficiary.  Place: Port-au-Prince: Groups visited: National Protected Areas Agency (ANAP) .  Place: Cap-Haitien. Groups visited Foundation for the Protection of Marine Biodiversity (FOPROBIM), Université de Limonade, Public University- Campus of Limonade, Initiative pour l’Environnement et le Developpement Integre en Haiti (IEDIH).  Place: Miami USA (via telephone). Person visited: Maxene Atis, TNC. Jamaica  Place: Kingston. Group visited: TNC.  Place: Kingston. Group visited (two interviews): Ministry of Industry, Commerce, Agriculture and Fisheries.  Place: Bluefields Bay. Group visited: Bluefields Bay Fishermen’s Friendly Society.  Place: Lionel Town. Group visited: C-CAM.  Place: Kingston (phone call to Pedro Cay). Group visited: Pedro Cay fishers.  Place: Kingston. Group visited: National Environment and Planning Agency.  Place: Kingston. Group visited: Jamaica Defence Force Coast Guard.  Place: Kingston. Individual visited: TNC contractor. 95 St Vincent and the Grenadines  Place: St. Vincent. Group visited: Fisheries Division.  Place: St. Vincent. Group visited: National Parks, Rivers and Beaches Authority.  Place: Tobago Cays Marine Park (TCMP), St. Vincent. Group visited: Tobago Cays Marine Park manager and staff (TCMP).  Place: Ashton, Union Island. Group visited (two individuals): Union Island Fishers Folks Cooperative Society and Mangrove Reforestation Contractor. A beneficiary of the CMBP Project for apiculture initiative and mangrove reforestation.  Place: Mayreau, SVG. Group visited (individual): Mayreau Explorers Cooperative Society Ltd. Beneficiary of the CMBP lobster pot project. 96 Annex V: Data Collection Instruments Field Interview Questionnaire Guide A multi-person evaluation team has been assembled to conduct field interviews as a main component of the Caribbean Marine Biodiversity Program (CMBP) Mid-term Evaluation. This questionnaire guide will ensure efficient and effective data collection, consistency of approach among all evaluators, and overall quality assurance as program partners and other relevant stakeholder groups are interviewed in priority sites across the region. USAID has created a set of overarching questions that will guide the mid-term evaluation. Building on this framework, the evaluation team has tailored questionnaires for five key interviewee groups, including the following (note there is no priority or logical sequence to this list; it has been created simply to organize data collection and analysis):  Group 1 – Government agencies/officials  Group 2 – Sub-grantees/MPA managers  Group 3 – Fishers and relevant stakeholders  Group 4 – Program implementer (TNC)  Group 5 – Program funder (USAID) Questionnaire structure A template data collection document (questionnaire) has been developed for each stakeholder group. USAID predefined questions are embedded in each template for easy reference during interviews, and to help enable structured data collection and subsequent analysis. Evaluators should note that primary interview questions are listed in blue. While all evaluators should maintain familiarity with USAID’s predefined questions, the primary questions in blue should be utilized to guide each interview. Pre-interview preparation Evaluators should read the Synthesis of TNC Progress Reports document prior to conducting interviews in any given program site to gain familiarity with program activities, achievements and challenges for which interviewees may have been involved, directly or indirectly. This will enable evaluator fluency of past and present program activities and thereby foster an ability to ask critical follow-up questions and dig deeper on issues as interviews unfold. A printed copy of the synthesis document should also be easily accessible during the interview if needed. Evaluation talking points at outset of interview All evaluators should open each interview with standardized talking points. That said, evaluators should strive to keep this opening to 5 minutes or less. Key talking points include:  Welcome, thank interviewee(s) for taking the time to help facilitate this important evaluation. (Evaluators should wear name tags for focus group discussions; potentially for participants as well; also, be cognizant to take too much time setting up)  Suggest a round of introductions that allows each individual present to share their name, affiliation/profession and brief description of their connection to the CMBP 97  Briefly but clearly review the purpose and expected benefits of the evaluation. Respectfully ask interviewees to allow you to get through the following points, then provide an opportunity for questions or comments before beginning the interview. o Mid-term evaluation requested by USAID and facilitated by the Alter Modus International o Provide USAID, TNC and other stakeholders with independent findings, conclusions and recommendations regarding CMBP since its inception o Evaluation will enable USAID and TNC to make any needed mid-course adjustments for remainder of CMBP, highlight successes to date, and bring evidence forward to design and/or implement future, regional marine biodiversity conservation initiatives  Evaluators will practice non-attribution in preparing the draft and final evaluation report, in other words: o All information collected will be kept confidential with the evaluation team o No responses will be attributed to specific individuals o Feel free to tell us anytime if you would like to speak off the record o Ultimately, participation in the interview is voluntary  We therefore encourage you to be candid in your responses in order to make this evaluation as beneficial as possible to all parties involved in or affected by the CMBP.  Finally, we will ask for your patience, as we have a lot of questions to get through so part of what we will need to do is keep us moving through the interview  Do you have any questions before we get started? Enabling candid interviewee responses In order for the field-based data collection process to be efficient and effective, evaluators must simultaneously build rapport and trust with stakeholders and skillfully facilitate interviews. To do this, evaluators should put the following principles into practice:  Listen actively – People want to be heard and know their viewpoints are valuable  Stay impartial – Focus at this stage on the process of conducting the interview rather than evaluating or passing judgement on anything that you hear  Show empathy – This is relevant when interviewees speak of challenging work or circumstances that are sometimes beyond their control but affect the work they do  Clarify when needed – Whenever you are unsure of something said, or didn’t understanding someone, pause and ask for clarification so that you get the data right  Draw people’s ideas out. Help participants speak clearly and share their comments. “Why is it that you would say that?” “Can you tell me more about that? “What do you mean when you say that?”  Paraphrase. Help participants see they have been heard by sharing their words. “I believe I heard you say…” “Let me see if I’m understanding you correctly…” “I think you said…did I get that right?  Manage time wisely – Make sure you manage time so that interviewees have the opportunity to answer all interview questions; this can be difficult but is of paramount importance to the data collection process that you complete the questionnaire. 98 Additional principles to guide focus group discussions  Stack people fairly. Help focus group discussion participants take turns in a conversation by “stacking” them in the order that they raise their hands.  Encourage people to contribute. Create opportunities for people to participate in the discussion without making them feel uncomfortable. “Who else has an idea to share?” “What do others think?” “I’d like to hear from those who haven’t spoken yet.”  Draw people’s ideas out. Help people to speak clearly and develop their ideas. “Why is it that you would say that?” “Can you tell me more about that? “What do you mean when you say that?”  Balance different perspectives. Broaden group participation and ensure a fair and inclusive discussion amongst all participants. “Are there other perspectives on this issue?” “Does everyone agree with what has been said?”  Listen for common ground. Help groups acknowledge different perspectives but also focus on areas of agreement. “I’m hearing differences but also similarities. For example…” “However, you all seem to agree on…” “Did I get this right?”  Clarify acronyms or uncommon labels. If someone in a group is using an acronym or a label that is perhaps unclear, whether to the evaluator or anyone else in the room, make sure to ask this person to clarify said acronym or label so everyone in the room understands its meaning. Procedures for saving and securing data Evaluators will take extra precautions to ensure that all collected data is safely and securely stored, both during and after stakeholder interviews. Extra precautions should include the following actions:  Before the interview o Create and save the data document by the standard file nomenclature below o Save each data document in a specific folder on the evaluator’s computer o Have a memory stick available for immediate use if needed  During the interview o Hit save no less than every one minute or after every question answered o Save only one version of the document throughout the interview (i.e. do not save different versions of the same document for one interview)  After the interview o Delete any extra versions that were created, internationally or unintentionally, immediately following the interview (i.e. save only one final version and delete all others to avoid confusion) o If internet is available, immediately email a copy of the data to both evaluators o Save a file clearly labeled “back-up” on an external drive (e.g. memory stick) o Clean up all notes no less than 6 hours after completing the interview following the post interview organizing steps outlined below 99 o Repeat process above following “clean-up of notes” guidance below o Create a Dropbox folder so evaluators can upload interview data online Standard file saving nomenclature Evaluators will use the below standard file naming systems when saving interview documents.  JAM_1_FisheriesDivision_11-16-17  JAM_2_C-CAM11-16-17  JAM_3_BluefieldsFishersFGD_11-16-17  JAM_4_TNC_11-16-17 Country codes: Jamaica – JAM Grenada – GRD Haiti – HTI St. Vincent and Grenadines - SVG Collecting notes during the interview Most if not all interviews will be facilitated by two evaluators. In this scenario, one evaluator should focus on reading through all questions listed in the questionnaire and listening actively to the responses provided. The other interviewer should focus on taking notes in the questionnaire template. Either evaluator can ask follow-up questions; pairs are encouraged to talk about how this may be done, working as a team, prior to conducting the interview. Each questionnaire provides a box associated with each question. All notes associated with a specific question should always be included only in the box provided. Responses to follow-up questions can also be noted in this same box. Finally, evaluators may, at their choosing, collect any supportive material that is either described verbally or provided in the “supporting evidence” row in the lower portion of the box. If only one evaluator is available to facilitate the interview, that evaluator must ensure the interview moves at a slow enough pace that quality data collection can still occur. This may entail a greater use of paraphrasing or summarizing interviewee responses, pausing to properly memorialize data, and asking the interviewee for patience as the evaluator ensures an effective data collection process. Cleaning up notes post-interview Evaluators will clean up and organize all notes following each interview. The ideal scenario is to do this immediately after the interview is completed, or shortly thereafter. If this is not possible, the notes should be cleaned up in the evening while the information is still fresh in the evaluator’s mind. Key steps for cleaning up and organizing data include the following:  Capture and create a summary of action items – Always capture needed follow-up actions throughout the interview, as well the responsible party and when the item needs to be completed. Create a bullet list of action items during the post-interview organization process and place the list in the summary box in the template document. This is particularly important if there is additional data or supporting documentation that is needed, or can be provided, that was unavailable during the interview. 100  Tighten/clean text – If you take notes in long-form, try to cut excessive or extra language so as to reduce review time during the data analysis stage (try to make the post-interview data review and analysis job easier rather than harder).  Highlight key findings – Key findings may be anything that surfaces during the interview which, relative to the CMBP goals and focus of the evaluation, seems particularly important, interesting, revealing or otherwise relevant. Feel free to also explain why you think something is a key finding. Even if brief, evaluators should attempt to highlight any key findings immediately following the interview. Place key findings in the summary box in the template document.  Highlight trends – Highlight issues, responses or other commentary that appears to be trending across interviews. Remember, a trend does not always mean responses are similar, though that is a good example. Trends can be surface level or underlying, and may reflect differences in perspective (e.g. everyone saw something differently) or common attributes (e.g. many or all respondents commented on a particular issue or challenge). Feel free to also provide brief text explaining why you think something represents a trend. Place trends in the summary box in the template document. Highlighting trends can be optional following the interview.  Highlight convergent and divergent thinking – Note when you hear similar perspectives (convergent) across interviews, but also dissimilar (divergent) perspectives.  Summarize comments – Following the interview, evaluators should provide summary comments in the box at the end of the data collection document. Evaluators should document a summary of action items (needed follow-up actions) captured during the interview, and also capture preliminary findings, trends and other notable issues that surfaced during the course of the interview. 101 Evaluation Questionnaire – Government Officials USAID predefined questions are embedded for easy reference during interviews. This will keep interviewers consistently abreast of the overarching expected results and pre-defined questions which guide the overall CMBP mid-term evaluation. The questions listed in blue below will guide interviews and are tailored to each interviewee category. Interviewee group categories  1 – Government agencies/officials  2 – Sub-grantees/MPA managers  3 – Fishers and relevant stakeholders  4 – Program implementer (TNC) Standard file saving nomenclature – Category, agency/organization name, focus group discussion, country, interview date  JAM_1_FisheriesDivision_11-16-17  JAM_2_C-CAM11-16-17  JAM_3_BluefieldsFishersFGD_11-16-17  JAM_4_TNC_11-16-17 Country codes Jamaica – JAM Grenada – GRD Haiti – HTI St. Vincent and Grenadines – SVG Background Information Evaluators: Interviewee group: Agency/Organization: Date: Location: Participant names/affiliation/contact: Name Affiliation Email Phone CMBP Expected Results  Expected Result 3: Sustainable Fisheries and Sustainable Livelihoods for Fishers, and  Expected Result 4: Expected management and governance of Marine Protected Areas (MPAs/MMAs) General questions to briefly explore at the outset (due to likely time limitations of government officials, other priority questions highlighted in red below) 102 What do you perceive as the most notable achievements of the CMBP? What do you think have been the biggest challenges of the CMBP? List of Predefined Interview Questions 1. To what extent has CMBP met its targets under Expected Result 4: Effective management and governance of MPA/MMAs? What implications does CMBP performance on Expected Result 4 have for achieving CMBP’s overall objective of reducing threats to marine-coastal biodiversity? 1a(i). What is the status (e.g., draft, approved, implemented, etc.) of existing and/or new MPA/MMA management plans in the four priority seascapes? Interview question: What is the status of existing and/or new MPA/MMA management plans in the priority CMBP seascapes (e.g., draft, approved, implemented, etc.)? Make sure to refer to the specific MPA/MMA(s). Facilitation note: Question is the same as framed for TNC. Will be good measure to assess level of knowledge/awareness that gov’t officials have of the status of management plans. Make sure notes are clear on status of plan. If officials do not have knowledge of plans (process or content), then interviewer may choose to skip the next question, as well as the follow-up questions listed here. Also requires desktop review of the yearly progress reports to assess the status of the MPA/MMA plans and how they were prepared. Notes here: Potential follow-up questions (select most appropriate question): - For MPA/MMAs that have been only drafted: What have been the main challenges during the drafting process? What has prevented the approval process? Is the draft being used and if not, what kind of management framework is in place, if any? - For MPAs/MMAs that have been approved: What were the main challenges in drafting and securing approval, and how were they overcome? Are key elements of the MPA/MMA already being implemented? If so, to what effect? - For MPA/MMAs that are being implemented: What are the main challenges to implementation? What positive impact is implementation already achieving, and how can this be built on? Supporting evidence provided during interview: Potential coding for this question: 1a(ii). What is the level of quality of these MMA management plans? (Only for countries with a drafted, approved and implemented MPA/MMAs) Interview question: What would you say are the strengths and what are the weaknesses of 1) the process of how the plans were developed and 2) the draft or final product that has been produced? Facilitation note: If officials do not have knowledge of plans (process or content), then evaluator may choose to skip the next question, as well as the potential follow-up questions 103 below. Also requires desktop review/analysis of drafted, approved and implemented management plans but may also be explored in interviews. See also interviewee MPA manager’s data collection sheet. Notes here: Process: Product: Time permitting - Potential follow-up questions (Ask respondents to provide brief responses to the following): - What stakeholder groups were involved the plan drafting process and what role did they play? - What were the strongest elements of the process that informed the resulting product? - How do you explain the weaknesses of the process or content of the document, and how can these be addressed? - Any suggestions on how management organizations can improve the plans? Any lessons learned for future plan development? Supporting evidence provided during interview: Potential coding for this question: 1a(iii). How are the MMA management plans being used, and by whom? (Only for countries with a drafted, approved and implemented MPA/MMAs) Interview question: Can you describe the processes and actions that [agency/organization name here] is taking to implement the management plan? Facilitation note: Again, high level gov’t officials may not have as much familiarity as TNC or sub-grantees. If not, interviewer should move quickly past this question, and skip the follow￾up questions. At the same time, it is important in and of itself to assess the level of knowledge that gov’t officials have of the process and end product of management plan development. But we do not want to create an awkward situation by pressing questions that high level official, due to the nature of their job, may not have details about. Notes here: Limited time so no follow-up questions Supporting evidence provided during interview: Potential coding for this question: 1b(i). In the four priority seascapes, which MMAs are using the Management Effectiveness Tracking Tool (METT) or similar tools? Interview question: Skip and go to next question. 104 Facilitation note: Answer via desktop review before conducing fieldwork and drop into interviewee category 2 data collection sheet. Notes here: Limited time so no follow-up questions Supporting evidence provided during interview: Potential coding for this question: 1b(ii). How are the results of the METT being used? Interview question: Several CMBP partners, notably local MPA management authorities, have completed the Management Effectiveness Tracking tool. How do you think management organizations are learning from and utilizing their respective METT results? Notes here: Limited time so no follow-up questions Supporting evidence provided during interview: Potential coding for this question: 1b(iii). What data or evidence does the METT provide regarding changes in MMA management effectiveness over time? Interview question: For organizations that have applied the METT more than once (tracked changes over time), what data or evidence have you seen that demonstrates that the METT accurately and effectively measures changes in management effectiveness over time? Facilitation note: Requires desktop review/analysis of METT evaluation scores and may be explored in interviews with MPA managers. Notes here: Only time permitting - Potential follow-up questions (explore only if gov’t official seems to have familiarity): - If there is a lack of data or evidence, what can be done to make these kinds of tools more practical and effective? - What other tools could be used to track and, ideally, improve management effectiveness over time? Supporting evidence provided during interview: Potential coding for this question: 1b(iv). Using this tool, has MMA management effectiveness changed (i.e., increased, decreased, or stayed the same) since CMBP’s inception? 105 Interview question: Again, for organizations that have applied the METT more than one, what changes have you have you seen in their performance, and what evidence demonstrates that those changes have indeed occurred. Facilitation note: Requires desktop review/analysis of METT evaluation scores and may be explored in interviews with MPA managers. Notes here: Only time permitting - Potential follow-up questions (explore only if gov’t official seems to have familiarity): - At what sites have changes in management effectiveness been demonstrated, and how can those changes be linked to application of the METT as a performance evaluation tool? - What aspects of management have shown an increase, a decrease or remained the same? - Do you think that the results of the METT reflect the reality? Why or why not? Supporting evidence provided during interview: Potential coding for this question: 1b(v). What factors or conditions (e.g., enabling environment, presence or absence of financing, local capacity, etc.) help explain why or how MMA management effectiveness has changed? Interview question: What are the key factors or conditions that contribute changes in management effectiveness (e.g. enabling environment, presence or absence of funding, local management capacity etc.)? For each factor or condition, why is it important and how does it contribute to improved management effectiveness? Notes here: List factors and/or condition and document why Potential follow-up questions: - Converse to the primary question, what key factors or conditions limit or prevent changes in management effectiveness, and what can be done to change these limitations or barriers to success? - What significant changes in management effectiveness have occurred under the CMBP due to the factors or conditions you cite, and at what sites (e.g. increases, decreases, remained the same etc.) Supporting evidence provided during interview: Potential coding for this question: 1c(i). What types of management infrastructure has CMBP provided? Interview question: What type of management infrastructure has CMBP provided at the priority sites which you support? 106 Facilitation note: Requires desktop review and may be explored in interviews with MPA managers and site visits. Notes here: Potential follow-up questions: - For each site, how were infrastructure needs identified, linked to CMBP goals and then prioritized/selected to receive support? Supporting evidence provided during interview: Potential coding for this question: 1c(ii). What plans has CMBP put in place to ensure that the infrastructure provided will continue to be used beyond the life of CMBP? Interview question: Are you aware of what is being done to maintain and sustain use of this infrastructure into the future? Facilitation note: This question may not be appropriate for government officials who played no role in directly supporting infrastructure development with CMBP resources. If so, ask as crafted above. Notes here: Potential follow-up questions: Supporting evidence provided during interview: Potential coding for this question: 2. To what extent has CMBP met its targets under Expected Result 3: More sustainable fisheries sector? What implications does CMBP performance on Expected Result 3 have for achieving CMBP’s overall objective of reducing threats to marine-coastal biodiversity? 2a(i). What do fishers and relevant stakeholders, such as local authorities and local fisher associations, identify as sustainable or unsustainable fishing practices? Interview question: What do fishers and local authorities identify as sustainable versus unsustainable fishing practices? What informs your knowledge of how they identify these issues? Notes here: Potential follow-up questions: - In what ways have the different awareness campaigns, training sessions, introduction of new fishing tools or other activities under CMBP been successful in increasing local authorities and fisher associations understanding of the differences between sustainable and unsustainable fishing practices? What would make those activities more effective? What other activities would you suggest? 107 - What challenges remain in defining these terms and getting key groups to adopt sustainable practices? Supporting evidence provided during interview: Potential coding for this question: 2a(ii). How is CMBP strengthening and measuring changes in local fisher associations’ capacity over time? Interview question: What is your perception of how CMBP is building and strengthening the capacity of local fisher associations, and how progress, or lack of progress, is measured over time? Notes here: Potential follow-up questions: - Have different fisher associations progressed differently under the CMBP? If so, why? - What tools or criteria are used to assess changes in fisher association capacity and strength? - How does CMBP’s efforts to build and strengthen fisher association capacity align with or influence the work of government agencies on this issue? Supporting evidence provided during interview: Potential coding for this question: 2b(i). What are CMBP, local authorities, and/or fishery associations doing to reinforce sustainable fishing practices and discourage unsustainable practices in the area? Interview question: What is being done by key program partners to reinforce sustainable fishing practices and discourage unsustainable practices at CMBP sites, and how effective are these efforts? Notes here: Only time permitting - Potential follow-up questions: - What are the best achievements to date, and how can these be built upon? Supporting evidence provided during interview: Potential coding for this question: 2b(ii). How is CMBP measuring changes in marine biodiversity and human well-being as a result of these pilots? Interview question: Are you aware of whether or not CMBP is measuring 1) changes in marine biodiversity and 2) changes in human well-being as a result of the program activities to date? If so, do you know what methods are they using? Notes here: Potential follow-up questions: 108 - What are the biggest challenges in applying tools to measure changes in marine biodiversity, and how can those challenges be overcome? - What are the biggest challenges in applying tools to measure changes in human well￾being, and how can those challenges be overcome? - Are you aware of any evidence-based data that demonstrates that CMBP has measurably improved marine biodiversity or human health or both? If so, please describe. Supporting evidence provided during interview: Potential coding for this question: 2b(iii). Based on this evidence, what are the prospects for successful expansion? Interview question: Based on the evidence you’ve described, or lack of available evidence, what is the prospect for further developing the local pilot projects, or, if they have been successful, replicating them in other areas? Notes here: Potential follow-up questions: - What are the biggest obstacles or challenges limiting the expansion of CMBP activities that reinforce and support sustainable fishing practices, and how can these challenges be overcome? - How can successful CMBP fisher programs be built upon and expanded? - What kind of support is needed for expansion? Supporting evidence provided during interview: Potential coding for this question: 2c(i). What corrective measures and/or technologies are CMBP, local authorities, and/or fishery associations promoting to mitigate or discontinue unsustainable practices? Interview question: Are aware of any actions and/or technologies are key CMBP program partners using to help reduce unsustainable fishing practices? If so, please describe. Notes here: Only time permitting - Potential follow-up questions: - What are the best achievements to date, and how can these be built upon? Supporting evidence provided during interview: Potential coding for this question: 2c(ii). What are the obstacles and/or gaps? Interview question: What are the obstacles, gaps or other challenges preventing implementation of these actions and technologies? Notes here: Only time permitting - Potential follow-up questions: 109 - What kind of coordination is needed between the government, CMBP, local management organizations and fishers in order to address the obstacles, gaps or challenges that you mention? - What impact will it have on marine biodiversity and human health if these obstacles, gaps or challenges are not addressed? Supporting evidence provided during interview: Potential coding for this question: 2d(i). To what extent are governments scaling up pilot activities implemented by CMBP that promote sustainable fishing practices? Interview question: What is your agency doing to help scale up CMBP pilot programs that are promoting sustainable fishing practices? Facilitation note: Requires interviewer familiarity with pilot projects by country/site; prepare by reading synthesis document. Notes here: Potential follow-up questions: - Are experiences and lessons from pilot projects being used to scale up efforts that promote sustainable fishing practices? If so, how? If not, is this a practical step forward. Please explain. - What are the main challenges to scaling up successful projects and what type of support is needed to make it happen? - Can your agency contribute to scaling up? If so, how? Supporting evidence provided during interview: Potential coding for this question: 3. What is the capacity (financial, managerial, and programmatic/technical) of CMBP’s local implementing partners (i.e. sub-grantees)? Facilitation note: Sub-grantees will complete and submit a self-assessment of capacity using USAID Organizational Capacity Assessment tool or other select capacity assessment tool. 3a(i). What implications does sub-grantees’ overall capacity have on CMBP’s ability to achieve expected results? Interview question: What implications does the sub-grantees’ overall capacity have on CMBP’s ability to achieve expected results #3 and #4? (Read the expected results for context and then explore the question for each sub-grantee for the near-term and the long-term) Notes here: Potential follow-up questions (Only ask if government official seems knowledgeable of sub￾grantees): - What are the sub-grantees biggest current capacity challenges, and how can they be addressed? - What kind of support would make the sub-grantees stronger partners/implementers? 110 Supporting evidence provided during interview: Potential coding for this question: 3b. What implications does sub-grantees’ capacity have for the sustainability of the activities implemented and results achieved by these organizations? Interview question: Do you think that the sub-grantees have the capacity to continue the work started after the end of the CMBP? Notes here: Potential follow-up questions Only ask if government official seems knowledgeable of sub￾grantees): - What organizational capacity challenges may prevent sub-grantees from sustaining their work over time? - What are the most successful elements of sub-grantee capacity development under the CMBP, and how can this be built on? - What kind of support is critical to ensure the near and long-term sustainability (i.e. durability over time) of activities implemented under the CMBP and results achieved by the sub-grantees to date? How long will this support be needed? - What is your level of confidence that sub-grantees now have the capacity to sustain their work over time 1) with additional CMBP support and 2) without additional CMBP support? Supporting evidence provided during interview: Potential coding for this question: Summary comments Action items – Capture the following: 1) the needed follow-up action; 2) who is the responsible party and 3) deadline for completion (e.g. sub-grantee needs to send supporting documentation they said they could provide, but which they did not have available during the interview) Preliminary findings: Trends (optional): Any other notable issues/items: 111 Evaluation Questionnaire – Sub-grantees/MPA Authorities USAID predefined questions are embedded for easy reference during interviews. This will keep interviewers consistently abreast of the overarching expected results and pre-defined questions which guide the overall CMBP mid-term evaluation. The questions listed in blue below will guide interviews and are tailored to each interviewee category. Interviewee group categories  1 – Government agencies/officials  2 – Sub-grantees/MPA managers  3 – Fishers and relevant stakeholders  4 – Program implementer (TNC) Standard file saving nomenclature – Category, agency/organization name, focus group discussion, country, interview date  JAM_1_FisheriesDivision_11-16-17  JAM_2_C-CAM11-16-17  JAM_3_BluefieldsFishersFGD_11-16-17  JAM_4_TNC_11-16-17 Country codes Jamaica – JAM Grenada – GRD Haiti – HTI St. Vincent and Grenadines – SVG Background Information Evaluators: Interviewee group: Agency/Organization: Date: Location: Participant names/affiliation/contact: Name Affiliation Email Phone CMBP Expected Results  Expected Result 3: Sustainable Fisheries and Sustainable Livelihoods for Fishers, and  Expected Result 4: Expected management and governance of Marine Protected Areas (MPAs/MMAs) General questions to briefly explore at the outset (other priority questions highlighted below) 112 What do you perceive as the most notable achievements of the CMBP? What do you think have been the biggest challenges of the CMBP? List of Predefined Interview Questions 1. To what extent has CMBP met its targets under Expected Result 4: Effective management and governance of MPA/MMAs? What implications does CMBP performance on Expected Result 4 have for achieving CMBP’s overall objective of reducing threats to marine-coastal biodiversity? 1a(i). What is the status (e.g., draft, approved, implemented, etc.) of existing and/or new MPA/MMA management plans in the four priority seascapes? Interview question: Have you developed or updated a management plan for the site or sites that you manage within CMBP priority seascapes? If yes, what is the status of the plan (e.g. draft, approved, implemented, etc.) If not, what has prevented you from taking this management planning step? Facilitation note: Make sure to note all sites under purview and be clear on status of plan. Also requires desktop review of the yearly progress reports to assess the status of the MPA/MMAs and how they were prepared. Notes here: Potential follow-up questions: - For MPA/MMAs that have been only drafted: What have been the main challenges during the drafting process? What has prevented the approval process? Is the draft being used and if not, what kind of management framework is in place, if any? - For MPAs/MMAs that have been approved: What were the main challenges in drafting and securing approval, and how were they overcome? Are key elements of the MPA/MMA already being implemented? If so, to what effect? - For MPA/MMAs that are being implemented: What are the main challenges to implementation? What positive impact is implementation already achieving, and how can this be built on? Supporting evidence provided during interview: Potential coding for this question: 1a(ii). What is the level of quality of these MMA management plans? (Only for countries with a drafted, approved and implemented MPA/MMAs) Interview question: What would you say are the strengths and what are the weaknesses of 1) the process of how the plans were developed and 2) the draft or final product that has been produced? Facilitation note: Requires desktop review/analysis of drafted, approved and implemented management plans but may also be explored in interviews. See also interviewee TNC’s data collection sheet. Notes here: 113 Process: Product: Time permitting - Potential follow-up questions (Ask respondents to provide brief responses to the following): - What stakeholder were involved in the plan drafting process and what role did they play? - What were the strongest elements of the process that informed the resulting product? - How do you explain the weaknesses of the process or content of the document, and how can these be addressed? - Any lessons learned for future plan development, for your organization or others? Supporting evidence provided during interview: Potential coding for this question: 1a(iii). How are the MMA management plans being used, and by whom? Facilitation note: Only for countries with drafted, approved and implemented MPA/MMAs Interview question: Can you describe the processes and actions that your agency/organization is taking to implement the management plan? Notes here: Time permitting - Potential follow-up questions: - What are the biggest challenges so far to implementing your management plan, and how are those challenges being addressed? - What are the best examples of implementation achievements so far, and how can these achievements be built upon? Supporting evidence provided during interview: Potential coding for this question: 1b(i). In the four priority seascapes, which MMAs are using the Management Effectiveness Tracking Tool (METT) or similar tools? Interview question: Skip and go to next question. Facilitation note: Answer via desktop review before conducing fieldwork. Notes here: For desktop review, not interviews - Potential follow-up questions: - How many times have you conducted the METT evaluation for the site or sites that you manage? - What years have you conducted the METT evaluation? 114 - What have been the main challenges in applying the METT or similar tools? What have been the main benefits? Supporting evidence provided during interview: Potential coding for this question: 1b(ii). How are the results of the METT being used? Interview question: How are you learning from and utilizing your METT results? Notes here: Time permitting - Potential follow-up questions: - Do you have procedures and mechanisms in place to formulate recommendations based on the results of the METT, and, if so, are those recommendations then implemented? - If there are no mechanisms to use the results of the METT in place, do you think there should be, and how could this be done? Supporting evidence provided during interview: Potential coding for this question: 1b(iii). What data or evidence does the METT provide regarding changes in MMA management effectiveness over time? Interview question: If you have applied the METT more than once (tracked changes over time), what data or evidence do you have that demonstrates that the METT accurately and effectively measures changes in management effectiveness over time? Facilitation note: Requires desktop review/analysis of METT evaluation scores and may be explored in interviews with MPA managers. Notes here: Time permitting - Potential follow-up questions: - How accurate do you think the METT results are? How can accuracy of responses be tested and validated? - If there is a lack of data or evidence to validate METT responses, what can be done acquire that evidence, or to make these kinds of tools more practical and effective? - What other tools could be used to track and, ideally, improve management effectiveness over time? Supporting evidence provided during interview: Potential coding for this question: 1b(iv). Using this tool, has MMA management effectiveness changed (i.e., increased, decreased, or stayed the same) since CMBP’s inception? 115 Interview question: If you have applied the METT more than once (tracked changes over time), what changes have you have you seen in your performance, and what evidence demonstrates that those changes have indeed occurred. Facilitation note: Requires desktop review/analysis of METT evaluation scores and may be explored in interviews with MPA managers. Notes here: Time permitting - Potential follow-up questions: - At what sites have changes in management effectiveness been demonstrated, and how can those changes be linked to application of the METT as a performance evaluation tool? - What aspects of management have shown an increase, a decrease or remained the same? - Do you think that the results of the METT reflect the reality? Why or why not? Supporting evidence provided during interview: Potential coding for this question: 1b(v). What factors or conditions (e.g., enabling environment, presence or absence of financing, local capacity, etc.) help explain why or how MMA management effectiveness has changed? Interview question: What are the key factors or conditions that contribute changes in management effectiveness (e.g. enabling environment, presence or absence of funding, local management capacity etc.)? For each factor or condition, why is it important and how does it contribute to improved management effectiveness? Notes here: Potential follow-up questions: - Converse to the primary question, what key factors or conditions limit or prevent changes in management effectiveness, and what can be done to change these limitations or barriers to success? - What significant changes in management effectiveness have occurred under the CMBP due to the factors or conditions you cite, and at what sites (e.g. increases, decreases, remained the same etc.) Supporting evidence provided during interview: Potential coding for this question: 1c(i). What types of management infrastructure has CMBP provided? Interview question: What type of management infrastructure has CMBP provided at the priority sites which you support? Facilitation note: Requires desktop review and may be explored in interviews with MPA managers and site visits. Notes here: 116 Time permitting - Potential follow-up questions: - For each site, how were infrastructure needs identified, linked to CMBP goals and then prioritized/selected to receive support? Supporting evidence provided during interview: Potential coding for this question: 1c(ii). What plans has CMBP put in place to ensure that the infrastructure provided will continue to be used beyond the life of CMBP? Interview question: What strategies or plans have you developed to demonstrate how the infrastructure will be maintained and sustained beyond the life of the CMBP? Does TNC request this information? Notes here: Time permitting - Potential follow-up questions: - What is the process by which to plan to use, maintain and sustain infrastructure over the long-term? - What additional support do you need to ensure longevity of infrastructure received under the CMBP? Supporting evidence provided during interview: Potential coding for this question: 2. To what extent has CMBP met its targets under Expected Result 3: More sustainable fisheries sector? What implications does CMBP performance on Expected Result 3 have for achieving CMBP’s overall objective of reducing threats to marine-coastal biodiversity? 2a(i). What do fishers and relevant stakeholders, such as local authorities and local fisher associations, identify as sustainable or unsustainable fishing practices? Interview question: What do fishers and local authorities identify as sustainable versus unsustainable fishing practices? What informs your knowledge of how they identify these issues? Notes here: Time permitting - Potential follow-up questions: - In what ways have the different awareness campaigns, training sessions, introduction of new fishing tools or other activities under CMBP been successful in increasing understanding among key groups of the differences between sustainable and unsustainable fishing practices? What would make those activities more effective? What other activities would you suggest? - What challenges remain in defining these terms and getting key groups to adopt sustainable practices? Supporting evidence provided during interview: 117 Potential coding for this question: 2a(ii). How is CMBP strengthening and measuring changes in local fisher associations’ capacity over time? Interview question: Please describe how CMBP is building and strengthening the capacity of local fisher associations, and how progress, or lack of progress, is measured over time? Notes here: Potential follow-up questions: - Have different fisher associations progressed differently under the CMBP? If so, why? - What tools or criteria do you use to assess changes in fisher association capacity and strength? - What additional support is still needed for fisher associations? Supporting evidence provided during interview: Potential coding for this question: 2b(i). What are CMBP, local authorities, and/or fishery associations doing to reinforce sustainable fishing practices and discourage unsustainable practices in the area? Interview question: What is being done by key CMBP partners, including your organization, to reinforce sustainable fishing practices and discourage unsustainable practices in at CMBP sites, and how effective are these efforts? Notes here: Time permitting - Potential follow-up questions: - What are the best achievements to date, and how can these be built upon? Supporting evidence provided during interview: Potential coding for this question: 2b(ii). How is CMBP measuring changes in marine biodiversity and human well-being as a result of these pilots? (In areas where the pilots are being implemented) Interview question: What methods, tools or approaches is CMBP using to measure 1) changes in marine biodiversity and 2) changes in human well-being as a result of the program activities you just described? Does your organization play a role in this process? If so, please describe. Notes here: Time permitting - Potential follow-up questions: - What are the biggest challenges in applying tools to measure changes in marine biodiversity, and how can those challenges be overcome? - What are the biggest challenges in applying tools to measure changes in human well￾being, and how can those challenges be overcome? 118 - Is there evidence-based data available that demonstrates that CMBP activities have measurably improved marine biodiversity or human health or both? If so, please describe. Supporting evidence provided during interview: Potential coding for this question: 2b(iii). Based on this evidence, what are the prospects for successful expansion? Interview question: What actions and/or technologies are key CMBP program partners using to help reduce unsustainable fishing practices? Notes here: Time permitting - Potential follow-up questions: - What are the biggest obstacles or challenges limiting the expansion of CMBP activities that reinforce and support sustainable fishing practices, and how can these challenges be overcome? - How can successful CMBP fisher programs be built upon and expanded? - What kind of support is needed for expansion? Supporting evidence provided during interview: Potential coding for this question: 2c(i). What corrective measures and/or technologies are CMBP, local authorities, and/or fishery associations promoting to mitigate or discontinue unsustainable practices? Interview question: What corrective measures and/or technologies are key program partners using to help discontinue or mitigate unsustainable fishing practices? Notes here: Time permitting - Potential follow-up questions: - What are the best achievements to date, and how can these be built upon? Supporting evidence provided during interview: Potential coding for this question: 2c(ii). What are the obstacles and/or gaps? Interview question: What are the obstacles, gaps or other challenges preventing implementation of these actions and technologies? Notes here: Time permitting - Potential follow-up questions: - What kind of coordination is needed between the government, CMBP, local management organizations and fishers in order to address the obstacles, gaps or challenges that you mention? - What kind of additional support is needed to address the obstacles, gaps or challenges that you mention? 119 - What impact will it have on marine biodiversity and human health if these obstacles, gaps or challenges are not addressed? Supporting evidence provided during interview: Potential coding for this question: 2d(i). To what extent are governments scaling up pilot activities implemented by CMBP that promote sustainable fishing practices? Interview question: What are relevant government agencies doing to help scale up CMBP pilot programs that are promoting sustainable fishing practices? Facilitation note: Requires interviewer familiarity with pilot projects by country/site; prepare by reading synthesis document (for areas with pilots). Notes here: Time permitting - Potential follow-up questions: - Are experiences and lessons from pilot projects being used to scale up efforts that promote sustainable fishing practices? If so, how? If not, is this a practical step forward. Please explain. - What are the main challenges to scaling up successful projects and what type of support is needed to make it happen? - Can your organization contribute to scaling up? If so, how? Supporting evidence provided during interview: Potential coding for this question: 3. What is the capacity (financial, managerial, and programmatic/technical) of CMBP’s local implementing partners (i.e. sub-grantees)? Facilitation note: Sub-grantees will complete and submit a self-assessment of capacity using USAID Organizational Capacity Assessment tool or other select capacity assessment tool. 3a(i). What implications does sub-grantees’ overall capacity have on CMBP’s ability to achieve expected results? Interview question: How does your current, overall capacity, as a local management organization, affect CMBP’s ability to achieve expected results #3 and #4?  Expected Result 3: Sustainable Fisheries and Sustainable Livelihoods for Fishers, and  Expected Result 4: Expected management and governance of Marine Protected Areas (MPAs/MMAs) Notes here: Time permitting - Potential follow-up questions: - What are your biggest current capacity challenges, and how can they be addressed? - What kind of support would strengthen your organization’s ability as a CMBP partner? Supporting evidence provided during interview: 120 Potential coding for this question: 3b. What implications does sub-grantees’ capacity have for the sustainability of the activities implemented and results achieved by these organizations? Interview question: How does your current, overall capacity, as a local management organization, affect your ability to sustain program activities and results achieved into the future. Notes here: Time permitting - Potential follow-up questions: - What capacity challenges may prevent you from sustaining your work over time? - What are the most successful elements of your capacity development under the CMBP, and how can this be built on? - What kind of support is critical to ensure the near and long-term sustainability (i.e. durability over time) of activities implemented under the CMBP and your results achieved to date? How long will this support be needed? - What is your level of confidence that you now have the capacity to sustain their work over time 1) with additional CMBP support and 2) without additional CMBP support? Supporting evidence provided during interview: Potential coding for this question: Summary comments Action items – Capture the following: 1) the needed follow-up action; 2) who is the responsible party and 3) deadline for completion (e.g. sub-grantee needs to send supporting documentation they said they could provide, but which they did not have available during the interview) Preliminary findings: Trends (optional): Any other notable issues/items: 121 Evaluation Questionnaire – Fishers and Relevant Stakeholders USAID predefined questions are embedded for easy reference during interviews. This will keep interviewers consistently abreast of the overarching expected results and pre-defined questions which guide the overall CMBP mid-term evaluation. The questions listed in blue below will guide interviews and are tailored to each interviewee category. Interviewee group categories  1 – Government agencies/officials  2 – Sub-grantees/MPA managers  3 – Fishers and relevant stakeholders  4 – Program implementer (TNC) Standard file saving nomenclature – Category, agency/organization name, focus group discussion, country, interview date  JAM_1_FisheriesDivision_11-16-17  JAM_2_C-CAM11-16-17  JAM_3_BluefieldsFishersFGD_11-16-17  JAM_4_TNC_11-16-17 Country codes Jamaica – JAM Grenada – GRD Haiti – HTI St. Vincent and Grenadines – SVG Background Information Evaluators: Interviewee group: Agency/Organization: Date: Location: Participant names/affiliation/contact: Name Affiliation Email Phone CMBP Expected Results  Expected Result 3: Sustainable Fisheries and Sustainable Livelihoods for Fishers, and  Expected Result 4: Expected management and governance of Marine Protected Areas (MPAs/MMAs) 122 General questions to briefly explore at the outset (other priority questions highlighted below) What do you perceive as the most notable achievements of the CMBP? What do you think have been the biggest challenges of the CMBP? List of Predefined Interview Questions 1. To what extent has CMBP met its targets under Expected Result 4: Effective management and governance of MPA/MMAs? What implications does CMBP performance on Expected Result 4 have for achieving CMBP’s overall objective of reducing threats to marine-coastal biodiversity? 1a(i). What is the status (e.g., draft, approved, implemented, etc.) of existing and/or new MPA/MMA management plans in the four priority seascapes? Interview question: Are you aware of the plans to manage [name MPA/MMA here]? If so, please describe what do you know about it. Facilitation note: Requires desktop review of the yearly progress reports to assess the status of the MPA/MMAs and how they were prepared. Notes here: Potential additional questions (only ask if fishers are aware of plan): - Were you involved in the development of the management plan for the area? If so, what role did you play? How did you contribute to the effort? Supporting evidence provided during the interview: Potential coding for this question: 1a(ii). What is the level of quality of these MMA management plans? (Only for countries with a drafted, approved and implemented MPA/MMAs) Only relevant if fishers said they are aware of the management plan or plans. What did you like (positives) and what did you not like (negatives) about 1) the way the plan was developed and 2) the actual plan document? Notes here: Supporting evidence provided during the interview: Potential coding for this question: 1a(iii). How are the MMA management plans being used, and by whom? (Only for countries with a drafted, approved and implemented MPA/MMAs) Interview question: Have you noticed any recent changes in how authorities are managing the [name of MPA/MMA]? 123 Facilitation note: If answered in the affirmative, capture 1) what activities and 2) about when they started to occur. Notes here: Potential follow-up questions: - What is positive about the kind of management you describe? What is negative? What improvements would you suggest? Supporting evidence provided during the interview: Potential coding for this question: 1b(v). What factors or conditions (e.g., enabling environment, presence or absence of financing, local capacity, etc.) help explain why or how MMA management effectiveness has changed? Interview question: Has work led by the local management authority help fish populations? If so, how has it helped? Notes here: Supporting evidence provided during the interview: Potential coding for this question: 1c(i). What types of management infrastructure has CMBP provided? Interview question: Has the CMBP provided any infrastructure for fishers? Please describe. Notes here: Potential follow-up questions: - How is the infrastructure beneficial to you? - How will you take care of the infrastructure so it lasts a long time? - What additional support could you use to help maintain the infrastructure? Supporting evidence provided during the interview: Potential coding for this question: 2. To what extent has CMBP met its targets under Expected Result 3: More sustainable fisheries sector? What implications does CMBP performance on Expected Result 3 have for achieving CMBP’s overall objective of reducing threats to marine-coastal biodiversity? 2a(i). What do fishers and relevant stakeholders, such as local authorities and local fisher associations, identify as sustainable or unsustainable fishing practices? Interview question: When you think of fish and fishing, what does sustainable mean to you, and what does unsustainable mean to you? 124 If they don’t know these terms or don’t have a response, that’s a finding that should be noted. But then ask it this way: What kind of fishing activities will keep the fishery around for your grandchildren, and what kind of fishing activities might result in the fishery is no longer being around for your grandchildren. Notes here: Potential follow-up questions: - In what ways have the different awareness campaigns, training sessions, introduction of new fishing tools or other activities under CMBP been successful in increasing your understanding of sustainable versus unsustainable fishing practices? What would make those activities more effective? What other activities would you suggest? -The fish sanctuary network workshop led by Rich -The Campaign for Conservation training – helpful to learn how to disseminate information - What challenges remain in getting fishers to adopt sustainable practices? Supporting evidence provided during the interview: Potential coding for this question: 2a(ii). How is CMBP strengthening and measuring changes in local fisher associations’ capacity over time? Interview question: How CMBP is helping build and strengthen your fisher association. What additional support do you still need? Notes here: Supporting evidence provided during the interview: Potential coding for this question: 2b(i). What are CMBP, local authorities, and/or fishery associations doing to reinforce appropriate fishing practices and discourage inappropriate practices in the area? Interview question: What is being done by your fisher association to support sustainable fishing practices and discourage unsustainable practices, and how effective are these efforts? Notes here: Potential follow-up questions: - What are the key challenges and how can they be addressed? - What are the best achievements to date, and how can these be built upon? Supporting evidence provided during the interview: Potential coding for this question: 2b(ii). How is CMBP measuring changes in marine biodiversity and human well-being as a result of these pilots? (In areas where the pilots are being implemented) 125 Interview question: Is CMBP doing any work to measure 1) changes in fish populations and 2) changes in the income of fishers? Notes here: Potential follow-up questions: - What are the biggest challenges in trying to measure these things? - Have CMBP activities improved fish populations or your life or both? If so, please describe. Supporting evidence provided during the interview: Potential coding for this question: 2b(iii). Based on this evidence, what are the prospects for successful expansion? Interview question: Has this project been a success? Please describe. Do you need more support to be even more successful? If yes, what kind of support? If they say the project has not been a success, ask why, and what have been the most difficult challenges? Notes here: Potential follow-up questions: - What are the biggest obstacles or challenges limiting the expansion of CMBP activities that reinforce and support sustainable fishing practices, and how can these challenges be overcome? - How can successful CMBP fisher programs be built upon and expanded? - What kind of support is needed for expansion? Supporting evidence provided during the interview: Potential coding for this question: 2c(i). What corrective measures and/or technologies are CMBP, local authorities, and/or fishery associations promoting to mitigate or discontinue unsustainable practices? Interview question: How is the CMBP helping you reduce bad or negative fishing practices (i.e. practices that are not good for the fishery)? Notes here: Time permitting - Potential follow-up questions: - What are the best achievements to date, and how can these be built upon? Supporting evidence provided during the interview: Potential coding for this question: 2c(ii). What are the obstacles and/or gaps? Interview question: Only ask if they are aware of CMBP support for sustainable fishing. What are the challenges when it comes to fishers being able give up bad practices? 126 Notes here: Potential follow-up questions: Supporting evidence provided during the interview: Potential coding for this question: 2d(i). To what extent are governments scaling up pilot activities implemented by CMBP that promote sustainable fishing practices? No question here for fishers. Notes here: Supporting evidence: policy documents, yearly progress reports, interviews Potential coding for this question: Summary comments Action items – Capture the following: 1) the needed follow-up action; 2) who is the responsible party and 3) deadline for completion (e.g. sub-grantee needs to send supporting documentation they said they could provide, but which they did not have available during the interview) Preliminary findings: Trends (optional): Any other notable issues/items: DELETED QUESTIONS – NOT RELEVANT FOR FISHERS 1b(i). In the four priority seascapes, which MMAs are using the Management Effectiveness Tracking Tool (METT) or similar tools? Not relevant for fishers. Notes here: Supporting evidence provided during the interview: Potential coding for this question: 1b(ii). How are the results of the METT being used? Not relevant for fishers. Notes here: 127 Supporting evidence provided during the interview: Potential coding for this question: 1b(iii). What data or evidence does the METT provide regarding changes in MMA management effectiveness over time? Not relevant for fishers. Notes here: Supporting evidence provided during the interview: Potential coding for this question: 1b(iv). Using this tool, has MMA management effectiveness changed (i.e., increased, decreased, or stayed the same) since CMBP’s inception? Not relevant for fishers. Notes here: Supporting evidence provided during the interview: Potential coding for this question: 1c(ii). What plans has CMBP put in place to ensure that the infrastructure provided will continue to be used beyond the life of CMBP? See above. Notes here: Potential follow-up questions: - What additional measures/ activities would help? - What additional support will you need after the end of the project and for how long? Supporting evidence provided during the interview: Potential coding for this question: 3. What is the capacity (financial, managerial, and programmatic/technical) of CMBP’s local implementing partners (i.e. sub-grantees)? Facilitation note: Sub-grantees will complete and submit a self-assessment of capacity using USAID Organizational Capacity Assessment tool or other select capacity assessment tool. 3a(i). What implications does sub-grantees’ overall capacity have on CMBP’s ability to achieve expected results? Interview question: Not relevant for fisher associations; they received support but are not considered a major sub-grantee. Notes here: 128 Supporting evidence provided during the interview: Potential coding for this question: 3b. What implications does sub-grantees’ capacity have for the sustainability of the activities implemented and results achieved by these organizations? Interview question: Not relevant for fisher associations; they received support but are not considered a major sub-grantee. Notes here: Supporting evidence provided during the interview: Potential coding for this question: 129 Evaluation Questionnaire – Program Implementer USAID predefined questions are embedded for easy reference during interviews. This will keep interviewers consistently abreast of the overarching expected results and pre-defined questions which guide the overall CMBP mid-term evaluation. The questions listed in blue below will guide interviews and are tailored to each interviewee category. Interviewee group categories  1 – Government agencies/officials  2 – Sub-grantees/MPA managers  3 – Fishers and relevant stakeholders  4 – Program implementer (TNC) Standard file saving nomenclature – Category, agency/organization name, focus group discussion, country, interview date  JAM_1_FisheriesDivision_11-16-17  JAM_2_C-CAM11-16-17  JAM_3_BluefieldsFishersFGD_11-16-17  JAM_4_TNC_11-16-17 Country codes Jamaica – JAM Grenada – GRD Haiti – HTI St. Vincent and Grenadines – SVG Background Information Evaluators Interviewee group: Agency/Organization: Date: Location: Participant names/affiliation/contact: Name Affiliation Email Phone CMBP Expected Results  Expected Result 3: Sustainable Fisheries and Sustainable Livelihoods for Fishers, and  Expected Result 4: Expected management and governance of Marine Protected Areas (MPAs/MMAs) 130 General questions to briefly explore at the outset (other priority questions highlighted below) What do you perceive as the most notable achievements of the CMBP? What do you think have been the biggest challenges of the CMBP? List of Predefined Interview Questions 1. To what extent has CMBP met its targets under Expected Result 4: Effective management and governance of MPA/MMAs? What implications does CMBP performance on Expected Result 4 have for achieving CMBP’s overall objective of reducing threats to marine-coastal biodiversity? 1a(i). What is the status (e.g., draft, approved, implemented, etc.) of existing and/or new MPA/MMA management plans in the four priority seascapes? Interview question: What is the status of existing and/or new MPA/MMA management plans in the priority CMBP seascapes that you support (e.g., draft, approved, implemented, etc.)? Make sure to refer to the specific MPA/MMA(s). Facilitation note: Make sure note all sites under purview and be clear on status of plan. Also requires desktop review of the yearly progress reports to assess the status of the MPA/MMAs and how they were prepared. Notes here: Potential follow-up questions (select most appropriate question): - For MPA/MMAs that have been only drafted: What have been the main challenges during the drafting process? What has prevented the approval process? Is the draft being used and if not, what kind of management framework is in place, if any? - For MPAs/MMAs that have been approved: What were the main challenges in drafting and securing approval, and how were they overcome? Are key elements of the MPA/MMA already being implemented? If so, to what effect? - For MPA/MMAs that are being implemented: What are the main challenges to implementation? What positive impact is implementation already achieving, and how can this be built on? Supporting evidence provided during interview: Potential coding for this question: 1a(ii). What is the level of quality of these MMA management plans? (Only for countries with a drafted, approved and implemented MPA/MMAs) Interview question: What would you say are the strengths and what are the weaknesses of 1) the process of how the plans were developed and 2) the draft or final product that has been produced? Facilitation note: Requires desktop review/analysis of drafted, approved and implemented management plans but may also be explored in interviews. See also interviewee MPA manager’s data collection sheet. 131 Notes here: Process: Product: Time permitting - Potential follow-up questions (Ask respondents to provide brief responses to the following): - What stakeholder groups were involved the plan drafting process and what role did they play? - What were the strongest elements of the process that informed the resulting product? - How do you explain the weaknesses of the process or content of the document, and how can these be addressed? - Any suggestions on how management organizations can improve the plans? Any lessons learned for future plan development? Supporting evidence provided during interview: Potential coding for this question: 1a(iii). How are the MPA/MMA management plans being used, and by whom? (Only for countries with a drafted, approved and implemented MPA/MMAs) Interview question: Can you describe the processes and actions that [agency/organization name here] is taking to implement the management plan? Notes here: Potential follow-up questions: - What are the biggest challenges so far to implementing management plans, and how are those challenges being addressed? - What are the best examples of implementation achievements so far, and how can these achievements be built upon? Supporting evidence provided during interview: Potential coding for this question: 1b(i). In the four priority seascapes, which MMAs are using the Management Effectiveness Tracking Tool (METT) or similar tools? Interview question: Skip and go to next question. Facilitation note: Answer via desktop review before conducing fieldwork and drop into interviewee category 2 data collection sheet. Still worth asking to determine the level of awareness of TNC staff regarding this issue. Supporting evidence provided during interview: Potential coding for this question: 132 1b(ii). How are the results of the METT being used? Interview question: We understand the METT has been applied to [name of sites] How do you think these organizations are learning from and utilizing their METT results? Notes here: Only time permitting - Potential follow-up questions: - What have been the main challenges for local managers in terms of applying the METT or similar tools? What have been the main benefits? - Are there procedures and mechanisms in place that enable managers to formulate recommendations based on the results of the METT, and are those recommendations then implemented? - If there are no mechanisms to use the results of the METT in place, do you think there should be, and how could this be done? Supporting evidence provided during interview: Potential coding for this question: 1b(iii). What data or evidence does the METT provide regarding changes in MMA management effectiveness over time? Interview question: For organizations that have applied the METT more than once (tracked changes over time), what data or evidence have you seen that demonstrates that the METT accurately and effectively measures changes in management effectiveness over time? Facilitation note: Requires desktop review/analysis of METT evaluation scores and may be explored in interviews with MPA managers. Notes here: Only time permitting - Potential follow-up questions: - How accurate do you think the METT results are? How can accuracy of responses be tested and validated? - If there is a lack of data or evidence to validate METT responses, what can be done acquire that evidence, or to make these kinds of tools more practical and effective? - What other tools could be used to track and, ideally, improve management effectiveness over time? Supporting evidence provided during interview: Potential coding for this question: 1b(iv). Using this tool, has MMA management effectiveness changed (i.e., increased, decreased, or stayed the same) since CMBP’s inception? Interview question: Again, for organizations that have applied the METT more than one, what changes have you have you seen in their performance, and what evidence demonstrates that those changes have indeed occurred. 133 Facilitation note: Requires desktop review/analysis of METT evaluation scores and may be explored in interviews with MPA managers. Notes here: Only time permitting - Potential follow-up questions: - At what sites have changes in management effectiveness been demonstrated, and how can those changes be linked to application of the METT as a performance evaluation tool? - What aspects of management have shown an increase, a decrease or remained the same? - Do you think that the results of the METT reflect the reality? Why or why not? Supporting evidence provided during interview: Potential coding for this question: 1b(v). What factors or conditions (e.g., enabling environment, presence or absence of financing, local capacity, etc.) help explain why or how MPA/MMA management effectiveness has changed? Interview question: When considering the organizations that have completed the METT more than one, what are the key factors or conditions that contribute changes in management effectiveness (e.g. enabling environment, presence or absence of funding, local management capacity etc.)? For each factor or condition, why is it important and how does it contribute to improved management effectiveness? Facilitation note: If they want time to review the METTs, they can get back to us via email or phone within one week. Notes here: Only time permitting - Potential follow-up questions: - Converse to the primary question, what key factors or conditions limit or prevent changes in management effectiveness, and what can be done to change these limitations or barriers to success? - What significant changes in management effectiveness have occurred under the CMBP due to the factors or conditions you cite, and at what sites (e.g. increases, decreases, remained the same etc.) Supporting evidence provided during interview: Potential coding for this question: 1c(i). What types of management infrastructure has CMBP provided? Interview question: What type of management infrastructure has CMBP provided at the priority sites which TNC supports? Facilitation note: Requires desktop review and may be explored in interviews with MPA managers and site visits. 134 Notes here: Only time permitting - Potential follow-up questions: - For each site, how were infrastructure needs identified, linked to CMBP goals and then prioritized/selected to receive support? Supporting evidence provided during interview: Potential coding for this question: 1c(ii). What plans has CMBP put in place to ensure that the infrastructure provided will continue to be used beyond the life of CMBP? Interview question: What plans has TNC put in place to ensure that the infrastructure provided will continue to be used beyond the life of CMBP? Notes here: Only time permitting - Potential follow-up questions: - What are the criteria for assessing sustainability plans for long-term infrastructure use and maintenance? - Are these criteria sufficient? If not, how can they be improved? - How should CMBP ensure the long-term use and maintenance of its infrastructure investments? Supporting evidence provided during interview: Potential coding for this question: 2. To what extent has CMBP met its targets under Expected Result 3: More sustainable fisheries sector? What implications does CMBP performance on Expected Result 3 have for achieving CMBP’s overall objective of reducing threats to marine-coastal biodiversity? 2a(i). What do fishers and relevant stakeholders, such as local authorities and local fisher associations, identify as sustainable or unsustainable fishing practices? Interview question: What do fishers and local authorities identify as sustainable versus unsustainable fishing practices? What informs your knowledge of how they identify these issues? Notes here: Only time permitting - Potential follow-up questions: - In what ways have the different awareness campaigns, training sessions, introduction of new fishing tools or other activities under CMBP been successful in increasing local authorities and fisher associations understanding of the differences between sustainable and unsustainable fishing practices? What would make those activities more effective? What other activities would you suggest? - What challenges remain in defining these terms and getting key groups to adopt sustainable practices? 135 Supporting evidence provided during interview: Potential coding for this question: 2a(ii). How is CMBP strengthening and measuring changes in local fisher associations’ capacity over time? Interview question: Please describe how TNC, via the CMBP, is building and strengthening the capacity of local fisher associations, and how progress, or lack of progress, is measured over time? Notes here: Only time permitting - Potential follow-up questions: - Have different fisher associations progressed differently under the CMBP? If so, why? - What tools or criteria do you use to assess changes in fisher association capacity and strength? - How do the results of your assessments inform TNC activities and associated support for fisher associations? Supporting evidence provided during interview: Potential coding for this question: 2b(i). What are CMBP, local authorities, and/or fishery associations doing to reinforce sustainable fishing practices and discourage unsustainable practices in the area? Interview question: What is being done by key program partners to reinforce sustainable fishing practices and discourage unsustainable practices in your priority CMBP sites, and how effective are these efforts? Notes here: Only time permitting - Potential follow-up questions: - What are the best achievements to date, and how can these be built upon? Supporting evidence provided during interview: Potential coding for this question: 2b(ii). How is CMBP measuring changes in marine biodiversity and human well-being as a result of these pilots? Interview question: What methods, tools or approaches is CMBP using to measure 1) changes in marine biodiversity and 2) changes in human well-being? Notes here: Only time permitting - Potential follow-up questions: - What are the biggest challenges in applying tools to measure changes in marine biodiversity, and how can those challenges be overcome? - What are the biggest challenges in applying tools to measure changes in human well￾being, and how can those challenges be overcome? 136 - Is there evidence-based data available that demonstrates that CMBP activities have measurably improved marine biodiversity or human health or both? If so, please describe. Supporting evidence provided during interview: Potential coding for this question: 2b(iii). Based on this evidence, what are the prospects for successful expansion? Interview question: Based on the evidence you’ve described, or lack of available evidence, what is the prospect for further developing the local pilot projects, or, if they have been successful, replicating them in other areas? Notes here: Only time permitting - Potential follow-up questions: - What are the biggest obstacles or challenges limiting the expansion of CMBP activities that reinforce and support sustainable fishing practices, and how can these challenges be overcome? - How can successful CMBP fisher programs be built upon and expanded? - What kind of support is needed for expansion? Supporting evidence provided during interview: Potential coding for this question: 2c(i). What corrective measures and/or technologies are CMBP, local authorities, and/or fishery associations promoting to mitigate or discontinue unsustainable practices? Interview question: What actions and/or technologies are key CMBP program partners using to help reduce unsustainable fishing practices? Notes here: Only time permitting - Potential follow-up questions: - What are the best achievements to date, and how can these be built upon? Supporting evidence provided during interview: Potential coding for this question: 2c(ii). What are the obstacles and/or gaps? Interview question: What are the obstacles, gaps or other challenges preventing implementation of these actions and technologies? Notes here: Potential follow-up questions: - What kind of coordination is needed between the government, CMBP, local management organizations and fishers in order to address the obstacles, gaps or challenges that you mention? 137 - What kind of additional support is needed to address the obstacles, gaps or challenges that you mention? - What impact will it have on marine biodiversity and human health if these obstacles, gaps or challenges are not addressed? Supporting evidence provided during interview: Potential coding for this question: 2d(i). To what extent are governments scaling up pilot activities implemented by CMBP that promote sustainable fishing practices? Interview question: What are relevant government agencies doing to help scale up CMBP pilot programs that are promoting sustainable fishing practices? Facilitation note: Requires interviewer familiarity with pilot projects by country/site; prepare by reading synthesis document. Notes here: Potential follow-up questions: - Are experiences and lessons from pilot projects being used to scale up efforts that promote sustainable fishing practices? If so, how? If not, is this a practical step forward. Please explain. - What are the main challenges to scaling up successful projects and what type of support is needed to make it happen? - Can TNC contribute to scaling up? If so, how? Supporting evidence provided during interview: Potential coding for this question: 3. What is the capacity (financial, managerial, and programmatic/technical) of CMBP’s local implementing partners (i.e. sub-grantees)? Facilitation note: Sub-grantees will complete and submit a self-assessment of capacity using USAID Organizational Capacity Assessment tool or other select capacity assessment tool. 3a(i). What implications does sub-grantees’ overall capacity have on CMBP’s ability to achieve expected results? Interview question: How does the sub-grantees’ overall capacity (each organization) affect CMBP’s ability to achieve expected results #3 and #4? (Read the expected results for context and then explore the question for each sub-grantee for the near-term and the long-term) Notes here: Potential follow-up questions: - What are the sub-grantees biggest current capacity challenges, and how can they be addressed? - What kind of support would make the sub-grantees stronger partners/implementers? Supporting evidence provided during interview: 138 Potential coding for this question: 3b. What implications does sub-grantees’ capacity have for the sustainability of the activities implemented and results achieved by these organizations? Interview question: How does the sub-grantees’ overall capacity (each organization) affect the ability to sustain program activities and results achieved into the future. Notes here: Potential follow-up questions: - What organizational capacity challenges may prevent sub-grantees from sustaining their work over time? - What are the most successful elements of sub-grantee capacity development under the CMBP, and how can this be built on? - What kind of support is critical to ensure the near and long-term sustainability (i.e. durability over time) of activities implemented under the CMBP and results achieved by the sub-grantees to date? How long will this support be needed? - What is your level of confidence that sub-grantees now have the capacity to sustain their work over time 1) with additional CMBP support and 2) without additional CMBP support? Supporting evidence provided during interview: Potential coding for this question: Summary comments Action items – Capture the following: 1) the needed follow-up action; 2) who is the responsible party and 3) deadline for completion (e.g. sub-grantee needs to send supporting documentation they said they could provide, but which they did not have available during the interview) Preliminary findings: Trends (optional): Any other notable issues/items: 139 Annex VI: MPA Management Plan and Action Plan Scores Action Plan for the Protection of Corals in Samana Bay Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence Vision and mission Priority level: need to have Indicator DOES NOT APPLY to ACTION PLAN Clear and overarching management goals have been defined Hi 3 3 The Action Plan does not present management goals per se, but there is a clear, general objective for the entire plan "Protect and conserve the ecological and functional integrity of coral reefs as ecosystems that play a key role in coastal construction and protection processes and are the support of the greatest marine biodiversity in the Samana Bay" (p 30), that could be described like an overarching goal. Management objectives have been defined across a range of management topics, but are not clear or concise and not SMART Med 2 2 There is a clear, general objective for the entire plan "Protect and conserve the ecological and functional integrity of coral reefs as ecosystems that play a key role in coastal construction and protection processes and are the support of the greatest marine biodiversity in the Samana Bay" and defined specific objectives for each of the four strategies: 1. Education and awareness raising; 2. Research; 3. Coral restoration; and 4. Policies and alliances. (p 30-35). In order for the score to reach a "3", these objectives need to be time-bound. Possible Subtotal Subtotal Score Subtotal Percentage 6 4 66.67% 140 Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence Plan does not demonstrate that site management is integrated within national development plans and institutional frameworks No score 0 0 Plan does not demonstrate that site management is integrated within the national development plans and institutional framework Plan clearly demonstrates that national laws which enable management of natural resources are integrated and help guide management actions Hi 3 3 "Decree 318 of 1986, Article 1 (p 29)". "Biodiversity Law 333-15, National Congress, 2015 (p 30)"."Law 64 of 2000, Chapter IV of the Biological Diversity, Article 136 (p 29)." "National Red List of Endangered, Threatened or Protected Wild Animals (Ministry of Environment, 2011) which has a total of 10 species of corals, 8 of which are in the category of Vulnerable (VU) and two in Danger (EP) (p 29)." Plan does not reference international agreements No score 0 0 Does not reference international agreements, but does mention national law very similar to the IUCN Endangered Species List: "National Red List of Endangered, Threatened or Protected Wild Animals (Ministry of Environment, 2011) which has a total of 10 species of corals, 8 which are in category of Vulnerable (VU) and two in Danger (EP) (p 29)." Plan does not make reference to rules and regulations that govern resource use with the MPA/MMA boundaries No score 0 0 The Action Plan does not make reference to rules and regulations that govern resource use with the MPA/MMA boundaries. Possible Subtotal Subtotal Score Subtotal Percentage 12 3 25.00% 141 Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence The Plan does not have an analysis of budgetary requirements for conducting all listed management activities No score 0 0 The Action Plan does not have an analysis of budgetary requirements for conducting all listed management activities A strategic planning process that identified limitations and opportunities for management was conducted, but did not inform the plan nor has it been revisited since it was first carried out Low 1 1 A strategic planning process that identifies limitations and opportunities has never been carried out. However, the plan does state "In general, studies of coral reefs in the Samaná region are scarce and micro localized." (p 8). In order to improve the score, a strategic planning process that identifies limitations and opportunities needs to be conducted with all relevant stakeholders and interested parties. The context in which the plan must operate (local, political, environmental, relevant laws) has been somewhat defined (by addressing 3 of 4 areas of context) Med 2 2 The Action Plan does mention the context in which it must operate, but does not clearly and completely define it. It does address the local, environmental and relevant laws context (p 35), and would improve if it would mention the policy context within which it must operate. A mechanism for periodic reporting has been created, is embedded in the plan, and is revisited every time the plan is updated Hi 3 Indicator DOES NOT APPLY to ACTION PLAN The plan demonstrates that the management authority has identified a diverse potential income portfolio, and is receiving funding from 1-2 sources Low 1 1 The Action Plan only makes generic reference to empowering communities to generate revenue through tourism, which we interpret as "identified some potential income sources", but does not state if it is receiving any funds from these sources, as defined in the Alliance Strategy on p 34-35. 142 The plan identifies training as an intermediate priority, partially explains the training the staff or resource users needs to carry out their roles and responsibilities, but does not include a calendar of training sessions that are needed to ensure staff knowledge/skills are up -to -date Med 2 2 The Action Plan mentions the need to train local partners to help in the coral restoration program (p 33 -34), as well as the need to train divers in the proper use of GPS units (p 31), and the need to train members of "CEBSE and Eco -Youth and will be the framework for training and explaining the importance of not using these fishing gear at least in the reef areas and of course in the area of the nursery" (p 34). The plan includes a comprehensive threats analysis that is used to prioritize and direct management actions Hi 3 Indicator DOES NOT APPLY to ACTION PLAN The plan includes a clearly defined zoning plan that covers the entire MPA/MMA and there are differentiated activities allowed in the different zones Hi 3 Indicator DOES NOT APPLY to ACTION PLAN The plan includes a vaguely defined enforcement plan Low 1 1 The Action Plan does not have an enforcement plan, but does mention enforcing regulations in two sections, both within the Alliance Strategy: "2. Request the collaboration of the authorities such as the Ministry of the Environment, the National Navy, Dominican naval auxiliaries, for the control and monitoring of the coral nursery, in an area that is marine protected area. (p 34)" and "5. To promote with the Ministry of Environment and Natural resources the regulations of maritime traffic and anchorage for the cruises in the Bay of Samana. While cruises do not pass through the managed area, this measure is preventive and/or restorative for those cora l -bottomed areas damaged by the cruise anchor in Samana Bay." (p 35) 143 The plan includes a vaguely defined infrastructure plan Low 1 1 The Action Plan does not have an infrastructure plan, but does mention the need for infrastructure within the Coral Restoration Strategy, and since this action plan has been created to promote coral reefs, the following are used as evidence to support a "0.5" score: "5. Have a minimum of three coral nurseries, placed strategically in different areas. This mechanism allows to have several points of extraction of coral tissue, in the case that there are losses in one of the nurseries by natural or anthropic factors. 6. Mark the marine area where the nurseries are present with buoys, in addition to the possible new installation points, facilitating their recognition." (p 34) The plan demonstrates that carrying capacity studies have been conducted, and the results of these studies inform management activities and the amount of people that can use and/or enjoy the MPA/MMA Hi 3 Indicator DOES NOT APPLY to ACTION PLAN The plan only vaguely mentions how resiliency to climate change is incorporated into management actions, through either adaptation or mitigation actions Low 1 1 Action Plan does not mention planning for resilience to climate change; however, it does mention "it is recommended to increase the number of nurseries and evaluate possible areas of transplantation, thus gradually creating biological corridors that allow larvae dispersion between nurseries and reefs in the area, in addition to increasing coral biomass and "genetic stocks" of better species adapted to climate changes." (28-29) Possible Subtotal Subtotal Score Subtotal Percentage 30 9 30.00% 144 Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence The plan does not demonstrate that community participation has occurred No score 0 0 The Plan does not demonstrate that community participation has occurred The plan describes how an Advisory Committee was established, was a key actor during the establishment of the MPA/MMA, and continues to inform the planning process Hi 3 Indicator DOES NOT APPLY to ACTION PLAN The plan does not demonstrate a process to balance of respecting cultural/social traditions with the MPA/MMA conservation goals No score 0 0 The planning process does not prioritize a balance of respecting cultural/social traditions with the MPA/MMA conservation goals The plan does not mention the importance of stakeholder engagement, consensus building or conflict management No score 0 0 Plan does not mention at all the importance of stakeholder engagement, or consensus building or conflict management The plan highlights the importance of education and describes a multifaceted program, mentioning at least four of these five methods: visual, active, fieldwork, radio, entertainment Hi 3 3 Visual: posters, banners and brochures. Active: train divers and clean-ups. Entertainment: games. Radio: local press radio and television (p 31-32) The plan does not have an outreach and communications plan No score 0 0 The Management Plan does not include a communications plan Possible Subtotal Subtotal Score Subtotal Percentage 15 3 20.00% 145 Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence The plan clearly demonstrates that a scientifically-sound baseline data set has been collected, analyzed and used to inform development of the MPA/MMA, and is used in active and adaptive management Hi 3 3 "In general, studies of coral reefs in the Samaná region are scarce and micro localized. (p 8)." Author did use the three existing sources of coral reef studies to create the action plan: Sang (1994), Torres (2013), and Lang & Marks (2016) (p 8-9). The plan clearly defines biophysical monitoring needs, describing the protocols to be used and the periodicity of monitoring campaigns Hi 3 3 Triannual reef monitoring using AGRRA. Monthly water quality monitoring for 1 year, and then every 3 months, with 7 parameters measured (p 32) The plan clearly defines socio￾economic monitoring needs, describing the monitoring protocol to be used and the periodicity of monitoring Hi 3 Indicator DOES NOT APPLY to ACTION PLAN The plan clearly defines governance monitoring needs, describing the monitoring protocol to be used and the periodicity of monitoring Hi 3 Indicator DOES NOT APPLY to ACTION PLAN The plan does not prioritize sustainable tourism and does not provide a list of "best management practices" for recreation providers No score 0 0 The Plan does not define sustainable tourism and does not provide a list of "best management practices" that recreation providers should adhere to The plan does not describe management effectiveness No score 0 0 The Plan does not appear to have applied management effectiveness tools to date nor does it 146 evaluation as part of monitoring and evaluation make reference to potential future application of such tools. The plan's monitoring program mentions the need for regular monitoring of locally important fisheries species, but does not appear to involve volunteer support from fishers or other groups Med 2 2 The Action Plan includes regular monitoring "To conduct a trianual assessment of the reef condition, for example with the AGRRA methodology" which includes a commercially important fisheries component (p 32); however, a distinction between fisheries independent monitoring (AGRRA), and fisheries dependent monitoring (landings) needs to be pointed out. In the case of this criteria, the emphasis is on fisheries dependent monitoring, and thus, does not acquire the complete "2" points under a "medium ranking", and neither does it mention the involvement of volunteers. Possible Subtotal Subtotal Score Subtotal Percentage 15 8 53.33% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence Maps, tables and graphics are used consistently throughout all components of the plan, and somewhat illustrate key features of the MPA/MMA or other useful information Med 2 2 Maps, tables and graphics are used to illustrate the findings mentioned in the plan, but are not used to show the sites where the action plan itself will be carried out. The plan meets fewer than six of the following criteria: clear and accessible; concise and comprehensive; accurate and objective; systematic and logical; acceptable and Low 1 1 The Action Plan meets only the following criteria: clear; concise; accurate; objective; systematic; acceptable, but it lacks accesibility, comprehensiveness, it is not very logical nor motivating; it also lacks a table of contents, a list of accronyms used and a list of graphics, maps or tables included. 147 motivating; precise and practical; focused and effective Possible Subtotal Subtotal Score Subtotal Percentage 6 3 50.00% Action Plan for the Protection of Corals in Samana Bay Total Possible 84 Total Score 30 Percentage Score 35.71% 148 Action Plan for the Protection of Marine Mammals in Samana Bay Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence Vision and mission are clearly defined Hi 3 Indicator DOES NOT APPLY to ACTION PLAN Clear and overarching management goals have been defined Hi 3 3 The Action Plan does not present management goals per se, but the four strategies presented can be interpreted as goals, and these have been clearly defined and are overarching: "a) Education and awareness; b) Research; c) Community integration; d) Conservation and protection; e) Establishment of inter-institutional and cross-sectoral alliances; f) Reinforcement of the legal framework." (p 42) See also page 44. Management objectives have been defined across a range of management topics, but are not clear or concise and not SMART Med 2 2 There is a clear objective for the entire Action Plan: "Contribute to the conservation of marine mammals in the Samana Bay, which compose regional biodiversity, based on the knowledge of their species and habitats, based on the knowledge of their species and their habitats, for their conservation and sustainable management." (p 42). And there are 6 general strategies for implementing the plan, each with its management objective, but none are SMART, as they do not include a time￾bound parameter. These are: "a) Education and awareness; b) Research; c) Community integration; d) 149 Conservation and protection; e) Establishment of inter-institutional and cross-sectoral alliances; f) Reinforcement of the legal framework." (p 42.) See also page 44. Possible Subtotal Subtotal Score Subtotal Percentage 6 5 83.33% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence Plan somewhat demonstrates how site management is integrated within national development plans and institutional frameworks Med 2 2 Plan did take into account other management plans, which are within the area in which this action plan will be implemented: "the Management plans of the marine mammal Sanctuary banks of Plata and Navidad (Ministry of Environment, 2015), Los Haitises National Park (Ministry of Environment, 2012); and the conservation Plan of the Mangrove National Park of Bajo Yuna (Ministry of Environment, 2012) and those of a rapid assessment of the ecotourism potential of the mangrove wildlife Refuge of La Jina (Bauer et al., 2012) are relevant." (p 7). "...the fundamental national legal provisions that serve as a framework for the implementation of the plan, in particular the management plan of the Marine Mammal Sanctuary of the banks of La Plata and Navidad 150 (Ministry of Environment, 2015), were identified." (p 11) Plan does not reference national laws which enable management of natural resources No score 0 0 Plan does not reference national laws Plan somewhat demonstrates that international agreements are integrated and help guide management actions Med 2 2 The Action Plan did base some of the management actions on "The Guidelines for action plans for the conservation of marine mammals in the Caribbean of the United Nations Environment Programme (UNEP, 2008) and the NOAA International Marine Mammals Action Plan (2012) were of great use." (p 11 -12) Plan only makes limited reference to rules and regulations that govern resource use with the MPA/MMA boundaries Low 1 1 The following are the only rules and regulations stated within the Action Plan: "1. the use of motorized or non - motorized aquatic equipment, such as Jet Sky or Kayak. 2. Change regulations for the observation of mothers and calves. In that sense, it is proposed to reduce to one or two (instead of three as currently exists) the number of boats observing at the same time a group of mother and calf, considering the prolonged times of occupation of this group by seasons in the bay, which would significantly reduce the risk of disturbance. 3. Do not observe individuals with serious injuries evident in their body, and in bad health to help in their recovery. 4. Prohibit aquatic events in bay waters, such as sailing 151 competitions, during whale season in order to avoid collisions or negative effects on humpback whales and especially on mothers and whales. 5. Regulate and / or limit the use of drones in the observation area." (p 58). In order to gaina higher score, the plan should clearly list more applicable rules and regulations, and not just limit itself to 5. Possible Subtotal Subtotal Score Subtotal Percentage 12 5 41.67% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence The Plan does not have an analysis of budgetary requirements for conducting all listed management activities No score 0 0 The Plan does not have an analysis of budgetary requirements for conducting all listed management activities A strategic planning process that identified limitations and opportunities for management has never been carried out No score 0 0 A strategic planning process that identifies limitations and opportunities has never been carried out The context in which the plan must operate (local, political, environmental, relevant laws) has been clearly and completely defined (by addressing all 4 areas of context) Hi 3 3 Plan mentions the relevant laws: "the Management plans of the marine mammal Sanctuary banks of Plata and Navidad (Ministry of Environment, 2015), Los Haitises National Park (Ministry of Environment, 2012); and the conservation Plan of the Mangrove National Park of Bajo Yuna (Ministry of Environment, 2012) and those of a rapid assessment of the ecotourism 152 potential of the mangrove wildlife Refuge of La Jina (Bauer et al., 2012) are relevant." (p 7). "...the fundamental national legal provisions that serve as a framework for the implementation of the plan, in particular the management plan of the Marine Mammal Sanctuary of the banks of La Plata and Navidad (Ministry of Environment, 2015), were identified." (p 11); the local context: "1. The local and regional tourism sector (hotel authorities and their diving concessionaires, tourist associations, hotels and restaurants and tourist population). 2. The fishing sector through its associations and stakeholders. 3. The tourist transport sector such as the Association of Boat Owners of the Samaná Bay (ASDUBAHISA) and the Las Cañitas - Magua Maritime Tourist Association (ASOTRATUCAMA)" (p 46); the political context: "Main Dominican public institutions of orientation, strategies, rectory, control and research relevant to the Action Plan, according to their intervention in different areas." (p 60); and the environmental context: "On the other hand, there are other organizations that play a key role in the development and implementation of the strategies presented at the national, regional and local level in the 153 conservation of coastal marine resources of the Samaná Bay play an essential role in the development and implementation of the strategies presented in this plan" (p 63). A mechanism for periodic reporting has not been created No score 0 0 A mechanism for periodic reporting has not been created Plan has not identified, described and prioritized administrative roles and procedures No score 0 0 The Plan has not identified, described and prioritized administrative roles and procedures The plan has not defined the decision -making structure or process No score 0 0 The Plan has not defined or described a structure and decision -making process The plan demonstrates that the management authority has identified a diverse potential income portfolio, and is receiving funding from 1 -2 sources Low 1 1 Plan has identified some potential income sources, but does not state if it is receiving any funds from these sources, as defined in the AStrategy for the Establishment of inter -institutional and cross -sectoral alliances (p 56 -57). Training is a low priority in the plan, and the plan does not explain staff or resource users training needs to carry out their roles and responsibilities Low 1 1 See Education and awareness strategy (p 46 -48) The plan includes a comprehensive threats analysis that is used to prioritize and direct management actions Hi 3 3 See pages 30 -39 for a complete analysis of "Anthropic impacts on marine mammal species in Bay." See page 44, a table where activites, objectives and the threats addressed are presented. The plan includes a clearly defined zoning plan that covers the entire MPA/MMA and there Hi 3 Indicator DOES NOT APPLY to ACTION PLAN 154 are differentiated activities allowed in the different zones The plan does not include an enforcement plan No score 0 0 The Plan does not have an enforcement plan The plan does not include an infrastructure plan No score 0 0 The Plan does not have an infrastructure plan Plan demonstrates that carrying capacity studies have been conducted, and the results of these studies inform management activities and the amount of people that can use and/or enjoy the MPA/MMA Hi 3 Indicator DOES NOT APPLY to ACTION PLAN The plan does not include an assessment of MPA/MMA carrying capacity No score 0 Plan does not mention planning for resilience to climate change No score 0 0 The plan does not mention planning for resilience to climate change Possible Subtotal Subtotal Score Subtotal Percentage 36 8 22.22% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence The plan demonstrates that community participation has occurred at one stage of protected area development: design, implementation and evaluation Low 1 1 The plan used information that was collected through community participation, but does mention that the communities are important during the implementation process. The plan describes how an Advisory Committee was established, was a key actor during the establishment of the Hi 3 Indicator DOES NOT APPLY to ACTION PLAN 155 MPA/MMA, and continues to inform the planning process The plan does not demonstrate a process to balance of respecting cultural/social traditions with the MPA/MMA conservation goals No score 0 0 The planning process does not prioritize a balance of respecting cultural/social traditions with the MPA/MMA conservation goals The plan does not mention the importance of stakeholder engagement, consensus building or conflict management No score 0 0 Plan does not mention at all the importance of stakeholder engagement, or consensus building or conflict management The plan highlights the importance of education and describes a multifaceted program, mentioning at least four of these five methods: visual, active, fieldwork, radio, entertainment Hi 3 3 See Education and awareness strategy (p 46-48) The plan includes a somewhat defined outreach and communications plan Med 2 2 See Education and awareness strategy (p 46-48) Possible Subtotal Subtotal Score Subtotal Percentage 15 6 40.00% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence The plan clearly demonstrates that a scientifically-sound baseline data set has been collected, analyzed and used to inform development of the MPA/MMA, and is used in active and adaptive management Hi 3 3 "A search of all unpublished or previously published information on marine mammal studies in the study area was carried out by visiting the libraries of several national institutions and websites. The library of the Nature Conservancy (TNC), the Dominican Foundation for Marine Studies 156 (FUNDEMAR) and the Marine Biology Research Center (CIBIMA) were reviewed. Special attention to library of the Center for the Conservation and Ecodevelopment of the Samaná Bay and its Environs (CEBSE, Inc.) -an institution that has carried out several general investigations in the region, and where the available information on marine Mammals of the Samaná Bay is compiled. Taking as a starting point the work on the coastal and marine biodiversity of the Samaná region of Betancourt (1998); As well as the synthesis. Additionally, the monitoring reports of whale watching from the year 2000 to the present were consulted: (Sang, 2000;) León, 2003; Betancourt, 2004; 2005; 2006; 2013; 2014, 2015; 2016 and 2017; Herrera -Moreno y Betancourt, 2009; Fernández de Robillard, 2011; 2012); Research carried out on humpback whales in the Samaná Bay (Baraff et al. (1991); Clapham et al. (1992; 1993), Mattia et al. (). (1994), Betancourt et al., 2012; Domínguez -Tejo, 2013, 2015); Shamir and Sánchez (2013) and the study on strandings (Veras, 2014). Studies carried out in Miches by the University of Columbia, based in New York, were incorporated through its center of Environment, Economy and Society 157 (Auil et al., 2015)." (p 6 -7). See also pages 12 -30 The plan mentions biophysical monitoring needs, but does not mention monitoring protocols t No score 0 0 The Plan mentions biophysical monitoring needs, but does not mention monitoring protocols to be used The plan clearly defines socio - economic monitoring needs, describing the monitoring protocol to be used and the periodicity of monitoring Hi 3 Indicator DOES NOT APPLY to ACTION PLAN The plan clearly defines governance monitoring needs, describing the monitoring protocol to be used and the periodicity of monitoring Hi 3 Indicator DOES NOT APPLY to ACTION PLAN The plan does not prioritize sustainable tourism and does not provide a list of "best management practices" for recreation providers No score 0 0 The Plan does not define sustainable tourism and does not provide a list of "best management practices" that recreation providers should adhere to The plan does not describe management effectiveness evaluation as part of monitoring and evaluation No score 0 0 The Plan does not contain any management effectiveness evaluation tools The plan's monitoring program mentions the need for regular monitoring of locally important fisheries species, but does not appear to involve volunteer support w fishers or other groups Med 2 The plan's monitoring program is not robust, does not mention the need for regular monitoring of Low 1 158 locally important fisheries species, and does not appear to involve volunteer support w fishers or other groups Plan does not mention monitoring locally important fisheries species No score 0 0 The Plan does not mention a plan for local fisheries monitoring Possible Subtotal Subtotal Score Subtotal Percentage 15 3 20.00% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence Maps, tables and graphics used consistently throughout all components, and clearly illustrate key features of the MPA/MMA or other useful information Hi 3 3 Maps, tables and graphics are used consistently throughout all components of the Plan, and clearly illustrate key features of the MPA or other useful information The plan meets fewer than six of the following criteria: clear and accessible; concise and comprehensive; accurate and objective; systematic and logical; acceptable and motivating; precise and practical; focused and effective Low 1 1 Plan meets only the following criteria: clear; concise; accurate; objective; systematic; acceptable, but it lacks accessibility, comprehensiveness, it is not very logical nor motivating; it also lacks a table of contents, a list of acronyms used and a list of graphics, maps or tables included. Possible Subtotal Subtotal Score Subtotal Percentage 6 4 66.67% Action Plan for the Protection of Marine Mammals in Samana Bay Total Possible 90 Total Score 31 Percentage Score 34.44% 159 Action Plan for the Estero Hondo Marine Mammal Sanctuary Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence Vision and mission are clearly defined Hi 3 Indicator DOES NOT APPLY to ACTION PLAN Clear and overarching management goals have been defined Hi 3 3 The Action Plan does not present management goals per se, but the four strategies presented can be interpreted as goals, and these have been clearly defined and are overarching: "a) Education and awareness; b) Research; c) Conservation and protection; d) Establishment of inter-institutional and inter-sectoral alliances; e) Strengthening the legal framework." (p 22) Management objectives have been defined across a range of management topics, but are not clear or concise and not SMART Med 2 2 There is a clear objective for the entire Action Plan: "Contribute to the conservation of the Marine Mammals of the Estero Hondo Marine Mammals Sanctuary and The North Marine Sanctuary, based on the knowledge of their species and their habitats, for their conservation and sustainable management." (p 22). And there are 6 general strategies for implementing the plan, each with its management objective, but none are SMART, as they do not include a time-bound parameter. These are: "a) Education and awareness; b) Research; c) Conservation and protection; d) Establishment of inter-institutional and 160 inter-sectoral alliances; e) Strengthening the legal framework." (p 22) See also table on p 22-23. Possible Subtotal Subtotal Score Subtotal Percentage 6 5 83.33% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence Plan somewhat demonstrates how site management is integrated within national development plans and institutional frameworks Med 2 2 "The Conservation Plan (CIBIMA / TNC, 2011) and the Management Plan (Ministry of Environment, 2014) of the Estero Hondo Marine Mammals Sanctuary are relevant." (p 6). In order to reach a haigher score, the current document would need to demonstrate how the content within these stated management plans is applied within this Action Plan. Plan clearly demonstrates that national laws which enable management of natural resources are integrated and help guide management actions Hi 3 3 "The Estero Hondo Marine Mammals Sanctuary was created by Article 37 of the Sectoral Law of Protected Areas 202-04, within Category I Areas of Strict Protection, National Red List (Environment Ministry, 2011), Later, Law 313-14 extends the limits of this Sanctuary of Estero Hondo under the denomination of Northern Marine Sanctuary." (p 3). "The manatee was protected by Law 5914-62, which was overruled by Law 307-04 on Fisheries and Aquaculture. At present, they are only protected by the Decrees 289-87 161 and 233-96 in some Protected Areas." (p 9) Plan somewhat demonstrates that international agreements are integrated and help guide management actions Med 2 2 "...activities compatible with the Generic Habitat / Species Management (IV-IUCN) category of the World Conservation Union (National Congress, 2014) (p 3). Appendix I of the Convention on International Trade in Endangered Species of Wild Fauna and Flora (CITES) and Annex II of the Cartagena Convention." (p 9). In order to reach a haigher score, the current document would need to demonstrate how the content within the International Agreements guides the activities within this Action Plan. Plan does not make reference to rules and regulations that govern resource use with the MPA/MMA boundaries No score 0 0 Plan does not make reference to rules and regulations that govern resource use with the MPA/MMA boundaries. The only reference to rules and regulations found within the Action Plan is: "National and international regulations and / or legal provisions related to protected areas, the use of the coastal zone and the protection of its biota, with emphasis on Marine Mammals (p 23), under Education and Awareness Strategy. Possible Subtotal Subtotal Score Subtotal Percentage 12 7 58.33% 162 Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence The Plan includes a general analysis of budgetary requirements, and has not identified, described and prioritized these requirements to efficiently conduct all listed management activities Low 1 1 "In relation to logistics, they have a 29- foot boat with two outboard motors in good condition and two motorcycles in poor condition. The fuel allocation is six thousand pesos per month (RD $ 6,000), an amount that is insufficient considering that the boat must patrol on a daily basis." (p 10) A strategic planning process that identified limitations and opportunities for management was conducted, but did not inform the plan nor has it been revisited since it was first carried out Low 1 1 There is no section within the plan where limitations and opportunities were identified, however, the pan does mention budget and operational limitations: "No current staff has received any level of training in relation to the conservation objects in the area, other than that derived from the experience gained during the performance of their duties. In relation to logistics, they have a 29-foot boat with two outboard motors in good condition and two motorcycles in poor condition. The fuel allocation is six thousand pesos per month (RD $ 6,000), an amount that is insufficient considering that the boat must patrol on a daily basis. The communications system is currently operating." (p 10) The context in which the plan must operate (local, political, environmental, relevant laws) has Med 2 2 Political context: " The present report covers the Protected Areas of the Estero Hondo Marine Mammals Sanctuary and The North Marine Sanctuary. Both 163 been somewhat defined (by addressing 3 of 4 areas of context) areas coincide in the space that includes the coastal and marine area of the Municipal District Estero Hondo, which belongs to the Municipality Villa Isabella, Puerto Plata Province. The North Marine Sanctuary also comprises part of the coastal zone of the Municipal District of La Isabela, located to the east, as well as the marine area adjacent to the Estero Hondo Sanctuary" (p 3). Local context: "The local management of the protected area of the Estero Hondo Marine Mammals Sanctuary (SMMEH) is carried out through an Administrator of the area, in coordination with the Provincial Director of the Environment. This administrator is also responsible for The North Marine Sanctuary (SMN)." (p 10). Environmental context: " In the coastal zone of the Estero Hondo Municipal District is located a shallow bay that communicates with the sea through Caño Estero Hondo, giving way to a reef lagoon. In this area nine ecosystems or environments (Table 1) have been delineated in its terrestrial, coastal and marine zone (Ministerio Ambiente, 2014). This bay receives the contribution of freshwater from the rivers La Ciénaga, Los Pilones and Solimán; the streams Estero Hondo, La Jaiba, Encantamiento and Caño Miguel 164 (Figure 2). This complex of environments, ecosystems and their associated biota provide habitat, shelter and food on a temporary or permanent basis, to the manatee (Trichechus manatus) and other species of Marine Mammals." (p 4) A mechanism for periodic reporting has not been created No score 0 0 A mechanism for periodic reporting has not been created The plan has not identified, described and prioritized administrative roles and procedures No score 0 0 The Plan has not identified, described and prioritized administrative roles and procedures The plan has not defined the decision -making structure or process No score 0 0 The Plan has not defined or described a structure and decision -making process The plan demonstrates that the management authority has identified a diverse potential income portfolio, and is receiving funding from 1 -2 sources Low 1 1 The Sanctuary Guides Association controls visits to the protected area, and is currently charged (RD $ 250.00) to foreign visitors, not nationals. Of this total amount, RD $ 100 is received by the Ministry of the Environment and the rest is distributed as follows: Commissions for guides (RD $ 90.00), Municipal Unit for Community Heritage, UMPC (RD $ 25.00), Community (RD $ 10.00), landowners (RD $ 10.00) and expendable material (RD $ 15.00). The Sanctuary Guides Association administers these funds. (p 10) 16 5 The plan does not include or mention any training needs or priorities for staff or resource users No score 0 0 The Plan does not include or mention any training needs for staff or resource users The plan contains a somewhat comprehensive threats analysis but is not used to direct management actions Med 2 2 They understand that fishing gear is a main threat (p 35), but they only reference marine mammals and not corals. Also effects of pollution on marine mammals (p 39 -40). It similarly lists these threats for manatees. "Anthropogenic impacts on mammal species: 1. Pollution of coastal and marine areas; 2. About the speed of boats; 3. About fishing; 4. About land tenure and use" (p 16 -20) The plan includes a clearly defined zoning plan that covers the entire MPA/MMA and there are differentiated activities allowed in the different zones Hi 3 Indicator DOES NOT APPLY to ACTION PLAN The plan does not include an enforcement plan No score 0 0 The Plan does not have an enforcement plan The plan does not include an infrastructure plan No score 0 0 The Plan does not have an infrastructure plan The plan demonstrates that carrying capacity studies have been conducted, and the results of these studies inform management activities and the amount of people that can use and/or enjoy the MPA/MMA Hi 3 Indicator DOES NOT APPLY to ACTION PLAN 166 The plan does not mention planning for resilience to climate change No score 0 0 The Plan does not mention planning for resilience to climate change Possible Subtotal Subtotal Score Subtotal Percentage 36 7 19.44% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence The plan demonstrates that community participation has occurred at one stage of protected area development: design, implementation and evaluation Low 1 1 Between 2009 and 2013, the Ministry of Tourism - MITUR and the National Institute of Technical Vocational Training (INFOTEP), with the financial support of the Japan International Cooperation Agency (JICA), implemented the Sustainable Tourism Project under Public-Private Partnerships in the Puerto Plata Province (TURISOPP) with the objective of promoting the development and promotion of tourist products and services under the public - private alliance scheme, using local resources in order to establish a sustainable tourism development mechanism with benefits to the local population. This initiative gave rise to the Municipal Units for Community Heritage (UMPC) for the tourist management of local natural and cultural values. (p 12) The plan describes how an Advisory Committee was established, was a key actor Hi 3 Indicator DOES NOT APPLY to ACTION PLAN 167 during the establishment of the MPA/MMA, and continues to inform the planning process The plan only vaguely demonstrates how early or current planning process may attempt to prioritize and achieve some balance of respecting traditional resource use and cultural protocols with conservation goals and sustainable resource use Low 1 1 The plan does not specifically address social issues, although it does mention how anthropogenic activities do impact the area, and it also states: "Develop a collaborative strategy with the participation of communities, key institutional actors and the private sector to achieve effective management and fundraising for the implementation of education, research, conservation and protection actions. A minimum economic contribution for each tourist and / or visitor to these protected areas would result in a local fund for the benefit of all. This collaborative strategy would allow greater knowledge about the Sanctuaries of Marine Mammals Estero Hondo and Marine Santuary of The North, with pride and identity with them." (p 29) The plan does not mention the importance of stakeholder engagement, consensus building or conflict management No score 0 0 Plan does not mention at all the importance of stakeholder engagement, or consensus building or conflict management The plan highlights the importance of education and describes a multifaceted program, mentioning at least four of these five methods: visual, active, fieldwork, radio, entertainment Hi 3 3 See Education and awareness strategy (p 23 -25) 168 The plan includes only a few ideas regarding outreach and communication Low 1 1 The plan does not have a communications plan, but it does include this important aspect within its Education and Awareness Strategy: "Promote communication and dissemination campaigns with local and regional media, such as the local and regional press, radio and television, covering Municipal Districts Estero Hondo and Villa Isabella on regional biodiversity, environmental values and services; and have a particular focus on knowledge and protection of Marine Mammals and their habitats." (p 23) Possible Subtotal Subtotal Score Subtotal Percentage 15 6 40.00% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence The plan clearly demonstrates that a scientifically-sound baseline data set has been collected, analyzed and used to inform development of the MPA/MMA, and is used in active and adaptive management Hi 3 3 The plan clearly demonstrates that it has knowledge of the area and has analyzed the data available, see p 3-8 The plan somewhat defines biophysical monitoring needs, and mentions monitoring protocols that currently exist, yet does not define which ptotocols are to be used Low 1 1 "• Equip the Administrator of Protected Areas with equipment for monitoring and tracking the manatee populations in Caño Estero Hondo and in the associated coastal coastal zone; as well as Marine Mammals present in the area. 169 It includes GPS, binoculars and camera with 300 mm lens. • Train the Administrator of Protected Areas and associated personnel in techniques for locating and identifying marine mammals and collecting habitat and environmental data that can be implemented as part of their monitoring and control responsibilities. The data collected should be part of a database of sightings of the species, which serves as feedback with the research strategy of this plan. • Conduct monitoring of water quality to the watercourses that reach Caño Estero Hondo such as Ríos La Jaiba, Soliman and Encantamiento and Caño Miguel, as well as at defined stations within and in the associated marine coastal zone. Such monitoring should be coordinated with the Environmental Quality Directorate of the Vice -Ministry of Environmental Management and with the Vice - Ministry of Coastal and Marine Resources." (p 26) The plan mentions biophysical monitoring needs, but does not mention monitoring protocols to be used No score 0 The plan clearly defines socio - economic monitoring needs, describing the monitoring Hi 3 Indicator DOES NOT APPLY to ACTION PLAN 170 protocol to be used and the periodicity of monitoring The plan clearly defines governance monitoring needs, describing the monitoring protocol to be used and the periodicity of monitoring Hi 3 Indicator DOES NOT APPLY to ACTION PLAN The plan somewhat prioritizes sustainable tourism and but does not provide a list of "best management practices" for recreation providers Low 1 1 The plan does not have a list of proctices it recommends, but it does state the need to address tourism and reduce its impacts through educational materials: "Continue the training of tourist guides for the care of visitors in the Sanctuary of Marine Mammals Estero Hondo and promote the incorporation of more local youth to this activity, ensuring that they are endorsed by the corresponding authorities, such as the Ministries of Environment and Tourism. Fulfilling this would raise the mission of local youth and the image of ecotourism in the region." (p 24) The plan does not prioritize sustainable tourism and does not provide a list of "best management practices" for recreation providers No score 0 The plan does not describe management effectiveness evaluation as part of monitoring and evaluation No score 0 0 The Plan does not contain any management effectiveness evaluation tools 171 The plan does not mention a plan for monitoring locally important fisheries species No score 0 0 The Plan does not mention a plan for local fisheries monitoring Possible Subtotal Subtotal Score Subtotal Percentage 15 5 33.33% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence Maps, tables and graphics are used consistently throughout all components of the plan, and clearly illustrate key features of the MPA/MMA or other useful information Hi 3 3 Maps, tables and graphics are used consistently throughout all components of the Plan, and clearly illustrate key features of the MPA or other useful information The plan meets fewer than six of the following criteria: clear and accessible; concise and comprehensive; accurate and objective; systematic and logical; acceptable and motivating; precise and practical; focused and effective Low 1 1 The Action Plan meets only the following criteria: clear; concise; accurate; objective; systematic; acceptable, but it lacks accessibility, comprehensiveness, it is not very logical nor motivating; it also lacks a table of contents, a list of acronyms used and a list of graphics, maps or tables included. Possible Subtotal Subtotal Score Subtotal Percentage 6 4 66.67% Action Plan for the Estero Hondo Marine Mammal Sanctuary Total Possible 90 Total Score 34 Percentage Score 37.78% 172 Adaptive Mooring Management Plan – Sandy Island Oyster Bed Marine Protected Area Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence Vision and mission are clearly defined Hi 3 Indicator DOES NOT APPLY to ACTION PLAN Clear and overarching management goals have been defined Hi 3 3 The goals of the plan are not stated as goals, per se, but the plan is very specific in that it aims to "put forth recommendations regarding mooring management best practices and recommends approaches to mitigate the threats of yachts. Sewage pollution was determined to be the most significant threat of yachts, as improper holding tank disposal or a lack of a holding tank leads to untreated sewage being directly discharged into the sea." (p 7). And these are all clearly addressed and are overarching throughout the plan. Management goals have been defined, but are not clear or overarching Med 2 Clear and concise management objectives across a range of management topics have been defined and are SMART (Specific, Measurable, Achievable, Relevant, and Timebound) Hi 3 3 The objectives of the plan are not stated as objectives, per se, but the plan is very specific in that it aims to "put forth recommendations regarding mooring management best practices and recommends approaches to mitigate the threats of yachts. Sewage pollution was determined to be the most significant threat of yachts, as improper holding tank disposal or a lack of a holding tank leads to 173 untreated sewage being directly discharged into the sea." (p 7). And these are all clearly addressed and are overarching throughout the plan. Possible Subtotal Subtotal Score Subtotal Percentage 6 6 100.00% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence Plan clearly demonstrates how site management is integrated within national development plans and institutional frameworks Hi 3 3 The Adpative Mooring Management Plan is presented as a separate plan, but is, in fact, a very important section of the SIOBMPA Management Plan. Although the mooring plan does not reference other national develop plans, it is clearly integrated with the one plan that enbales it to manage and protect natural resources within the MPA it was created for. Plan clearly demonstrates that national laws which enable management of natural resources are integrated and help guide management actions Hi 3 3 The Adpative Mooring Management Plan is presented as a separate plan, but is, in fact, a very important section of the SIOBMPA Management Plan. Although the mooring plan does not reference national law, it is clearly integrated with the one plan that enbales it to manage and protect natural resources within the MPA it was created for. Plan clearly demonstrates that international agreements are Hi 3 3 The plan clearly states and explains an international agreement, which is the basis for an important section of the 174 integrated and help guide management actions document: "The International Convention for the Prevention of Pollution from Ships (MARPOL) provides a global set of protocols relating to ship pollution, including regulations regarding the discharge of sewage from ships (Annex IV). The regulations address the ships' equipment and systems for the control of sewage discharge, the provision of port reception facilities for sewage, and requirements for survey and certification. Annex IV, which was revised in 2004 and adopted in 2005, apply to ships certified to carry 15 or more passengers, or that exceed 400 gross tonnages." (p 46) Plan clearly describes rules and regulations that govern resource use with the MPA/MMA boundaries Hi 3 3 The entire management plan states many applicable regulations, all pertaining to minimizing and eliminating the threats that the plan was created for. There are examples such as: "the MPA regulations (e.g. no - anchoring unless in anchoring zone, no waste/holding tank disposal)." (p 50). " the MPA regulations (e.g. no -anchoring unless in anchoring zone, no waste/holding tank disposal)." (p 47). "Regulations prohibit the discharge of 175 sewage into the sea within a specified distance from the nearest land. Ships may discharge comminuted and disinfected sewage at a distance of more than three nautical miles, while sewage that is not comminuted or disinfected may be discharged at a distance of more than 12 nautical miles from the nearest land. (p 46)." "Visitors have been welcomed and briefed on park regulations and services: o no￾anchoring o holding tank must remain closed o speed limit o proper garbage disposal o no-touching or taking anything from the reef o no fishing o reef-friendly sunscreen o contact information to report incidences (request that any issues with the moorings be reported; request that observations of other visitors breaking regulations be reported; offer assistance if anyone is harassing the visitor or theft occurs) (p42-43) Possible Subtotal Subtotal Score Subtotal Percentage 12 12 100.00% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence The Plan includes a robust analysis of budgetary requirements, and has identified, described and prioritized these requirements to Hi 3 3 As the plan focuses specifically on moorings, it does address that "Well￾managed mooring systems can be a successful sustainable financing 176 efficiently conduct all listed management activities mechanism, particularly in MPAs that are popular yachting destinations." ( 43) A strategic planning process that identified limitations and opportunities for management was conducted, its findings incorporated into the plan, and a similar process is carried out at least every two years Hi 3 3 As the plan focuses specifically on moorings, it does address that "The SIOBMPA management plans, both the original and the version revised in 2015, were assessed to determine what conservation objectives and supporting activities were relevant to consider within this management plan. Discussions with park management staff provided insight on the current mooring management practices and suggestions for future mooring management." "Grenada’s legislation was reviewed in order to identify relevant policies and determine the capacity for MPA regulations to be legally enforced. Relevant international agreements that Grenada has committed to were reviewed to reveal opportunities for recommended management measures to support such commitments." (p 17) The context in which the plan must operate (local, political, environmental, relevant laws) has been clearly and completely defined (by addressing all 4 areas of context) Hi 3 3 Local : "The primary activities that occur within the SIOBMPA are spear fishing, pot fishing, seine fishing, SCUBA diving, recreational use, water taxi use and charter craft usage." (p 14). Political : " The Fisheries Division of the Ministry of Agriculture, Lands, Forestry, Fisheries and the 177 Environment is the main government agency responsible for the management of the SIOBMPA, as well as all other MPAs in Grenada." (p 16). Environmental : "The SIOBMPA management plans, both the original and the version revised in 2015, were assessed to determine what conservation objectives and supporting activities were relevant to consider within this management plan." (p 17). Relevant laws : " Grenada’s legislation was reviewed in order to identify relevant policies and determine the capacity for MPA regulations to be legally enforced." (p 17) A mechanism for periodic reporting has been created, and is embedded in the plan Med 2 2 As the plan focuses solely on mooring management, it does not specifically present a section to ensure transparency. It does, however, put forth a mechanism for periodic adaptation of the plan, as "It was determined that as management strategies are applied and evaluated, monitoring data is collected and interpreted, and policies or regulations are implemented, aspects of the management approach will need to be adapted." (p 53) The plan has not identified, described and prioritized administrative roles and procedures No score 0 0 The Plan has not identified, described and prioritized administrative roles and procedures 178 The plan has not defined the decision -making structure or process No score 0 0 The Plan has not defined or described a structure and decision -making process The plan demonstrates that the management authority has identified a diverse potential income portfolio, and is receiving funding from a few sources Med 2 2 As the plan focuses specifically on moorings, it does address that "Well - managed mooring systems can be a successful sustainable financing mechanism, particularly in MPAs that are popular yachting destinations." ( 43). "pump -out fees would be collected to generate income over the long -term." (p 48). A higher score could be achieved if the plan stated where it is currently receiving funds from. The plan identifies training as an intermediate priority, partially explains the training the staff or resource users needs to carry out their roles and responsibilities, but does not include a calendar of training sessions that are needed to ensure staff knowledge and skills are up -to - date Med 2 2 The plan does mention training, but does not develop the training needs, or specify peridiocity, neither does it mention the need to train MPA staff in the installation of moorings. " The proposed mooring maintenance plan for SIOBMPA draws from the ‘Standard Operating Procedures for Mooring Maintenance’ (SOPMM). All staff working on mooring maintenance should be familiar with the SOPMM." (p 39) And also states that "The creation of a pump -out facility could be an alternative livelihood project associated with SIOBMPA. Funding support would be needed to purchase the equipment and provide training, but pump -out fees would be collected to 179 generate income over the long term." (p 48) The plan includes a comprehensive threats analysis that is used to prioritize and direct management actions Hi 3 3 P 10 -11 adequately explain the threats that carrying capacity and anchoring pose to the MPA. While p 28 -29 present a more complete list of threats to the MPA, such as sewage, improper solid waste disposal, sunscreen, damage to corals by snorkelers, the use of anti -fouling paint and spear fishing. The plan includes a clearly defined zoning plan that covers the entire MPA/MMA and there are differentiated activities allowed in the different zones Hi 3 3 The plan clearly states where moorings, for all different types of visiting boats, should be installed. It defines what type of anchoring system should be used according to the type of substrate at the sites and specifies coordinates of where these should be installed to avoind over -crowding and collision bewteen moored boats. See p 30 -36 for all these proposed sites. The plan includes an enforcement plan that is only vaguely defined Low 1 1 Due to the nature of the plan, which focuses solely on management of moorings, enforcement is mentioned, as one of the threats identified: " he majority of the interviewed charter yacht companies expressed support for enforced holding tank regulations within the SIOBMPA and all were willing to provide educational materials to their guests if the information was sent to them."(p 51). An enforcement plan is not defined or stated. 180 The plan includes a clearly defined infrastructure plan Hi 3 3 The plan clearly states where moorings, for all different types of visiting boats, should be installed. It defines what type of anchoring system should be used according to the type of substrate at the sites and specifies coordinates of where these should be installed to avoind over -crowding and collision bewteen moored boats. See p 30 -36 for all these proposed sites. The plan has somewhat defined MPA/MMA carrying capacity, but it is unclear if this informs management activities Med 2 2 Carrying capacity is clearly stated and defined within the plan, although a carrying capacity plan is not presented. The authors clearly state how to detemine this: "Overcrowding of an MPA can reduce its tourism appeal and decrease safety; therefore, in addition to the environmental impacts of yachts, overcrowding should be considered when establishing the yachting carrying capacity of an MPA" (p 10). "The development of this management plan involved research aimed to ensure that the addition of mooring systems within the park does not lead to an exceeded carrying capacity. Primary consideration was given to the environmental threats associated with yachts visiting the park, mainly sewage pollution, as well as the issue of overcrowding." (p 52) The plan only vaguely mentions how resiliency to climate change is Low 1 1 Due to the nature of the plan, which focuses solely on management of 181 incorporated into management actions, through either adaptation or mitigation actions moorings, climate change is not an integral part of this type of activity; however, the authors did insert a reference to climate change and how anchoring can reduce resilience: "Carilli et al. (2009) suggested that chronic stress from anchoring reduces coral resilience to global climate change." (p 10) Possible Subtotal Subtotal Score Subtotal Percentage 42 28 66.67% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence The plan demonstrates that community participation has occurred at all three stages of protected area development: design, implementation and evaluation Hi 3 3 The entire mooring plan was created through the application of polls, questionnaires and interviews; as such, the plan demonstrates that there was a high level of participation. " Information from key stakeholders including charter yacht companies, MPA management staff and dive operators was incorporated into the management plan. Stakeholder input was key in determining the feasibility of proposed management approaches, including potential strategies to mitigate the threats associated with increased mooring systems." (p 6) And p 60-63 for polls used. The plan describes how an Advisory Committee was Hi 3 Indicator DOES NOT APPLY to ACTION PLAN 182 established, was a key actor during the establishment of the MPA/MMA, and continues to inform the planning process The plan clearly demonstrates how early or current planning processes attempt to prioritize and achieve a balance of respecting traditional resource use and cultural protocols with conservation goals and sustainable resource use Hi 3 3 The entire mooring plan was created through the application of polls, questionnaires and interviews; as such, the plan demonstrates that social issues are core. " Information from key stakeholders including charter yacht companies, MPA management staff and dive operators was incorporated into the management plan. Stakeholder input was key in determining the feasibility of proposed management approaches, including potential strategies to mitigate the threats associated with increased mooring systems." (p 6) The plan somewhat mentions the importance of stakeholder engagement, consensus building and conflict management, and somewhat describes processes or mechanisms which staff and stakeholders have used in the past or may apply in the future Med 2 2 The entire mooring plan was created through the application of polls, questionnaires and interviews; as such, the plan demonstrates that consesus was a core of the plan. " Information from key stakeholders including charter yacht companies, MPA management staff and dive operators was incorporated into the management plan. Stakeholder input was key in determining the feasibility of proposed management approaches, including potential strategies to mitigate the threats associated with increased 183 mooring systems." (p 6). In order to achieve a higher score, a section that describes how to manage conflict should be included. The plan highlights the importance of education and describes a multifaceted program, mentioning at least four of these five methods: visual, active, fieldwork, radio, entertainment Hi 3 3 "Adopting the self-regulatory approach necessitates the development and distribution of site -specific educational materials. In the case of SIOBMPA, informative materials are needed to raise awareness on the threats of anchoring. It will be important to ensure that sailors are aware of the opportunities to use the moorings, the best -practices for mooring use, the mooring fees and payment methods, and the MPA regulations (e.g. no -anchoring unless in anchoring zone, no waste/holding tank disposal). There are already brochures for the SIOBMPA, which could be updated to include the aforementioned topics, and printed and distributed. Charter yacht companies could ensure each yacht has a laminated version and could review it during their safety briefs. Having laminated copies would cut down on printing resources. Tourism bureaus, customs offices, dive shops and hotels could have copies of the brochure available. Publications targeted at sailors, such as Caribbean Compass should also provide this information. This information, as well as maps with 184 the mooring fields, anchoring zones and park boundaries, should also be available on the MPAs’ websites or social media accounts (SIOBMPA does not currently have a website, but has Facebook). Additionally, it is recommended that navigational companies be approached to include MPA regulations, updated anchoring zone and mooring field locations and information on waste pump -out facilities." (p 50 -51) The plan includes a clearly defined and complete outreach and communications plan Hi 3 3 Adopting the self-regulatory approach necessitates the development and distribution of site -specific educational materials. In the case of SIOBMPA, informative materials are needed to raise awareness on the threats of anchoring. It will be important to ensure that sailors are aware of the opportunities to use the moorings, the best -practices for mooring use, the mooring fees and payment methods, and the MPA regulations (e.g. no -anchoring unless in anchoring zone, no waste/holding tank disposal). There are already brochures for the SIOBMPA, which could be updated to include the aforementioned topics, and printed and distributed. Charter yacht companies could ensure each yacht has a laminated version and could review it during their safety briefs. Having 185 laminated copies would cut down on printing resources. Tourism bureaus, customs offices, dive shops and hotels could have copies of the brochure available. Publications targeted at sailors, such as Caribbean Compass should also provide this information. This information, as well as maps with the mooring fields, anchoring zones and park boundaries, should also be available on the MPAs’ websites or social media accounts (SIOBMPA does not currently have a website, but has Facebook). Additionally, it is recommended that navigational companies be approached to include MPA regulations, updated anchoring zone and mooring field locations and information on waste pump-out facilities. (p 50-51) Possible Subtotal Subtotal Score Subtotal Percentage 15 14 93.33% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence The plan clearly demonstrates that a scientifically-sound baseline data set has been collected, analyzed and used to inform development of the MPA/MMA, and is used in active and adaptive management Hi 3 3 As the plan focuses on the impacts of anchoring and waste water discharge, proposing solutions for these, the plan does base itself on a scientifically￾sound baseline data set has been collected, analyzed and used to inform development of the plan. 186 The plan defines biophysical monitoring needs, and mentions the monitoring protocols to be used and the periodicity of monitoring campaigns Med 2 2 The plan states the needs for monitoring: "It was determined that as management strategies are applied and evaluated, monitoring data is collected and interpreted, and policies or regulations are implemented, aspects of the management approach will need to be adapted. Water quality monitoring has been recommended as a method of monitoring the severity of improper waste disposal from yachts, and in combination with visitation and yacht characteristic data, is intended to enhance the understanding of the MPA’s yachting carrying capacity." (p 53). But protocols and periodicity of these was not defined The plan somewhat defines socio - economic monitoring needs, and mentions monitoring protocols that currently exist, yet does not define which protocols are to be used Med 2 2 The plan states the needs for monitoring: "It was determined that as management strategies are applied and evaluated, monitoring data is collected and interpreted, and policies or regulations are implemented, aspects of the management approach will need to be adapted. Water quality monitoring has been recommended as a method of monitoring the severity of improper waste disposal from yachts, and in combination with visitation and yacht characteristic data, is intended to enhance the understanding of the MPA’s yachting carrying capacity." (p 187 53). But protocols and periodicity of these was not defined. The plan loosely defines governance needs, but does not mention monitoring protocols to be used No score 0 0 The Plan loosely defines governance indicators that are to be monitored, but does not mention the monitoring protocol to be used The plan somewhat prioritizes sustainable tourism and provides a list of "best management practices" that recreation providers should adhere to. This list must include: three to five best practices for recreational scuba -diving services; three to five best practices for recreational snorkeling services; and three to five best practices for recreational boat operations Med 2 2 The entire plan, focused on mooring management, visitation, waste water and zoning can be interpreted as defining sustainable tourism and how to achieve it through the components stated and analyzed. This evaluation tool or matrix would need to be revisited in order to achieve a higher score. Application of management effectiveness evaluation tools (e.g. METT) is described as an element of monitoring and evaluation, and is sometimes used to inform and update the overall plan and needed management activities Med 2 2 The plan puts forth a mechanism for periodic adaptation of the plan, as "It was determined that as management strategies are applied and evaluated, monitoring data is collected and interpreted, and policies or regulations are implemented, aspects of the management approach will need to be adapted." (p 53) The plan's monitoring program includes regular monitoring of locally important fisheries species, and involves volunteer support in the monitoring process from fishers or other groups Hi 3 Indicator DOES NOT APPLY to ACTION PLAN 188 The plan does not mention a plan for monitoring locally important fisheries species No score 0 0 The Plan does not mention a plan for local fisheries monitoring Possible Subtotal Subtotal Score Subtotal Percentage 18 11 61.11% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence Maps, tables and graphics are used consistently throughout all components of the plan, and clearly illustrate key features of the MPA/MMA or other useful information Hi 3 3 Maps, tables and graphics are used consistently throughout all components of the Plan, and clearly illustrate key features of the MPA or other useful information The plan meets at least 10 or more of the following criteria: clear and accessible; concise and comprehensive; accurate and objective; systematic and logical; acceptable and motivating; precise and practical; focused and effective Hi 3 3 The Plan meets at least 10 or more of the following criteria: clear and accessible; concise and comprehensive; accurate and objective; systematic and logical; acceptable and motivating; precise and practical; focused and effective Possible Subtotal Subtotal Score Subtotal Percentage 6 6 100.00% Adaptive Mooring Management Plan – Sandy Island Oyster Bed Marine Protected Area Total Possible 99 Total Score 77 Percentage Score 77.78% 189 Adaptive Mooring Management Plan – Tobago Cays Marine Park Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence Vision and mission are clearly defined Hi 3 Indicator DOES NOT APPLY to ACTION PLAN Clear and overarching management goals have been defined Hi 3 3 The goals of the plan are not stated as goals, per se, but the plan is very specific in that it aims to "put forth recommendations regarding mooring management best practices and recommends approaches to mitigate the threats of yachts. Sewage pollution was determined to be the most significant threat of yachts, as improper holding tank disposal or a lack of a holding tank leads to untreated sewage being directly discharged into the sea." (p 7). And these are all clearly addressed and are overarching throughout the plan. Clear and concise management objectives across a range of management topics have been defined and are SMART (Specific, Measurable, Achievable, Relevant, and Timebound) Hi 3 3 The objectives of the plan are not stated as objectives, per se, but the plan is very specific in that it aims to "put forth recommendations regarding mooring management best practices and recommends approaches to mitigate the threats of yachts. Sewage pollution was determined to be the most significant threat of yachts, as improper holding tank disposal or a lack of a holding tank leads to untreated sewage being directly discharged into the sea." (p 7). And 190 these are all clearly addressed and are overarching throughout the plan. Possible Subtotal Subtotal Score Subtotal Percentage 6 6 100.00% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence Plan clearly demonstrates how site management is integrated within national development plans and institutional frameworks Hi 3 3 The Adpative Mooring Management Plan is presented as a separate plan, but is, in fact, a very important section of the tcmp Management Plan. Although the mooring plan does not reference other national develop plans, it is clearly integrated with the one plan that enbales it to manage and protect natural resources within the MPA it was created for. Plan clearly demonstrates that national laws which enable management of natural resources are integrated and help guide management actions Hi 3 3 The Adpative Mooring Management Plan is presented as a separate plan, but is, in fact, a very important section of the TCMP Management Plan. Although the mooring plan does not reference national law, it is clearly integrated with the one plan that enbales it to manage and protect natural resources within the MPA it was created for. Plan clearly demonstrates that international agreements are integrated and help guide management actions Hi 3 3 The plan clearly states and explains an international agreement, which is the basis for an important section of the document: "The International Convention for the Prevention of 191 Pollution from Ships (MARPOL) provides a global set of protocols relating to ship pollution, including regulations regarding the discharge of sewage from ships (Annex IV). The regulations address the ships' equipment and systems for the control of sewage discharge, the provision of port reception facilities for sewage, and requirements for survey and certification. Annex IV, which was revised in 2004 and adopted in 2005, apply to ships certified to carry 15 or more passengers, or that exceed 400 gross tonnages." (p 42) Plan clearly describes rules and regulations that govern resource use with the MPA/MMA boundaries Hi 3 3 The entire management plan states many applicable regulations, all pertaining to minimizing and eliminating the threats that the plan was created for. There are examples such as: "the MPA regulations (e.g. no - anchoring unless in anchoring zone, no waste/holding tank disposal)." " the MPA regulations (e.g. no -anchoring unless in anchoring zone, no waste/holding tank disposal)." (p 47). "Regulations prohibit the discharge of sewage into the sea within a specified distance from the nearest land. Ships may discharge comminuted and disinfected sewage at a distance of more than three nautical miles, while 192 sewage that is not comminuted or disinfected may be discharged at a distance of more than 12 nautical miles from the nearest land. (p 42)." "Visitors have been welcomed and briefed on park regulations and services: o no￾anchoring o holding tank must remain closed o speed limit o proper garbage disposal o no-touching or taking anything from the reef o no fishing o reef-friendly sunscreen o contact information to report incidences (request that any issues with the moorings be reported; request that observations of other visitors breaking regulations be reported; offer assistance if anyone is harassing the visitor or theft occurs) (p 37-38 Possible Subtotal Subtotal Score Subtotal Percentage 12 12 100.00% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence The Plan includes a robust analysis of budgetary requirements, and has identified, described and prioritized these requirements to efficiently conduct all listed management activities Hi 3 3 As the plan focuses specifically on moorings, it does address that "Well￾managed mooring systems can be a successful sustainable financing mechanism, particularly in MPAs that are popular yachting destinations." (p 38) A strategic planning process that identified limitations and Hi 3 3 As the plan focuses specifically on moorings, it does address that "The 193 opportunities for management was conducted, its findings incorporated into the plan, and a similar process is carried out at least every two years TCMP management plans, both the original and the version revised in 2015, were assessed to determine what conservation objectives and supporting activities were relevant to consider within this management plan. Discussions with park management staff provided insight on the current mooring management practices and suggestions for future mooring management." "St. Vincent and the Grenadines’ legislation was reviewed in order to identify relevant policies and determine the capacity for MPA regulations to be legally enforced. Relevant international agreements that St. Vincent and the Grenadines have committed to were reviewed to reveal opportunities for recommended management measures to support such commitments." (p 15) The context in which the plan must operate (local, political, environmental, relevant laws) has been clearly and completely defined (by addressing all 4 areas of context) Hi 3 3 Local: "Yachting is the primary tourism activity occurring within the Tobago Cays Marine Park (TCMP). Anchor damage to coral reef and seagrass ecosystems is believed to be one of the key threats posed by yachts. In addition, anchors can act as a vector for the invasive seagrass Halophila stipulacea.." (p 6). Political: "The TCMP is managed under the National Parks, Rivers, and Beaches Authority of St. Vincent and the Grenadines in a 194 management arrangement between the Fisheries and Forestry Divisions, and the Marine Parks Board (Hoggarth, 2007). Co -management systems are in place for certain sites or park features, giving a share of the park’s management responsibilities to NGOs, such as Sustainable Grenadines Inc. (Hoggarth, 2007). The Marine Park Authority is made up of approximately ten staff that work out of the TCMP Visitors Centre, located on Union Island. The Park Manager, currently Kenneth Williams, leads the team of rangers and support staff. The Marine Park Authority does not rely on government subventions, but is self￾financed, gaining income from visitation and mooring fees." (p 15) "Environmental: "The TCMP management plan was assessed to determine what conservation objectives and supporting activities were relevant to consider within this management plan. Discussions with park management staff provided insight on the current mooring management practices and suggestions for future mooring management." (p 15). Relevant laws: " St. Vincent and the Grenadines’ legislation was reviewed in order to identify relevant policies and determine the capacity for MPA 195 regulations to be legally enforced. Relevant international agreements that St. Vincent and the Grenadines has committed to were reviewed to reveal opportunities for recommended management measures to support such commitments." (p 15) A mechanism for periodic reporting has been created, and is embedded in the plan Med 2 2 As the plan focuses solely on mooring management, it does not specifically present a section to ensure transparency. It does, however, put forth a mechanism for periodic adaptation of the plan, as "As management strategies are applied and evaluated, monitoring data is collected and interpreted, and policies or regulations are implemented, aspects of the management approach will need to be adapted." (p 49) The plan has not identified, described and prioritized administrative roles and procedures No score 0 0 The Plan has not identified, described and prioritized administrative roles and procedures The plan has not defined the decision -making structure or process No score 0 0 The Plan has not defined or described a structure and decision -making process The plan demonstrates that the management authority has identified a diverse potential income portfolio, and is receiving funding from a few sources Med 2 2 As the plan focuses specifically on moorings, it does address that "Well - managed mooring systems can be a successful sustainable financing mechanism, particularly in MPAs that are popular yachting destinations." (p 38). "pump -out fees would be collected 196 to generate income over the long -term." (p 44). A higher score could be achieved if the plan stated where it is currently receiving funds from. The plan identifies training as an intermediate priority, partially explains the training the staff or resource users needs to carry out their roles and responsibilities, but does not include a calendar of training sessions that are needed to ensure staff knowledge and skills are up -to -date Med 2 2 The plan does mention training, but does not develop the training needs, or specify peridiocity, neither does it mention the need to train MPA staff in the installation of moorings. " The proposed mooring maintenance plan for SIOBMPA draws from the ‘Standard Operating Procedures for Mooring Maintenance’ (SOPMM). All staff working on mooring maintenance should be familiar with the SOPMM." (p 35) And also states that "The creation of a pump -out facility could be an alternative livelihood project associated with TCMP. Funding support would be needed to purchase the equipment and provide training, but pump -out fees would be collected to generate income over the long term." (p 44) The plan includes a comprehensive threats analysis that is used to prioritize and direct management actions Hi 3 3 P 10 -11 adequately explain the threats that carrying capacity and anchoring pose to the MPA. While p 24 -26 present a more complete list of threats to the MPA, such as sewage, improper solid waste disposal, sunscreen, damage to corals by snorkelers, the use of anti -fouling paint and spear fishing. 197 The plan includes a clearly defined zoning plan that covers the entire MPA/MMA and there are differentiated activities allowed in the different zones Hi 3 3 The plan clearly states where moorings, for all different types of visiting boats, should be installed. It defines what type of anchoring system should be used according to the type of substrate at the sites and specifies coordinates of where these should be installed to avoind over -crowding and collision bewteen moored boats. See p 26 -32 for all these proposed sites. The plan includes an enforcement plan that is only vaguely defined Low 1 1 Due to the nature of the plan, which focuses solely on management of moorings, enforcement is mentioned, as one of the threats identified: "The majority of the interviewed charter yacht companies expressed support for enforced holding tank regulations within the TCMP and all were willing to provide educational materials to their guests if the information was sent to them."(p 48). An enforcement plan is not defined or stated. The plan includes a clearly defined infrastructure plan Hi 3 3 The plan clearly states where moorings, for all different types of visiting boats, should be installed. It defines what type of anchoring system should be used according to the type of substrate at the sites and specifies coordinates of where these should be installed to avoind over -crowding and collision between moored boats. See p 26 -32 for all these proposed sites. 198 The plan has somewhat defined MPA/MMA carrying capacity, but it is unclear if this informs management activities Med 2 2 Carrying capacity is clearly stated and defined within the plan, although a carrying capacity plan is not presented. The authors clearly state how to detemine this: "Overcrowding of an MPA can reduce its tourism appeal and decrease safety; therefore, in addition to the environmental impacts of yachts, overcrowding should be considered when establishing the yachting carrying capacity of an MPA" (p 10). "The development of this management plan involved research aimed to ensure that the addition of mooring systems within the park does not lead to an exceeded carrying capacity. Primary consideration was given to the environmental threats associated with yachts visiting the park, mainly sewage pollution, as well as the issue of overcrowding." (p 48) The plan only vaguely mentions how resiliency to climate change is incorporated into management actions, through either adaptation or mitigation actions Low 1 1 Due to the nature of the plan, which focuses solely on management of moorings, climate change is not an integral part of this type of activity; however, the authors did insert a reference to climate change and how anchoring can reduce resilience: "Carilli et al. (2009) suggested that chronic stress from anchoring reduces coral resilience to global climate change." (p 10) 199 Possible Subtotal Subtotal Score Subtotal Percentage 42 28 66.67% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence The plan demonstrates that community participation has occurred at all three stages of protected area development: design, implementation and evaluation Hi 3 3 The entire mooring plan was created through the application of polls, questionnaires and interviews; as such, the plan demonstrates that there was a high level of participation. "Information from key stakeholders including charter yacht companies, TCMP management staff and dive operators was incorporated into the management plan. Stakeholder input was key in determining the feasibility of proposed management approaches, including potential strategies to mitigate the threats associated with potential increases in visitation rates." (p 6) The plan describes how an Advisory Committee was established, was a key actor during the establishment of the MPA/MMA, and continues to inform the planning process Hi 3 Indicator DOES NOT APPLY to ACTION PLAN The plan clearly demonstrates how early or current planning processes attempt to prioritize and achieve a balance of respecting traditional resource use and cultural protocols Hi 3 3 The entire mooring plan was created through the application of polls, questionnaires and interviews; as such, the plan demonstrates that social issues are core. "Information from key 200 with conservation goals and sustainable resource use stakeholders including charter yacht companies, TCMP management staff and dive operators was incorporated into the management plan. Stakeholder input was key in determining the feasibility of proposed management approaches, including potential strategies to mitigate the threats associated with potential increases in visitation rates." (p 6) The plan somewhat mentions the importance of stakeholder engagement, consensus building and conflict management, and somewhat describes processes or mechanisms which staff and stakeholders have used in the past or may apply in the future Med 2 2 The entire mooring plan was created through the application of polls, questionnaires and interviews; as such, the plan demonstrates that consesus was a core of the plan. "Information from key stakeholders including charter yacht companies, TCMP management staff and dive operators was incorporated into the management plan. Stakeholder input was key in determining the feasibility of proposed management approaches, including potential strategies to mitigate the threats associated with potential increases in visitation rates." (p 6). In order to achieve a higher score, a section that describes how to manage conflict should be included. The plan highlights the importance of education and describes a multifaceted program, mentioning at least four of these five methods: Hi 3 3 "Adopting the self-regulatory approach necessitates the development and distribution of site -specific educational materials. In the case of TCMP, informative materials are needed to 201 visual, active, fieldwork, radio, entertainment raise awareness on the threats of anchoring. It will be important to ensure that sailors are aware of the opportunities to use the moorings, the best -practices for mooring use, the mooring fees and payment methods, and the MPA regulations (e.g. no - anchoring unless in anchoring zone, no waste/holding tank disposal). There are already brochures for the TCMP, which could be updated to include the aforementioned topics, and printed and distributed. Charter yacht companies could ensure each yacht has a laminated version and could review it during their safety briefs. Having laminated copies would cut down on printing resources. Tourism bureaus, customs offices, dive shops and hotels could have copies of the brochure available. Publications targeted at sailors, such as Caribbean Compass should also provide this information. This information, as well as maps with the mooring fields, anchoring zones and park boundaries, should also be available on the MPA’s website. Additionally, it is recommended that navigational companies be approached to include MPA regulations, updated anchoring zone and mooring field locations and information on waste pump -out facilities." (p 47) 202 The plan includes a clearly defined and complete outreach and communications plan Hi 3 3 "Adopting the self-regulatory approach necessitates the development and distribution of site -specific educational materials. In the case of TCMP, informative materials are needed to raise awareness on the threats of anchoring. It will be important to ensure that sailors are aware of the opportunities to use the moorings, the best -practices for mooring use, the mooring fees and payment methods, and the MPA regulations (e.g. no - anchoring unless in anchoring zone, no waste/holding tank disposal). There are already brochures for the TCMP, which could be updated to include the aforementioned topics, and printed and distributed. Charter yacht companies could ensure each yacht has a laminated version and could review it during their safety briefs. Having laminated copies would cut down on printing resources. Tourism bureaus, customs offices, dive shops and hotels could have copies of the brochure available. Publications targeted at sailors, such as Caribbean Compass should also provide this information. This information, as well as maps with the mooring fields, anchoring zones and park boundaries, should also be available on the MPA’s website. Additionally, it is recommended that 203 navigational companies be approached to include MPA regulations, updated anchoring zone and mooring field locations and information on waste pump-out facilities." (p 47) Possible Subtotal Subtotal Score Subtotal Percentage 15 14 93.33% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence The plan clearly demonstrates that a scientifically-sound baseline data set has been collected, analyzed and used to inform development of the MPA/MMA, and is used in active and adaptive management Hi 3 3 As the plan focuses on the impacts of anchoring and waste water discharge, proposing solutions for these, the plan does base itself on a scientifically￾sound baseline data set has been collected, analyzed and used to inform development of the plan. The plan defines biophysical monitoring needs, and mentions the monitoring protocols to be used and the periodicity of monitoring campaigns Med 2 2 The plan states the needs for monitoring: "Water quality monitoring has been recommended as a method of monitoring the severity of improper waste disposal from yachts, and in combination with visitation and yacht characteristic data, is intended to enhance the understanding of the MPAs’ yachting carrying capacity. This will be crucial in achieving a level of tourism that does notcompromise the MPAs’ ecosystem health or reduce the tourism appeal. A more extensive, well maintained mooring system could contribute to the success of the Tobago 204 Cays Marine Park since moorings reduce the threat of anchor damage, generate income and ensure safety to yachts. The recommended management and monitoring measures aim enhance the understanding of the MPA’s carrying capacity. This will be crucial in achieving a level of tourism that does not compromise the MPA’s ecosystem health." (p 7). But protocols and periodicity of these was not defined. The plan somewhat defines socio - economic monitoring needs, and mentions the monitoring protocol to be used and the periodicity of these monitoring campaigns Med 2 2 The plan states the needs for monitoring: "Water quality monitoring has been recommended as a method of monitoring the severity of improper waste disposal from yachts, and in combination with visitation and yacht characteristic data, is intended to enhance the understanding of the MPAs’ yachting carrying capacity. This will be crucial in achieving a level of tourism that does notcompromise the MPAs’ ecosystem health or reduce the tourism appeal. A more extensive, well maintained mooring system could contribute to the success of the Tobago Cays Marine Park since moorings reduce the threat of anchor damage, generate income and ensure safety to yachts. The recommended management and monitoring measures aim enhance the understanding of the MPA’s 205 carrying capacity. This will be crucial in achieving a level of tourism that does not compromise the MPA’s ecosystem health." (p 7). But protocols and periodicity of these was not defined. The plan loosely defines governance needs, but does not mention monitoring protocols to be used No score 0 0 The Plan loosely defines governance indicators that are to be monitored, but does not mention the monitoring protocol to be used The plan somewhat prioritizes sustainable tourism and provides a list of "best management practices" that recreation providers should adhere to. This list must include: three to five best practices for recreational scuba -diving services; three to five best practices for recreational snorkeling services; and three to five best practices for recreational boat operations Med 2 2 The entire plan, focused on mooring management, visitation, waste water and zoning can be interpreted as defining sustainable tourism and how to achieve it through the components stated and analyzed. This evaluation tool or matrix would need to be revisited in order to achieve a higher score. Application of management effectiveness evaluation tools (e.g. METT) is described as an element of monitoring and evaluation, and is sometimes used to inform and update the overall plan and needed management activities Med 2 2 The plan states the needs for monitoring: "Water quality monitoring has been recommended as a method of monitoring the severity of improper waste disposal from yachts, and in combination with visitation and yacht characteristic data, is intended to enhance the understanding of the MPAs’ yachting carrying capacity. This will be crucial in achieving a level of tourism that does notcompromise the MPAs’ ecosystem health or reduce the 206 tourism appeal. A more extensive, well maintained mooring system could contribute to the success of the Tobago Cays Marine Park since moorings reduce the threat of anchor damage, generate income and ensure safety to yachts. The recommended management and monitoring measures aim enhance the understanding of the MPA’s carrying capacity. This will be crucial in achieving a level of tourism that does not compromise the MPA’s ecosystem health." (p 7). But protocols and periodicity of these was not defined. The plan's monitoring program includes regular monitoring of locally important fisheries species, and involves volunteer support in the monitoring process from fishers or other groups Hi 3 Indicator DOES NOT APPLY to ACTION PLAN The plan does not mention a plan for monitoring locally important fisheries species No score 0 0 The Plan does not mention a plan for local fisheries monitoring Possible Subtotal Subtotal Score Subtotal Percentage 18 11 61.11% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence Maps, tables and graphics are used consistently throughout all components of the plan, and Hi 3 3 Maps, tables and graphics are used consistently throughout all components of the Plan, and clearly illustrate key 207 clearly illustrate key features of the MPA/MMA or other useful information features of the MPA or other useful information The plan meets at least 10 or more of the following criteria: clear and accessible; concise and comprehensive; accurate and objective; systematic and logical; acceptable and motivating; precise and practical; focused and effective Hi 3 3 The Plan meets at least 10 or more of the following criteria: clear and accessible; concise and comprehensive; accurate and objective; systematic and logical; acceptable and motivating; precise and practical; focused and effective Possible Subtotal Subtotal Score Subtotal Percentage 6 6 100.00% Adaptive Mooring Management Plan – Tobago Cays Marine Park Total Possible 99 Total Score 77 Percentage Score 77.78% 208 Sandy Island Oyster Bed Marine Protected Area Management Plan Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence Vision and mission are not defined at all No score 0 0 Vision and mission are not defined at all Clear and overarching management goals have been defined Hi 3 3 "Expert review highlighted that the objectives, goals and proposed activities in the 2007 draft management plan were not designed in such a way as to be conducive to straight-forward implementation. Upon review, the 2007 draft management plan was, for the most part, found not to have not been implemented as envisioned in the section on conservation objectives and strategies. To address the updated management objectives and the threats discussed during consultation meetings, updated management goals and activities are proposed (P 29-39)." Clear and concise management objectives across a range of management topics have been defined and are SMART (Specific, Measurable, Achievable, Relevant, and Timebound) Hi 3 3 The plan has clear and concise management objectives across a range of management topics have been defined and are SMART (Specific, Measurable, Achievable, Relevant, and Timebound), as the plan has been created to be carried out between 2015 and 2025. (p 29-39). Possible Subtotal Subtotal Score Subtotal Percentage 9 6 66.67% 209 Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence Plan clearly demonstrates how site management is integrated within national development plans and institutional frameworks Hi 3 3 See p 40-41, where the legal framework of the MPA is stated, and includes national development plans. Plan clearly demonstrates that national laws which enable management of natural resources are integrated and help guide management actions Hi 3 3 See p 40-41, where the legal framework of the MPA is stated, and includes national laws and how these were a basis for the plan. Plan only makes limited reference to international agreements and it is unclear if these agreements help guide management actions Low 1 1 The Plan only mentions that "Grenada ratified the Convention on Biological Diversity (CBD), declaring its commitment to establish protected areas. As a member of the CBD, Grenada committed to preserve 10% of its terrestrial and marine habitat." (p 40). In order to acquire a higher score, the plan could have made a stronger reference to International Agreements (such as Ramsar/IUCN/Cartagena Convention/ UNDP SDG), to provide a better reference that these help guide management actions. The authors would like to note that the Caribbean Challenge is not mentioned, of which Grenada is a part of. Plan clearly describes rules and regulations that govern resource use with the MPA/MMA boundaries Hi 3 3 The Plan clearly states the rules and regulations that apply to the different zones within the MPA, and these were established through a stakeholder workshop (p 42-43): "Restricted 210 Fishing Zone: Only three types of fishing will be permitted: Hand lining, trolling and net fishing. SIOBMPA fishing regulations apply on line test, mesh size and techniques for net usage. Recreational Non-extractive Zone: Only recreational activities will be allowed. Extraction of any resource, living or dead is prohibited. All vessels must operate at a no-wake speed within the zone and follow designated route into and away from shore. Reef Fish Protection Zone: Extraction of reef fish is prohibited. The only fishing allowed is seine fishing for pelagic species, following Sandy Island/Oyster Bed MPA fishing regulations." Possible Subtotal Subtotal Score Subtotal Percentage 12 10 83.33% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence The Plan does not have an analysis of budgetary requirements for conducting all listed management activities No score 0 0 The Plan does not have an analysis of budgetary requirements for conducting all listed management activities. A strategic planning process that identified limitations and opportunities for management was conducted, its findings incorporated into the plan, and a Hi 3 3 The plan links threats to management goals and activities. Further, the plan notes that these issues were thoroughly explored during workshops which were held during the 2015 plan revision 211 similar process is carried out at least every two years process and insinuates that similar workshops will be held in the future. The context in which the plan must operate (local, political, environmental, relevant laws) has been somewhat defined (by addressing 3 of 4 areas of context) Med 2 2 The context in which the Plan must operate has been somewhat defined by addressing 3 of 4 areas of context: local, environmental, relevant laws: "A site -level, local co -management arrangement was deemed appropriate due to the MPA’s distance from mainland Grenada, its decentralized governance structures with powers delegated to the Ministry of Petite Martinique and Carriacou Affairs, and given the size and composition of the MPA user groups. However, the Marine Protected Areas Regulations have been problematic to interpret and implement particularly when defining the governance structure for a MPA and the relationship between the National MPA Board, site -level management Boards and MPA managers." (p 42) A mechanism for periodic reporting has not been created No score 0 0 The plan does not describe any kind of mechanism for periodic reporting. The plan has somewhat identified and described administrative roles and procedures, but has not prioritized these procedures Med 2 2 Administration is mentioned as a section in the plan, but is not developed: "Staff: insert org chart; Training; Facilities, equipment; Insert mooring maintenance; SOPs and/or Plan (p 49). That said, the plan does describe the roles and responsibilities 212 of various staff members in fair detail (p 38 - 40) The plan has not defined the decision -making structure or process No score 0 0 The Plan has not defined or described a structure and decision -making process The plan demonstrates that the management authority has identified a diverse potential income portfolio, and is receiving funding from 1 -2 sources Low 1 1 The Plan only makes very limited reference to potential revenue generating mechanisms. However, the plan demonstrates past and present sources of funding to support various management activities and the 2015 workshops that informed update of the Plan. The plan identifies training as an intermediate priority, partially explains the training the staff or resource users need to carry out their roles and responsibilities, but does not include a calendar of training sessions that are needed to ensure staff knowledge and skills are up -to -date Med 2 2 The plan makes reference to various training that staff have received in the past across a range of activities, but does not clearly articulate future training needs or provide any kind of training calendar. The plan includes a comprehensive threats analysis that is used to prioritize and direct management actions Hi 3 3 "During the 2015 update of the management plan, the current threats facing each of the conservation targets were assessed." (p 26 -28) The plan includes a clearly defined zoning plan that covers the entire MPA/MMA and there are differentiated activities allowed in the different zones Hi 3 3 "The zoning plan for the SIOBMPA was a direct result of the stakeholder workshop held in April 2006. The zoning of the SIOBMPA supports the implementation of the conservation actions that were identified in the previous steps and contributes to 213 achieving the overall objectives of the SIOBMPA." (p 42 -43) The plan includes a somewhat defined enforcement plan Med 2 2 The Plan clearly demonstrates, in their accomplishments section (p 9), that they have carried out activities geared towards enforcement, such as: Enforcement: 1. Enforcement trainings; 2. Development and implementation of park regulations; 3. Establishment of patrol regime; 4. Working with boaters to prevent tying up in mangroves during non -emergencies; 5. Production of park work boat (locally built as part of C -FISH project). The plan includes a clearly defined infrastructure plan Hi 3 The plan includes a somewhat defined infrastructure plan Med 2 2 The Plan clearly demonstrates, in their accomplishments section (p 9), that they have carried out activities geared towards an infrastructure plan, such as: Development of a mooring buoy program: 1. Deployment of 21 moorings; 2. Training in mooring maintenance. The plan would acquire a bigger score if it had a section of infrastructure needed and to be built, and not just those accomplished to date. The plan does not include an assessment of MPA/MMA carrying capacity No score 0 0 The Plan does not include an assessment of the MPA carrying capacity The plan does not mention planning for resilience to climate change No score 0 0 The Plan does not mention planning for resilience to climate change 214 Possible Subtotal Subtotal Score Subtotal Percentage 42 20 47.62% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence The plan demonstrates that community participation has occurred at two stages of protected area development: design, implementation or evaluation Med 2 2 The plan mentions participation of the community during the design phase, "A series of consultation meetings were held to discuss accomplishments, emerging issues, and management challenges." (p 6), and "The zoning plan for the SIOBMPA was a direct result of the stakeholder workshop held in April 2006. (p 42)," but it does not mention how community participation will be ensured in the implementation and evaluation phases, which would help to increase the score. The plan does not have provisions for the creation of an Advisory Committee and such a committee has not been established No score 0 0 The Plan does not have provisions for the creation of an Advisory Committee and this committee has not been established. The plan somewhat demonstrates how early or current planning process may attempt to prioritize and achieve some balance of respecting traditional resource use and cultural protocols with conservation goals and sustainable resource use Med 2 2 The plan somewhat mentions the importance of stakeholder Low 1 1 The plan mentions participation of the community during the design phase, "A 215 engagement, consensus building and conflict management, but does not describe processes or mechanisms which could be used series of consultation meetings were held to discuss accomplishments, emerging issues, and management challenges." (p 6), and, "The zoning plan for the SIOBMPA was a direct result of the stakeholder workshop held in April 2006. (p 42)," which can be seen as consensus building, but it does not mention how conflicts wil be managed, which would help to increase the score. The plan somewhat highlights the importance of education and describes a multifaceted program, mentioning at least three of these five methods: visual, active, fieldwork, radio, entertainment Med 2 2 The Plan clearly demonstrates, in their accomplishments section (p 9), that they have carried out activities geared towards educaiton, such as: Traditional print and radio media: Signage, brochures, posters, radio broadcasts. Community events: Beach cleanups, vegetation plantings, Beach Fest, Fisher engagements, Fisher trainings on FADs. Youth programs: Kids With Cameras, IMANI Youth Training Project community partnership. Social Media: and novel media outlets: Facebook, Twitter, YouTube, Google+, three (3) short videos, local jingle. The plan would acquire a better score if it had a section of how it will carry out an education program, and not just those accomplished to date. (p 9 -10) 216 The plan includes only a few ideas regarding outreach and communication Low 1 1 The plan mentions the importance of outreach and communications throuhgout, and includes some actions, but does not have include a defined outreach and communications plan. The following text was included: Please note that in this process the following sections were identified as needing revision, but they were not able to be updated in this review. Further effort is still needed to: Update the SIOBMPA stakeholder analysis; Update the SIOBMPA communications plan; Update the SIOBMPA financial plan (p 7). Possible Subtotal Subtotal Score Subtotal Percentage 18 8 44.44% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence The plan clearly demonstrates that a scientifically-sound baseline data set has been collected, analyzed and used to inform development of the MPA/MMA, and is used in active and adaptive management Hi 3 3 The Plan clearly demonstrates that a scientifically-sound baseline data set has been collected, analyzed and used to inform development of the MPA, and is used in active and adaptive management. See p 13-25, where all the Key Resources are stated, analyzed and used to create the plan. The plan defines biophysical monitoring needs, and mentions the monitoring protocols to be Med 2 2 The Plan clearly demonstrates, in their accomplishments section (p 9), that they have carried out activities geared towards monitoring, such as: 217 used and the periodicity of monitoring campaigns Biophysical Monitoring: Baseline surveys and use of existing data on coral reef health, Coral monitoring element of Caribbean Reef Buddy project, Reef Check training, AGRRA training and establishment of permanent monitoring sites, Beach profiling training. Research: Working with Canadian PhD student. The plan would acquire a better score if it had a section of how it will carry out the biophysical monitoring program, and not just those accomplished to date. (p 9-10) The plan somewhat defines socio - economic monitoring needs, and mentions the monitoring protocol to be used and the periodicity of these monitoring campaigns Med 2 2 The Plan clearly demonstrates, in their accomplishments section (p 9), that they have carried out activities geared towards socio -economic monitoring, such as: Socio -Economic Monitoring: Component of C -FISH program. The plan would acquire a better score if it had a section of how it will carry out the socio -economic monitoring program, and not just those accomplished to date. (p 9 -10). "SIOBMPA conducted socioeconomic monitoring using the SocMon protocol in 2012. A copy of the report on findings is attached." No report was attached to the MGMT Plan analyzed (p 49). The plan loosely defines governance needs, but does not No score 0 0 The Plan loosely defines governance indicators that are to be monitored, but 218 mention monitoring protocols to be used does not mention the monitoring protocol to be used The plan does not prioritize sustainable tourism and does not provide a list of "best management practices" for recreation providers No score 0 0 The Plan does not define sustainable tourism and does not provide a list of "best management practices" that recreation providers should adhere to. The Sandy Island Oyster Bed MPA has a great potential for tourism aligned with conservation of the resources it boasts. Not having a plan that clearly defines sustainable tourism and provides a list of "best management practices" that recreation providers should adhere to, is a missed opportunity to guide these activities towards a more sustainable use of these resources. The plan does not describe management effectiveness evaluation as part of monitoring and evaluation No score 0 0 The Plan does not contain any management effectiveness evaluation tools The plan's monitoring program mentions the need for regular monitoring of locally important fisheries species, but does not appear to involve volunteer support from fishers or other groups Med 2 2 The Plan mentions monitoring "As part of the network monitoring efforts, in 2014 a long term coral reef monitoring site marked with two steel rebar points was established in SIOBMPA, the location of which is shown in the map below (LTM = long -term monitoring site). Also shown are past AGRRA (Atlantic and Gulf Rapid Reef Assessment protocol) monitoring sites from 2005, and past RC (Reef Check protocol) sites from 2011 -2012. A copy 219 of the Reef Check report is also included in this section." which includes a commercially important fisheries component (p 47); however, a distinction between fisheries independent monitoring (AGRRA), and fisheries dependent monitoring (landings) needs to be pointed out. In the case of this criteria, the emphasis is on fisheries dependent monitoring, and thus, does not acquire the complete "2" points under a "medium ranking", and neither does it mention the involvement of volunteers. Possible Subtotal Subtotal Score Subtotal Percentage 21 9 42.86% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence Maps, tables and graphics are used consistently throughout all components of the plan, and clearly illustrate key features of the MPA/MMA or other useful information Hi 3 3 Maps, tables and graphics are used consistently throughout all components of the Plan, and clearly illustrate key features of the MPA or other useful information The plan meets seven to nine of the following criteria: clear and accessible; concise and comprehensive; accurate and objective; systematic and logical; acceptable and motivating; precise Med 2 2 The Plan meets the following criteria: clear; concise; comprehensive; accurate; objective; logical; acceptable; practical; and focused. But needs some revision to make it more accessible, systematic, motivating, more precise 220 and practical; focused and effective and more effective, which would allow it to reach a "high" score. Possible Subtotal Subtotal Score Subtotal Percentage 6 5 83.33% Sandy Island Oyster Bed Marine Protected Area Management Plan Total Possible 108 Total Score 58 Percentage Score 53.70% 221 Three Bays National Park Management Plan Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence Only vision or mission are clearly defined, the other category is somewhat defined Med 2 2 Vision: "The ecologically rich and fragile marine, coastal, and terrestrial resources in the PA3B, most notably the complex of coral reefs, seagrass beds, mangrove forests, aquatic habitats and tropical dry forests, with their associated archaeological and historic features, are assured their long￾term protection and the maintenance of their biological diversity in perpetuity, while, in this spirit, providing for the sustainable use of select resources in support of local communities." (p xi). Mission is not defined. Clear and overarching management goals have been defined Hi 3 3 Clear and overarching management goals have been defined for each of the 9 conservation programs: "1) Biodiversity Conservation Program; 2) Sustainable Resource Use Program; 3) Environmental Education and Awareness Program; 4) Cultural Resource Conservation Program; 5) Ecotourism Program; 6) Community Management Program; 7) Governance, Co-management and Enforcement Program ; 8) Infrastructure Program; 9) Monitoring Program" (p 75). For a complete list of these goals, see p 76- 102 222 Clear and concise management objectives across a range of management topics have been defined and are SMART (Specific, Measurable, Achievable, Relevant, and Timebound) Hi 3 3 SMART, clear and concise management objectives have been defined for each of the 9 conservation programs: "1) Biodiversity Conservation Program; 2) Sustainable Resource Use Program; 3) Environmental Education and Awareness Program; 4) Cultural Resource Conservation Program; 5) Ecotourism Program; 6) Community Management Program; 7) Governance, Co-management and Enforcement Program ; 8) Infrastructure Program; 9) Monitoring Program" (p 75). For a complete list of these objectives, see p 76-102 Possible Subtotal Subtotal Score Subtotal Percentage 9 8 88.89% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence Plan clearly demonstrates how site management is integrated within national development plans and institutional frameworks Hi 3 3 "The PA3B exists by virtue of the Presidential Decree creating The Protected Area of Managed Natural Resources of the Three Bays, adopted at the National Palace in Port-au￾Prince, Haiti on February 13, 2014 (see Appendix 1). The subsequent declaration by Order in the Official Journal of the Republic of Haiti was published on March 21, 2014. The ongoing management and 223 administration of the Protected Area falls under the jurisdictional authority of Haiti’s National Agency of Protected Areas (ANAP), itself created through the General Decree on Environmental Management in 2005. ANAP will exercise its authority directly and in collaboration with its local partners, authorities and municipal governments. A PA3BManagement Authority (PMA) will be created to provide a local governance and administrative structure for PA3B and will establish a warden service, enforce regulations and carry out monitoring and surveillance. A proposed governance structure is outlined in Section 7.0 of this plan. In time, the 1978 Fisheries Law may be revised to support new initiatives in the management of fisheries within PA3B." (p 13 -14) Plan clearly demonstrates that national laws which enable management of natural resources are integrated and help guide management actions Hi 3 3 The PA3B exists by virtue of the Presidential Decree creating The Protected Area of Managed Natural Resources of the Three Bays, adopted at the National Palace in Port -au - Prince, Haiti on February 13, 2014 (see Appendix 1). The subsequent declaration by Order in the Official Journal of the Republic of Haiti was published on March 21, 2014. The ongoing management and administration of the Protected Area 224 falls under the jurisdictional authority of Haiti’s National Agency of Protected Areas (ANAP), itself created through the General Decree on Environmental Management in 2005. ANAP will exercise its authority directly and in collaboration with its local partners, authorities and municipal governments. A PA3BManagement Authority (PMA) will be created to provide a local governance and administrative structure for PA3B and will establish a warden service, enforce regulations and carry out monitoring and surveillance. A proposed governance structure is outlined in Section 7.0 of this plan. In time, the 1978 Fisheries Law may be revised to support new initiatives in the management of fisheries within PA3B. (p 13 -14) Plan clearly demonstrates that international agreements are integrated and help guide management actions Hi 3 3 "The implementation of the management plan will be cognizant of the obligations that Haiti has accepted by virtue of being a signatory nation to international conventions or regional agreements, including the World Heritage Convention and the Convention on Biological Diversity. Of equal importance is awareness of the environment -oriented international and regional arrangements such as the Caribbean Challenge Initiative of which Haiti is a signatory member and 225 other regional arrangements that Haiti is not a signatory to, including the International Coral Reef Initiative (ICRI) (although FOPROBIM is a member), the Cartagena Convention, the Convention on Wetlands of International Importance especially as Waterfowl Habitat (RAMSAR) and the Convention on International Trade in Endangered Species of Wild Flora and Fauna (CITES)." (p 16) Plan clearly describes rules and regulations that govern resource use with the MPA/MMA boundaries Hi 3 3 The rules and regulations are clearly described in the zoning section, where it states: "Zoning is a classification tool that spatially recognizes these differences in management focus and objectives, and guides the application of appropriate management strategies. Within a geographic area identified by a zone, the management emphasis will be applied equally where the levels of resource protection are consistent throughout the zone and similar levels and types of uses are allowed or not allowed as the case may be. In this way, zoning is primarily intended to achieve an appropriate balance between the conservation and protection of natural resources and the degree and types of human use." (p 65). All the pertinent rules and regulations can be found on p 69 -72). 226 Possible Subtotal Subtotal Score Subtotal Percentage 12 12 100.00% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence The Plan includes a robust analysis of budgetary requirements, and has identified, described and prioritized these requirements to efficiently conduct all listed management activities Hi 3 3 The plan presents a robust budget per conservation program. See p 105-113. A strategic planning process that identified limitations and opportunities for management was conducted, but did not inform the plan, and is not carried out at least every two years Med 2 2 The plan itself does not have a section where it describes that a strategic planning process was carried out, that would identify limitations and opportunities. However, the plan is very thorough and complete, and does identify opportunities, especially in the ecotourism strategy (p 60-62) The context in which the plan must operate (local, political, environmental, relevant laws) has been clearly and completely defined (by addressing all 4 areas of context) Hi 3 3 The context in which the Plan must operate: local (p 36), relevant laws and policital: p 6-15; environmental: p 17- 32 has been clearly and completely defined. A mechanism for periodic reporting has not been created No score 0 0 The plan does not mention a mechanism for reporting. The plan has clearly identified, described and prioritized administrative roles and procedures Hi 3 3 The cooperative management of protected areas is becoming common￾place throughout the Caribbean, and that will likely be no less the case in Haiti. Cooperative management, also 227 commonly referred to as co - management, is essentially the sharing of responsibilities and, in some cases the authorities, among government agencies, stakeholders, local governments and other partners for the management of protected area resources (McConney et al., 2003). The Presidential Decree clearly places primary management responsibility with the Department of Environment, and, by extension ANAP, and recognizes the responsibilities of the Ministry of the Interior and Territorial Communities and the departments of Public Works, Transport and Communications; Economy and Finance; Agriculture; Natural Resources and Rural Development; and Energy. There is also a dedicated commitment by the Government of Haiti to cooperate with a wide range of stakeholders, local governments and other partners in the implementation of this management plan. (p 103) The Plan has a clearly defined structure for and describes the decision -making process Hi 3 3 "The cooperative management of protected areas is becoming common - place throughout the Caribbean, and that will likely be no less the case in Haiti. Cooperative management, also commonly referred to as co - management, is essentially the sharing of responsibilities and, in some cases 228 the authorities, among government agencies, stakeholders, local governments and other partners for the management of protected area resources (McConney et al., 2003). The Presidential Decree clearly places primary management responsibility with the Department of Environment, and, by extension ANAP, and recognizes the responsibilities of the Ministry of the Interior and Territorial Communities and the departments of Public Works, Transport and Communications; Economy and Finance; Agriculture; Natural Resources and Rural Development; and Energy. There is also a dedicated commitment by the Government of Haiti to cooperate with a wide range of stakeholders, local governments and other partners in the implementation of this management plan." (p 103) The plan demonstrates that the management authority has identified a diverse potential income portfolio, and is receiving funding from 1 -2 sources Low 1 1 The plan states that ecotourism activities can be used as sources of income for the surrounding communitties and help become alternative livelihood sources (p 60 - 62), but does not state if the management authroity is receiving funding from any sources. The plan identifies training as a high priority, states training needs for the staff or resource users to Hi 3 3 Training is a high priority in the Management Plan; the plan states training needs for the staff or resource 229 carry out their roles and responsibilities, and includes a calendar of training sessions that are needed to ensure staff knowledge and skills are up - t o - date users to carry out their roles and responsibilities, and these have been defined for each of the 9 conservation programs: "1) Biodiversity Conservation Program; 2) Sustainable Resource Use Program; 3) Environmental Education and Awareness Program; 4) Cultural Resource Conservation Program; 5) Ecotourism Program; 6) Community Management Program; 7) Governance, Co -management and Enforcement Program ; 8) Infrastructure Program; 9) Monitoring Program" (p 75). For a complete list of these goals, see p 76 - 102 The plan includes a comprehensive threats analysis that is used to prioritize and direct management actions Hi 3 3 "5.0 Threats and Issues Analysis with Key Findings. Notwithstanding the rich resource values in PA3B, the area is beset with numerous threats that have, and unless addressed, will continue to impact upon its ecological integrity. A Threat Assessment was undertaken for the PA3B in 2015 by TNC, the purpose of which is to inform the preparation of this management plan and to guide future management and monitoring (TNC, 2015). The assessment recognizes that climate change is an overarching threat to PA3B, and indeed all of the specific and local threats assessed and reviewed below must all be understood in the context of the 230 risks associated with climate change. Section 5.1 below provides a prognosis for the likely impacts of climate change and their implications for community growth and development within PA3B. Section 5.2 then addresses each specific threat directly." (p 41 -59) The plan includes a clearly defined zoning plan that covers the entire MPA/MMA and there are differentiated activities allowed in the different zones Hi 3 3 "Zoning is a classification tool that spatially recognizes these differences in management focus and objectives, and guides the application of appropriate management strategies. Within a geographic area identified by a zone, the management emphasis will be applied equally where the levels of resource protection are consistent throughout the zone and similar levels and types of uses are allowed or not allowed as the case may be. In this way, zoning is primarily intended to achieve an appropriate balance between the conservation and protection of natural resources and the degree and types of human use." (p 65). All the pertinent rules and regulations can be found on p 69 -72). The plan includes a clearly defined enforcement plan Hi 3 3 There is an enforcement plan clearly defined, and its goal is: "To ensure effective governance, co -management, administration and law enforcement for PA3B through the development of a protected area management authority, thorough laws and regulations, the 231 placement of sufficient planning, management and law enforcement staff and collaboration with government agencies, local governments and communities." (p 97). The complete strategic actions of this program can be found in p 97 -99. The plan includes a clearly defined infrastructure plan Hi 3 3 There is an infrastructure plan, and its goal is: "To strengthen the PMA through the development of the necessary infrastructure to facilitate effective protected area management." (p 99). And all its activities can be found on p 99 -100. The plan has only vaguely defined MPA/MMA carrying capacity Low 1 1 The plan does not a section dedicated to carrying capacity, but it does express the need to develop one: "Integrate ecotourism objectives into biodiversity conservation initiatives in PA3B where possible, given constraints with carrying capacity and resource sensitivities to disturbance." (p 110) The plan clearly highlights how resiliency is incorporated into management actions that aim to avoid or minimize climate related impacts, through either adaptation or mitigation actions Hi 3 3 "As an overarching concern, the plan addresses the need to prepare for what now appears to be the inevitable impacts of climate change. Within that context, a number of threats and issues are described and ‘key findings’ identified that, in turn, form the basis of both the Vision and the Zoning Plan, and drive the development of nine management programs and their goals, objectives, and strategic actions." (p xi) 232 "The Community Development Zone recognizes those communities within PA3B’s boundaries and the need for their ongoing management, especially given the possible impacts of climate change and sea level rise." (p xii). "Recent climate change models for Haiti predict increases of 0.9° C by 2020 and 1.8°C by 2050. Other climate models for the greater region (including PA3B) for the period 2046 -2065 predict maximum and minimum yearly average temperatures to increase 1.5 to 2°C" (p 18). "Threat Assessment was undertaken for the PA3B in 2015 by TNC, the purpose of which is to inform the preparation of this management plan and to guide future management and monitoring (TNC, 2015). The assessment recognizes that climate change is an overarching threat to PA3B, and indeed all of the specific and local threats assessed and reviewed below must all be understood in the context of the risks associated with climate change. Section 5.1 below provides a prognosis for the likely impacts of climate change and their implications for community growth and development within PA3B. Section 5.2 then addresses each specific threat directly." (p 41 and see p 41 -47 for how each threat is addressed) 233 Possible Subtotal Subtotal Score Subtotal Percentage 42 36 85.71% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence The plan demonstrates that community participation has occurred at all three stages of protected area development: design, implementation and evaluation Hi 3 3 Plan highlights the importance of stakeholder engagement, consensus building and conflict management, and states: " The plan will be undertaken in phases based on continued consultations with communities and stakeholders and the availability of human, technical and financial resources." (p xiii). " The purpose of the five-year review is to re-visit the management plan with an in-depth consultation program to review progress on the plan’s implementation and revise direction as necessary without undertaking a full management planning process." (p 3). " The plan should be undertaken in phases based on continued consultations with communities, stakeholders and government officials and on the availability of human, technical and financial resources" (p 3). " In early 2016, arrangements were made with Global Parks to initiate the management planning process beginning with a desktop review of these and other background materials 234 and an initial field visit during the week of April 4 -9 to understand context and the needs of local communities and stakeholders. Information gaps were identified and information gathered, to the extent possible, to fill these gaps. An interim report and background materials were prepared for stakeholder consultations and meetings with the Steering Committee held in May and early June 2016." (p 4). The plan describes how an Advisory Committee was established, was a key actor during the establishment of the MPA/MMA, and continues to inform the planning process Hi 3 3 " In 2012 a process for developing a management plan for the area which would become the PA3B began with the formation of a Steering Committee (Comité de Suivi) in Port -au -Prince among agencies of the Government of Haiti and the main donors involved in the funding and implementation of PA3B. This committee met regularly and provided senior level guidance for the management planning process and coordination of preliminary initiatives that would lead to the management of the PA3B. This committee is composed of representatives of key entities, chaired by the National Agency of Protected Areas (ANAP), and includes the United States Agency for International Development (USAID), the Inter -American Development Bank (IDB), the Ministry of Environment (MOE), the United Nations 235 Development Programme (UNDP), Marine and Navigational Services of Haiti (SEMANAH), the Technical Execution Unit (UTE) of the Ministry of Economy and Finance, and Haiti’s Inter -ministerial Committee for Territorial Planning (CIAT). Other institutions and government departments occasionally participate depending on the agenda to be addressed." (p 3) The plan clearly demonstrates how early or current planning processes attempt to prioritize and achieve a balance of respecting traditional resource use and cultural protocols with conservation goals and sustainable resource use Hi 3 3 The Management Plan highlights the importance of social issues, and has detailed actions described in the Cultural Resource Conservation Program, that has the following goal: "To research, protect and present the rich cultural heritage of PA3B and the north coast of Haiti." (p 88). All the startegic actions for the program can be found in p 88 -90. " In 2014 and 2015, with the inception of the USAID - sponsored CMBP and the IDB - sponsored ecological baseline assessment ANAP, in collaboration with TNC, FOPROBIM and members of the Steering Committee held consultations with local stakeholders in an attempt to create a local Stakeholder’s Committee for the park (Comite d’Appui)." (p 3) The plan highlights the importance of stakeholder engagement, Hi 3 3 Plan highlights the importance of stakeholder engagement, consensus 236 consensus building and conflict management, and describes processes or mechanisms which staff and stakeholders have used in the past or may apply in the future building and conflict management, and states: " The plan will be undertaken in phases based on continued consultations with communities and stakeholders and the availability of human, technical and financial resources." (p xiii). " The purpose of the five -year review is to re -visit the management plan with an in -depth consultation program to review progress on the plan’s implementation and revise direction as necessary without undertaking a full management planning process." (p 3). " The plan should be undertaken in phases based on continued consultations with communities, stakeholders and government officials and on the availability of human, technical and financial resources" (p 3). " In early 2016, arrangements were made with Global Parks to initiate the management planning process beginning with a desktop review of these and other background materials and an initial field visit during the week of April 4 -9 to understand context and the needs of local communities and stakeholders. Information gaps were identified and information gathered, to the extent possible, to fill these gaps. An interim report and background materials were prepared for stakeholder 237 consultations and meetings with the Steering Committee held in May and early June 2016." (p 4). The plan highlights the importance of education and describes a multifaceted program, mentioning at least four of these five methods: visual, active, fieldwork, radio, entertainment Hi 3 3 The Management Plan highlights the importance of education, and has detailed actions described in the Environmental Education and Awareness Program, which has the following goal: "To increase the level of awareness and knowledge of the residents and stakeholders of local communities of the ecological, social and economic values of PA3B, and the role of protected area management in ensuring sustainability and stable livelihoods." (p86). All the startegic actions for the program can be found in p 86-87. The plan includes a somewhat defined outreach and communications plan Med 2 2 The plan does not have a Communications plan, per se, but the Environmental Education and Awareness Program does list some activities that would normally be listed under a communications plan. See p 86-87 for a list of these activities. Possible Subtotal Subtotal Score Subtotal Percentage 18 17 94.44% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence The plan clearly demonstrates that a scientifically-sound baseline data Hi 3 3 The Plan clearly demonstrates that a scientifically-sound baseline data set 238 set has been collected, analyzed and used to inform development of the MPA/MMA, and is used in active and adaptive management has been collected, analyzed and used to inform development of the MPA, and is used in active and adaptive management. See p 17 -35, which are the Physical Environment and Biological Resource Descriptions. The plan clearly defines biophysical monitoring needs, describing the protocols to be used and the periodicity of monitoring campaigns Hi 3 The plan defines biophysical monitoring needs, and mentions the monitoring protocols to be used and the periodicity of monitoring campaigns Med 2 2 The plan does have a Monitoring Program, with clearly defined activities (see p 100 -102), but does not state protocols to be used or periodicity of these. The plan somewhat defines socio - economic monitoring needs, and mentions the monitoring protocol to be used and the periodicity of these monitoring campaigns Med 2 2 The plan does not have a Socio - economic monitoring program per se, but the activities under the Community Management Program can be linked to these. See p 95 -96 for these. The plan clearly defines governance monitoring needs, describing the monitoring protocol to be used and the periodicity of monitoring Hi 3 3 The Plan clearly defines governance indicators within the "Governance, Co - management and Enforcement Program" , whose goal is: "To ensure effective governance, co -management, administration and law enforcement for PA3B through the development of a protected area management authority, thorough laws and regulations, the placement of sufficient planning, management and law enforcement staff and collaboration with government 239 agencies, local governments and communities." (p 97). The complete strategic actions of this program can be found in p 97-99. The plan clearly prioritizes sustainable tourism and provides a list of "best management practices" that recreation providers should adhere to. This list includes: six or more best practices for recreational scuba-diving services; six or more best practices for recreational snorkeling services; and six or more best practices for recreational boat operations Hi 3 3 The Plan clearly defines sustainable tourism through its Ecotourism Program, that has the followimg goal: To ensure PA3B leads the integration of biodiversity conservation into a tourism strategy for Haiti’s north-east coast, and embraces ecotourism, including adventure, cultural and social tourism, as a key pillar in its overall development and management strategy." (p 91). All the objectives and actions for this program can be found in p 91-94. The plan does not describe management effectiveness evaluation as part of monitoring and evaluation No score 0 0 The Plan does not contain any management effectiveness evaluation tools The plan does not mention a plan for monitoring locally important fisheries species No score 0 0 The Plan does not mention a plan for local fisheries monitoring Possible Subtotal Subtotal Score Subtotal Percentage 21 13 61.90% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence Maps, tables and graphics are used consistently throughout all components of the plan, and Hi 3 3 Maps, tables and graphics are used consistently throughout all components of the Plan, and clearly illustrate key 240 clearly illustrate key features of the MPA/MMA or other useful information features of the MPA or other useful information The plan meets at least 10 or more of the following criteria: clear and accessible; concise and comprehensive; accurate and objective; systematic and logical; acceptable and motivating; precise and practical; focused and effective Hi 3 3 The Plan meets at least 10 or more of the following criteria: clear and accessible; concise and comprehensive; accurate and objective; systematic and logical; acceptable and motivating; precise and practical; focused and effective Possible Subtotal Subtotal Score Subtotal Percentage 6 6 100.00% Sandy Island Oyster Bed Marine Protected Area Management Plan Total Possible 108 Total Score 92 Percentage Score 85.19% Pedro Cays and Surrounding Waters Management Plan Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence Vision and mission are clearly defined Hi 3 3 "Vision: A future where effective and collaborative management of Pedro Cays biodiversity and ecosystem health support sustainable livelihoods" "Mission: To provide guidance for the protection and restoration of the resources and biodiversity of Pedro Cays and ensure viable and sustainable livelihoods, particularly for the fishing community." (p 11) 241 Clear and overarching management goals have been defined Hi 3 3 The plan has 4 goals, each one pertaining to a management programme: conservation, monitoring and evaluation, sustainable fisheries, socio-economic sustainability and governance and management. (p 65) Management objectives have been defined across a range of management topics, but are not clear or concise and not SMART Med 2 2 The plan has 22 objectives, divided in each one of the goals. Only two objectives (16 and 21) are timebound in the text, the others include some timeframes in their strategy tables. (p 67-101) Possible Subtotal Subtotal Score Subtotal Percentage 9 8 88.89% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence Plan clearly demonstrates how site management is integrated within national development plans and institutional frameworks Hi 3 3 Plan makes reference to the recommendation of the National Biodiversity Strategy and Action Plan to declare Pedro Cays as protected area status. Appendix 3 further mentions the National Development Plan, Vision 2030 and provides a description of the relevant points that were taken into account for the Pedro Cays Management Plan. It also mentions the Draft National Fisheries Policy, the National Policy for the System of Protected Areas. (p 10, 107, 110) Plan clearly demonstrates that national laws which enable Hi 3 3 Appendix 3 provides a list and description of relevant national laws, 242 management of natural resources are integrated and help guide management actions while also linking the provisions to the management actions for Pedro Cays. (p 10, 108-112) Plan somewhat demonstrates that international agreements are integrated and help guide management actions Med 2 2 Plan makes limited reference to the CITES Convention and UNCLOS, in the description of the national laws. (p 108 and 112) Plan clearly describes rules and regulations that govern resource use with the MPA/MMA boundaries Hi 3 3 The zoning plan section mentions some rules and regulations that apply to the different areas (p 102-103). Appendix 3 which provides a list and description of relevant national laws also mentions its regulations. (p 108-112) Possible Subtotal Subtotal Score Subtotal Percentage 12 11 91.67% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence The Plan includes a somewhat robust analysis of budgetary requirements, and has only identified, but not described and prioritized these requirements to efficiently conduct all listed management activities Med 2 2 The plan includes a list of "priority projects" with estimated costs. However, there is no analysis of the general budgetary requirements. There is a mention of "a workplan based on the priorities and estimated timeline is attached along with a budget." But the attachment was not found. (p 92) A strategic planning process that identified limitations and opportunities for management was conducted, but did not inform the Low 1 1 There is a mention to an activity in August-November 2011 for "Identification of constraints, opportunities and threats" but its 243 plan nor has it been revisited since it was first carried out findinds are not included, and there is no evidence it was revisited for the 2015 updated version of the management plan. (p 119) The context in which the plan must operate (local, political, environmental, relevant laws) has been clearly and completely defined (by addressing all 4 areas of context) Hi 3 3 The local context is defined (p 37 -40), the political context (p 57 -61), the environmental is amply defined in p 28 -36 and 41 -46; and the relevant laws are listed in Appendix 3 (p 108 -112). A mechanism for periodic reporting has been created, but is not embedded in the plan Low 1 1 The introduction to the management provides for "regular meetings" and in the Governance and Management Programme it includes a recommendation to work "in collaboration with … relevant agencies through a mechanism including a committee which would report to a series of authorities" but does not establish a timeframe and the mechanism itself. (p 91) The plan has somewhat identified and described administrative roles and procedures, but has not prioritized these procedures Med 2 2 The plan contains a table to list the organisations and describe their mandates, objectives and responsabilities but has not described their procedures. (p 21 -22) The plan has a somewhat defined decision -making structure and the decision -making process is also somewhat described Med 2 2 The plan includes some recommendations for the decision - making process: "stakeholders will be consulted in decision -making and informed of all decisions" and "the best scientific information available will be used in making management 244 decisions" (p 64). The plan does not aim to create a new structure ("without necessarily creating a new structure") and somewhat describes procedures to accompany the decision -making process. (p 90 -92) The plan demonstrates that the management authority has identified a diverse potential income portfolio, and is receiving funding from several sources Hi 3 3 "with funding from the 'management and Conservation of Reef Biodiversity and Reef Fisheries Pilot Project'... the full project was funded by the UNDP, GEF, UNEP and CEP... the implementing agency was the TNC" (p ii) "The initial capital for development may be from a combination of funds from the Fisheries Improvement and Development Fund and bi - and/or multilateral funding sources e.g. European Union, Japan." (p 90) "with funding from the CLME project" (p 119) The plan identifies training as a high priority, states training needs for the staff or resource users to carry out their roles and responsibilities, and includes a calendar of training sessions that are needed to ensure staff knowledge and skills are up -to - date Hi 3 3 Objective 18 states: "To provide adequate, trained personnel" (p 96). Table 28, in for staffing needs, does not include training (p 97). Training is considered to be "a crosscutting strategy" (p 65). The sustainable fisheries management program includes "education and skills training system and facilities" (p 76). "...plans for sustainable use… should include… training" (p 84) 245 The plan includes a comprehensive threats analysis that is used to prioritize and direct management actions Hi 3 3 The plan describes the threats to biodiversity and natural resources, with figures of primary and secondary threats. It also includes a threats rating analysis table. (p 47 -57) The plan has a somewhat defined zoning plan that covers more than half of the entire MPA/MMA, and there are differentiated activities somewhat defined that are allowed in the different zones Med 2 2 It is unclear whether the zoning plan covers more than half of the protected area as it states that "this chapter has been limited due to inadequate information… however the following areas have been identified as Zones," and it then enlists zones and describe the activities allowed within them. (p 102 -103) The plan includes an enforcement plan that is only vaguely defined Low 1 1 The plan mentions the need for ongoing and permanent enforcement of several laws, rules and regulattions, but does not present an enforcement plan. The plan includes a vaguely defined infrastructure plan Low 1 1 The Plan does not have an infrastructure plan, however, it does mention the need to buoys to mark the boundaries of SFCA (p 102). The plan has somewhat defined MPA/MMA carrying capacity, but it is unclear if this informs management activities Med 2 2 "The socioeconomic study prepared by Rhiney et al 2013 includes assessment of carrying capacity… The assessment pointed out that…"The plan includes as objective 10" prepare a plan (including carrying capacity…" (p 83 -84) The plan clearly highlights how resiliency is incorporated into management actions that aim to avoid or minimize climate related Hi 3 3 Objective 22 is "To ensure that climate change and disaster management is considered in all relevant programmes and activities" and includes a summary 246 impacts, through either adaptation or mitigation actions for Climate Change as a threat (p 54 & 101). Possible Subtotal Subtotal Score Subtotal Percentage 42 29 69.05% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence The plan demonstrates that community participation has occurred at two stages of protected area development: design, implementation or evaluation Med 2 2 The plan "...was prepared using a participary approach" (p 10), "the foundation of the management plan was gleaned from field research and local community" (p 14). "Fishers should be involved in governance and management" (p 59). There is evidence that community participated in the "analysis and evaluation of data" (p 119), but Table 10 "monitoring and evaluation" does not include the community. (p 71) The plan describes how an Advisory Committee was established, was a key actor during the establishment of the MPA/MMA, and continues to inform the planning process Hi 3 3 "this team acted as the advisory committee during the preparation of the management plan" (p ii) The plan clearly demonstrates how early or current planning processes attempt to prioritize and achieve a balance of respecting traditional resource use and cultural protocols with conservation goals and sustainable resource use Hi 3 3 The plan mentions cultural heritage as early as page viii and 10. Also, Objective 5 of the plan is "to facilitate preservation of cultural heritage by assisting the Jamaica National Heritage Trust" (p 75). 247 The plan somewhat mentions the importance of stakeholder engagement, consensus building and conflict management, but does not describe processes or mechanisms which could be used Low 1 1 "key stakeholders who will be critical to the partnership arrangements, and between whom there will need be commitments made regarding roles and responsibilities" (21) Plan also includes a section about stakeholders in which it describes the current problems but does not define mechanisms for consensus building and conflict manager The plan somewhat highlights the importance of education and describes a multifaceted program, mentioning at least three of these five methods: visual, active, fieldwork, radio, entertainment Med 2 2 Education is considered a "cross￾cutting strategy" by the plan (p 65) It is considered in several sections but it never describes a multifaceted program. The plan does not have an outreach and communications plan No score 0 0 The management plan does not include a communications plan Possible Subtotal Subtotal Score Subtotal Percentage 18 11 61.11% Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence The plan clearly demonstrates that a scientifically-sound baseline data set has been collected, analyzed and used to inform development of the MPA/MMA, and is used in active and adaptive management Hi 3 3 The plan references to Edwards (2001), Webber (2002), Zans (1958), Aiken (2001), Hay (2006), among others. (p 32-58) The plan defines biophysical monitoring needs, and mentions the monitoring protocols to be Med 2 2 Objective 4 is "to conduct monitoring and evaluation activities that will guide adaptative management and aid assessment of management 248 used and the periodicity of monitoring campaigns effectiveness" and table 10 includes periodicity of monitoring campaigns (p 71). It also mentions AGRRA as a monitoring protocol. The plan loosely defines socio￾economic needs, but does not mention monitoring protocols to be used No score 0 0 The Plan loosely defines socio￾economic indicators that are to be monitored, but does not mention monitoring protocol(s) to be used The plan loosely defines governance needs, but does not mention monitoring protocols to be used No score 0 0 The Plan loosely defines governance indicators that are to be monitored, but does not mention the monitoring protocol to be used The plan does not prioritize sustainable tourism and does not provide a list of "best management practices" for recreation providers No score 0 0 The Plan does not define sustainable tourism and does not provide a list of "best management practices" that recreation providers should adhere to, although it does mention a budget for this: " Sustainable Use and Ecotourism Plans estimated at JA$15 million and JA$4 million each" (p 92). Management effectiveness evaluation not described as part of monitoring and evaluation No score 0 0 The Plan does not contain any management effectiveness evaluation tools Monitoring program includes regular monitoring of locally important fish species, involves volunteers in the monitoring from fishers and other groups Hi 3 3 The plan mentions AGRRA and Fish per catch unit effort monitoring. (p 76- 77) Possible Subtotal Subtotal Score Subtotal Percentage 21 8 38.10% 249 Evaluation Criteria Ranking Scoring Criteria Plan Status Notes/Supporting Evidence Maps, tables, graphics are used consistently throughout all components of the plan, and clearly illustrate key features of the MPA or other useful information Hi 3 3 Maps, tables and graphics are used all over the plan, and help understand better what is in the text. Meets seven to nine criteria: clear and accessible; concise and comprehensive; accurate and objective; systematic and logical; acceptable and motivating; precise a practical; focused and effective Med 2 2 The plan is not concise, but it meets the rest of the criteria. Possible Subtotal Subtotal Score Subtotal Percentage 6 5 83.33% The Pedro Cays and Surrounding Waters Management Plan Total Possible 108 Total Score 72 Percentage Score 66.67% 250 Annex VII – TNC Statement of Differences Comments to the CMBP Mid-term Evaluation Report Submitted by The Nature Conservancy April 16, 2018 I. Executive Summary Expected Result 4: Effective management and governance of MPAs Performance indicators (page 2) “Indicator 3 (new laws, policies and plans) has produced mixed results; overachieved in Jamaica and fully achieved in Haiti, but half way in Grenadine Bank, and 0% of target completion in Samana Bay.” The statement about the result in Samana Bay is incorrect. One policy instrument was adopted by the Dominican government with the declaration of the six NTRs (no￾take reserves). This was included in our Y3 annual report, both in the body of the report and the MEL plan. AMI note: The information is based on CMBP’s Annex A_CMBP Year 3 Monitoring and Evaluation Report 30Sep2017. There is 0 for the FY 2017 in the table: CMBP Indicator targets 3 progress as of September 30, 2017. Minor modifications made to the report text to indicate that although the monitoring reports indicates a value of 0 there was a policy instrument adopted by the Dominican government with the declaration of the six no-take reserves, representing half of the target for indicator 3 in DR. MPA management plans and action plans (page 2) “The action plans for coral and marine mammal conservation in Samana Bay are of sub￾standard quality and require significant improvements.” This statement is misleading. CMBP did not understand these action plans would be assessed by the same criteria as the management plans. Action plans have a different purpose and structure than management plans, and we believe some of the criteria used to assess the management plans are not relevant for action plans, such as the legal and institutional framework, management planning and activities, stakeholder analysis and engagement and monitoring and evaluation and adaptive management. These are the criteria in which the action plans were scored low. Note the Estero Hondo Marine Mammal Sanctuary is not located in the Samana seascape. Also, the evaluation team did not review the final action plans, as they were in the process of being translated and copy edited when the evaluation began. The final documents are now uploaded to the BOX folder set up for the evaluation team. 251 AMI note: A note was added to the executive summary to alert the reader about the importance of reading the evaluation limitations section of the report, regarding the interpretation of management plans evaluations results for Action Plans developed by CMBP. MPA governance and management, including patrolling and outreach (page 2) This section is incomplete, as there is no mention of patrolling, outreach and education, monitoring and general management activities conducted at Three Bays National Park in Haiti, which were a major CMBP contribution. AMI note: Minor modifications made to the report text based on this clarification. Management Effectiveness tracking scores (METT) (page 3) “Slight improvements seen in METT scores in both Grenadine Bank MPAs and Three Bays Park in Haiti” This statement is not correct. The METT score for Three Bays National Park increased eleven percentage points over the previous year, primarily due to the management and zoning plans and the engagement of trained wardens. The METT scores for both Grenadine Bank MPAs (SIOBMPA and TCMP) increased over ten percentage points. AMI note: Minor modifications made to the report text based on this clarification. “Most organizations are applying the incorrect version of the METT, the terrestrial, and not marine, version. Only MPA authorities in the Grenadine Bank have consistently applied the marine METT.” This statement is incorrect and misleading. CMBP uses the WWF 2007 version of the METT for the seascapes in Haiti, DR and Jamaica. In the GB, a version of this WWF 2007 METT, modified for the ECMMAN project, is used. There is no official “marine METT” tool. While the WWF METT tool has a few questions that are specific to the management of terrestrial protected areas, most questions are quite general and applicable to marine protected area management. CMBP assessments do not include components of the METT tool that are only applicable to terrestrial protected areas. AMI note: Minor modifications made to the report text based on this clarification. Please note the World Bank developed a marine METT in 2004. This is the document that AMI refers to as the marine METT. The document can be found online here. Expected Result 3: Sustainable fisheries and sustainable livelihoods for fishers Fisheries sector action plans (pages 3 and 87) 252 This section is incomplete, as there is no mention of a Fisheries Sector Action Plan completed for Three Bays National Park in Haiti. AMI note: Minor modifications made to the report text based on this clarification. Capacity of fisher associations (page 4) “Community leaders on some islands across the Grenadine Bank report that fisher associations are experiencing difficulties getting organized, and thus most associations are in an early stage of development at best. Building fisher associations into truly organized and active groups is challenging, particularly on Union Island and Mayreau.” The general conclusion about the status of fisher associations is accurate. However, the second sentence is a bit misleading. While CMBP’s livelihood activities in the Grenadine Bank may contribute to strengthened organizations, the main objective is to develop sustainable and supplementary livelihoods for cooperative members. In Mayreau we are assisting the Mayreau Explorers, a community cooperative that includes fishers. In our view, Mayreau Explorers is an active and relatively well￾organized group, though they could certainly benefit from capacity building support. In addition, the summary does not mention CMBP activities in Haiti, where strengthening fisher organizations is a major focus of our work. AMI note: Minor modifications made to the report text based on this clarification. Capacity of fisher associations (page 4) This section is incomplete, as there is no mention of capacity building efforts or improvement at Three Bays National Park in Haiti. AMI note: Minor modifications made to the report text based on this clarification. Evaluation of CMBP sub-grantee organization capacity (page five) “Evaluation of CMBP sub-grantee organization capability found that CEBSE is at a Moderate Level of organizational capacity, and SUSGREN, C-CAM and FOPROBIM at a Strong Level of organizational capacity. The Evaluation team concludes that all sub-grantees organizations have an adequate level of management, financial and programmatic capacity to support CMBP’s efforts to achieve expected results. Moreover, sub-grantees have the organizational capacity needed to sustain activities into the future.” We will look forward to more information about the methodology used to reach this conclusion. As noted above, most TNC staff, including all the Seascape Supervisors and the Grants Specialist, believe none of the sub-grantees have sufficient organizational capacity to sustain their activities into the future. Over the past two years, TNC has conducted in-depth, comprehensive sub-recipient monitoring activities for all four NGO sub-award partners. This has included site visits and 253 internal audits to assess their organizational capacity. Our findings show that fundraising and financial accounting and management needed significant strengthening across all four partners. Through our extensive work with partners on monitoring and evaluation (e.g. METT), we have also concluded that this area represents another important need in terms of capacity building. AMI note: Minor modifications made to the report text based on this clarification. II. Expected Result 4: Effective Management and Governance of MPAs Progress towards performance indicators and strengthening collaborative management (page 22) “CMBP has chosen not to include, for example, the resolution that creates six no-take zones in Samana Bay, enacted by the Dominican Republic Ministry of Environment.” This statement is incorrect. As described above, in our Year 3 annual report, both in the body of the report and the MEL plan, we included the declaration of the six NTRs (no-take reserves) as an achievement of CMBP. AMI note: The information is based on CMBP’s Annex A_CMBP Year 3 Monitoring and Evaluation Report 30Sep2017. There is 0 for the FY 2017 in the table: CMBP Indicator targets 3 (Number of new laws, policies, strategies, plans, or regulations addressing biodiversity conservation officially proposed or adopted as a result of USG assistance) progress as of September 30, 2017. Minor modifications made to the report text to indicate that although the monitoring reports indicates a value of 0 there was a policy instrument adopted by the Dominican government with the declaration of the six no￾take reserves, representing half of the target for indicator 3 in DR. Tracking protected area management effectiveness (page 23) “Practitioners applied the terrestrial METT version scorecard was used. The total possible score for each site is 102 in the years 2015 and 2016. With 30 questions that can score 0-3 points, plus 12 “additional point” questions that can score 0-1 points, the maximum total possible score is 102. It appears there is a mistake in the 2017 application of the tool, where scores incorrectly indicate a total possible score of 145*.” There was indeed a mistake in the METT scores for the DR. The total score should have been out of 102 for all MMAs assessed. We appreciate the correction. Observations and Recommendations (Table 7, page 25) “Jurisdiction and competencies between MARENA and CODOPESCA should be clarified to ensure appropriate management of fisheries resources in Samana Bay”. 254 This statement is incorrect. CMBP conducted legal research to identify possible overlapping competencies between MARENA and CODOPESCA. The research did not find any overlapping competencies. This report can be found in the file: GreEnergy.Producto A V09122016 final.docx in the following URL: https://tnc.app.box.com/file/99963630381 AMI note: The lack of clarity on the agency responsible for the management of the no￾take zone is a perception of a CODOPESCA staff interviewed. Even though the decree indicates MARENA as the responsible entity to manage the Fisheries Exclusion Zones, an interviewee from CODOPESCA expressed that they should be directly involved in its management. Jurisdictions should be made clear to all stakeholders, including government agencies. Minor modifications were made to the report text. MPA management (page 25) “Even though the areas declared constitute a shrimp fisheries management intervention, the participation of the CODOPESCA was limited, and the final document lacks guidance in terms of fishing gears and navigation regulations in the areas.” This statement is incorrect. CODOPESCA actively participated, along with TNC and MARENA, from the earliest stage in the drafting and decision-making process for the creation of these no-take zones. The resolution specifically states that capture or extraction of organisms for fishing purposes is prohibited. Capture and extraction activities are allowed only for research purposes. Regarding navigation, the regulation clearly allows passage of vessels and people within the boundaries of the established no-take-zone areas. AMI note: Minor modifications were made to the report text. “The limited participation of the CODOPESCA was due in part, to a conflict in jurisdiction with the Ministry of Environment, and apparently, the management plans for these no-take zones will be developed by MARENA and not CODOPESCA.” This statement is incorrect. There is no conflict in jurisdiction. The Ministry of Environment is the competent authority for the no-take zones and responsible for development of the management plans. CODOPESCA regulates the fishing sector. The two agencies were co-authors of the resolution, verifying the hatchery and spawning potential of the areas and assessing the size for each area. AMI note: The decree is signed by the ministry of MARENA and not CODOPESCA. “However, some indicated that the participation of the CODOPESCA was somehow limited in the final technical process, and that CODOPESCA and MARENA have not reached an agreement when it comes to the responsibility for managing the no-take zones.” 255 This statement is incorrect. CMBP facilitated a series of meetings with technical staff of both CODOPESCA and MARENA to reach consensus regarding the number of no￾take-zones, their size and other aspects of the management plans. Draft documents were sent to the legal departments of both institutions for review. AMI note: Minor modifications were made to the report text. Enabling factors and conditions (page 27) “Several actions by CMBP and have resulted in positive outcomes for marine biodiversity in the Samana Bay area. These include the signing of an agreement between Fondo Marena and the Caribbean Challenge Initiative, which resulted in increased funding.” This statement is incorrect. Fondo MARENA signed an agreement with the Caribbean Biodiversity Fund (CBF), not the Caribbean Challenge Initiative (CCI). Positive outcomes are expected only in the Samana Bay area but also nation-wide. AMI note: Minor modifications were made to the report text. Management infrastructure (page 27) “In terms of sustainability beyond CMBP, the cooperative that co-manages Cayo La Farola.” This statement is incorrect, as the cooperative has a concession agreement to perform tourism operations. This is not a co-management arrangement. AMI note: Minor modifications were made to the report text. Summary Analysis of Findings (page 28) “Improvements in management effectiveness, though slight, at both TCMP and SIOBMPA over three years of applying the Marine METT.” The statement is misleading because the METT scores at both sites more than doubled in a two-year period. At SIOBMPA, the score increased from 38 in 2015 to 84 in 2017 (a 2.2-fold increase) and at TCMP, the score increased from 47 in 2015 to 102 in 2017 (also a 2.2-fold increase). AMI note: Minor modifications made to the report text based on this clarification. Key challenges that have limited progress (page 28) The list of key challenges that have limited progress toward program targets should include Caribsave’s withdrawal from the project. This affected continuity of project interventions; for example, delaying the hiring of a manager for SIOBMPA. AMI note: Added to the list of challenges a note saying that “TNC identified the withdrawal of Caribsave from the project as a challenge”. 256 “Unfortunately, SIOBMPA went without a manager for nearly two years, even though TNC did all it could to advertise and recruit the position. The first two selected persons turned down the job; a third person accepted the job in late 2017.” (page 31) It would be more accurate to state that “Unfortunately, SIOBMPA went without a manager for nearly two years, even though TNC and partners did all they could to advertise and recruit the position. Caribsave initially recruited a manager, but then withdrew from the project. Then TNC selected a candidate who later turned down the job. Finally, SUSGREN recruited the present manager, who accepted the job in late 2017.” AMI note: Minor modifications made to the report text based on this clarification. Tracking and improving MPA management effectiveness “Supported by TNC facilitation, SIOBMPA and TCMP staff slightly modified the marine version of their respective METT tracking tool to fit the local context” (page 33) The statement is misleading. The ECMMAN program modified the WWF METT version, not a marine version. AMI note: Minor modifications made to the report text based on this clarification. “The newly established support network, led by TNC and SUSGREN with TCMP and SIOBMPA staff, fishers, key government agencies and the wider community, appears as the most positive and observable manifestation of collaborative governance. Moreover, a network of MPA practitioners from both countries now convenes annually to discuss challenges, improve management frameworks and plan for the future.” (page 34) This statement appears to be referring to the Grenadines Network of Marine Protected Areas, which is led by SUSGREN, not TNC and SUSGREN. A more accurate statement would be “The Grenadines Network of Marine Protected Areas, a support network led by SUSGREN with personnel from TCMP, SIOBMPA and other MPAs, key government agencies and other stakeholders, appears as the most positive and observable manifestation of collaborative governance. Moreover, this network of MPA practitioners from both countries convenes annually to discuss challenges, improve management frameworks and plan for the future. CMBP has supported and strengthened this Network, for example by financing two annual Network meetings and by enabling SUSGREN to hire a marine protected areas coordinator who coordinated collaboration between member MPAs year-round, rather than only during annual meetings.” AMI note: Minor modifications made to the report text based on this clarification. “Management activities and staff time may be too narrowly focused on user fee collection.” 257 “Create specific agenda item at the next MPA network workshop to address management priorities beyond user fees.” (Table 13, page 34) These observations refer to TCMP management activities and staff time, rather than CMBP activities and staff time. Also, SUSGREN questions whether the recommended solution will solve the problem. The focus of the MPA staff on fee collection is due to the priorities of the MPA senior management and MPA board for the management of the MPA. To influence their priorities would require deeper engagement with these stakeholders, such as through a participatory MPA management planning process. AMI note: Minor modifications made to the report text based on this clarification. Observations and recommendations (Table 13, page 34) “Little evidence seen of early launch of protect parrotfish campaign. TNC parrotfish campaign needs more support and could learn from Belize”. Given the limited resources, the parrotfish campaign was designed to target a broad audience using only social media. There were no plans to have campaign materials (e.g., signs and banners) present in each seascape. However, with additional funding and time, it is possible to increase and expand efforts across all CMBP seascapes and other Caribbean countries where TNC works. These efforts could include use of social media, on-the-ground campaign materials and banners installed in selected areas. AMI note: No comment. Evaluation of Three Bays National Park management plan (page 37) “The development of the management plan, was guided by the sub-grantee, FOPROBIM, with support from Global Parks/International Caucus Foundation.” This statement is incomplete. It should acknowledge TNC’s key role in guiding development of the management plan (or simply credit the CMBP team). AMI note: Minor modifications made to the report text based on this clarification. Summary Analysis of Findings (pages 37 and 39) “METT scores for the Three Bays National Park increased 7 points from 2016 to 2017.” (page 37) “The Three Bays National Park obtained an increase of 8 points in its METT score, thanks to the completion of the management and zoning plans.” (page 39) There is a discrepancy between the two statements above. In both cases, the correct increase in the METT score is eight points. Also, there is no mention of the FoProBiM wardens being hired as a major contributor to the increase in METT score from 2016 258 to 2017 for Three Bays National Park. Though wardens are not yet fully trained and functional, they are now actively patrolling. AMI note: Minor modifications made to the report text based on this clarification. Tracking management effectiveness (page 39) “Since the program has been implemented for over 3 years now, it is time to progressively transfer the leadership and responsibility for the METT process to ANAP’s local and national authorities. CMBP should consider building ANAP’s capacity to monitor management effectiveness. With this capacity ANAP, as Haiti’s national agency for protected areas, can lead and facilitate METT processes in other areas in Haiti where FOPROBIM does not work.” We agree ANAP’s capacity to monitor management effectiveness should be increased, and the agency should take more responsibility for the METT process. However, to avoid bias, it is a good practice for a third party to lead the METT assessment process. AMI note: we agree. Management infrastructure (page 42) “According to TNC and the sub-grantee, CMBP has already provided diving equipment for the monitoring of marine resources in the Three Bays National Park, such as tanks and compressor.” The statement is correct. However, it is not yet clear if the staff has the capacity to use the equipment and monitor marine resources. CMPB will continue to follow up on this. AMI note: No comment. “To ensure that the infrastructure and equipment will continue to be used beyond the life of CMBP, TNC made sure to select FOPROBIM, a group with a longstanding reputation. Numerous training sessions were provided by FOPROBIM to fisher associations that would help ensure management infrastructure sustainability. TNC also hired the firm Integrated Development in Haiti (IEDIH) to mentor fisher associations and to provide complementary training in management and organizational strengthening.” This statement is misleading because the capacity-building training and mentorship being provided to fisher organizations through IEDIH are primarily intended to strengthen organizational development, not strengthen park management. AMI note: Minor modifications made to the report text based on this clarification. 259 Refinement and approval of the Pedro Cays and Surrounding Waters Management Plan, and emergence of a plan for South West Cay (page 43) 1st para., 5th line: Change spelling from “Otuaokan” to “Otuokon” 2nd para., line 2: “… the no-take South West Cay Fish Sanctuary, which lies within the larger Pedro Cays marine ecosystem” should read: “the no-take South West Cay Fish Sanctuary, which lies within the larger Pedro Bank marine ecosystem”. The cays are a small part of the Pedro Bank ecosystem. AMI note: Minor modifications made to the report text based on this clarification. Collaborative governance, management capacity and infrastructure, and challenges (page 45) “The Pedro Cays and Surrounding Waters Management Plan covers a large area which is a far and often challenging boat trip from the main island of Jamaica. As noted, C-CAM has taken on management responsibility for the South West Cay Fish Sanctuary, and under CMBP focused much of its efforts on monitoring, patrolling, outreach and education of fishers, and keeping demarcation buoys for this no-take fish sanctuary in working order.” The first paragraph might be more accurate and set the context more clearly by stating the South West Cay Fish Sanctuary is the only declared protected area on the Pedro Bank, and therefore the only site available for official management. Suggested wording: “The Pedro Cays and Surrounding Waters Management Plan covers a large area, which is a far and often challenging boat trip from the main island of Jamaica. As noted, C-CAM has taken on management responsibility for the South West Cay Fish Sanctuary, the only officially declared protected area on the Pedro Bank, and under CMBP…” The MSP is for the Pedro Bank and not the Pedro Cays only, so the following sentences should be corrected: Line 4: “…and the Marine Spatial Plan (MSP) for the Pedro Bank officially proposed…” Line 8: “… the MSP for the Bank” … AMI note: Minor modifications made to the report text based on this clarification. “CMBP partially supported development of the Pedro Cays management plan, the MSP for the area, and led, with consultant support, development the Fisheries Strategic Action Plan for the Jamaican Seascape: Pedro Bank and the Southwest Coast.” This statement is incomplete. CMBP prepared the Rapid Fisheries Sector Assessment and a Fisheries Strategic Action Plan. The action plan was based on the assessment. AMI note: Minor modifications made to the report text based on this clarification. 260 “Government officials indicate that the management plan has been presented before an Inter-Ministerial Committee, which has signed off on major aspects of the plan; however, the committee has not yet formally approved the full document as of February 2018. Given the advanced stage of the process, formal approval may be realized with strategic coordination and communication with key state agency office holders/stakeholders. It is anticipated that formal approval of the management plan can be achieved within the current CMBP time period.” This statement might be based on a misunderstanding between the management plan and the MSP. TNC received a letter from NEPA fully accepting the management plan, which was submitted to an inter-agency GEF-UNDP Project Steering Committee comprised of government and NGO entities. However, the Pedro Bank MSP has not yet been submitted by NEPA to the National Council on Oceans and Coastal Zone Management, an inter-ministerial committee, for review and approval. AMI note: Minor modifications made to the report text based on this clarification. Collaborative governance, management capacity and infrastructure, and challenges (page 46) “The Jamaica Defence Force (JDF) Coast Guard and Marine Police, and the government more generally, are trying to bring greater organization, safety, waste management and societal norms to the cays, and to the fisher encampments that are a regular presence”. It should be clarified that the South West Cay is not inhabited and there are no “encampments” (except nesting birds of several species). The organization, waste management and safety really refer to the other two Cays – Middle Cay and SE (Top) Cay. Also, the report implies the Marine Police falls under the JDF rather than the the Jamaica Constabulary Force (the police). AMI note: Minor modifications made to the report text based on this clarification. “For example, although there is broad interest among many agencies in the management of important natural resources at in the Pedro Cays (e.g. conch, lobster, fish), no agency steps up and takes responsibility for implementation of the Pedro Cays management plan.” In the draft National Cays Policy, NEPA has been tasked to create a Cays management unit as a governance approach for managing all Cays. The policy has not yet been approved. The evaluation report implies since no one entity is responsible for the management plan, it is not being implemented. Multi-agency management is a legitimate form of management, and there are assigned roles for different entities in both the management plan and the MSP. C-CAM is implementing aspects of the management plan. AMI note: Minor modifications made to the report text based on this clarification. 261 “The absence of such collaboration may limit the ability for management to be scaled up, and thereby reduce threats to marine biodiversity in priority CMBP sites and across the Jamaica seascape.” (page 48) While these entities were involved in the two-year participatory development process of the Pedro MSP, we agree collaboration could be better and more frequent, and we agree to share again, and more widely, both the Pedro Cays management plan and the Pedro MSP. AMI note: we agree. Capacity building of protected area management organizations (page 44) “C-CAM received a boat to conduct enforcement, staff supplies (e.g. uniforms, boots etc.), and AGRAA survey equipment. South West Cay demarcation moorings have been replaced twice. And office-based staff have been provided computers and software technology to assist C-CAM with sub-grantee reporting requirements. These staff will soon get training in Quickbooks.” This statement is correct, but omits that the staff is now SCUBA certified. The staff will receive AGRRA basic training this year. AMI note: Minor modifications made to the report text based on this clarification. Evolving and needed partnerships with government (page 48) “It is also important to note that fish sanctuary management organizations, such as C-CAM, BBFFS and BREDS-Treasure Beach Foundation (Galleon St. Elizabeth SFCA) have benefitted significantly from their existing relationships with the Fisheries Division and NEPA, as well as evolving relationships with the national JDF Marine Police. C-CAM in particular is strengthening its working partnership with the Marine Police, which commonly provides transport assistance for staff to the Pedro Cays.” This statement is incorrect and misleading. The JDF Coast Guard provides transport assistance for CCAM staff going to the Cays. Also, the Coast Guard and Marine Police fall under two different agencies. The Coast Guard falls under the Jamaica Defence Force (the army) and the Marine Police falls under the Jamaica Constabulary Force (the police). AMI note: Minor modifications made to the report text based on this clarification. III. Expected Result 3: Sustainable Fisheries and Sustainable Livelihoods for Fishers Progress in adopting sustainable fishing practices, strengthening fisher associations and alternative sources of income (page 52) 262 “As to the economic alternatives and market creation efforts for the Women Artisan Association in Los Corozos, these CMBP activities are clearly not bringing the expected benefits. The production of crafts has stalled for months, and it seems that members of the association are no longer interested in producing crafts made from coconut, an alternative income activity probably selected unilaterally by CMBP. The association expressed interest in crafts made from different material (e.g. conch), candle making and purses. Nevertheless, the members of women’s association are fervent defenders of the mangrove ecosystem, which is a “positive side effect” of CMBP implementation.” This statement is incorrect. This women’s group initiated their coconut craft production prior to CMBP with funding from Esperanza Bank Foundation. CMBP agreed to continue assistance since the initiative was already established. Also, the mangrove restoration pilot was the group’s main activity, not a side effect. The crafts business was more of a side effect. AMI note: Minor modifications made to the report text based on this clarification. Fisher association capacity strengthening and the promotion of sustainable practices (page 54) “It appears that suriperas are cheaper to buy and maintain, but there are questions about their shrimp fishing effectiveness, compared with licuadoras.” We believe this statement is incorrect. Suriperas are indeed less expensive than licuadoras but they are not currently available for purchase. CMBP hopes to create a market for them. In regard to the fishing effectiveness, results of the feasibility study on the suripera performed in 2016 shows that suriperas have better performance and effectiveness in terms of CPUE (Catch Per Unit Effort). AMI note: the report of the suripera trials made no comparison with licuadora nets in terms of CPUE. Strengthening of fisher association capacities (page 56) “For example, CODOPESCA participated in suripera trials, but have not seen the results. The idea of organizing a workshop with all institutions and organizations, to socialize all ongoing and recently finished projects in Samana Bay was suggested.” This statement is incorrect. We did share the results of the trials during fisherfolk roundtables and organized a workshop in Santo Domingo to present the results of the feasibility study of the suriperas. CODOPESCA attended this workshop along with many other stakeholders. AMI note: Minor modifications made to the report text based on this clarification. 263 “Other complaints from fishers were the requirement from TNC to sign a document for the release of responsibility in their participation on fishing gear trails (sic) and that they have not received the report with the results of these fishing gear trials”. While some fishers may not have received the report, this statement is incorrect. During the fisherfolk roundtables of November of 2016, TNC presented the results of the fishing gear trials and later handed out the report to the fishers who attended. This activity was included in our Annual Report. AMI note: Minor modifications made to the report text based on this clarification. Summary Analysis of Findings (page 57) Key achievements should include providing HACCP training and equipment for cold storage of catches provided to over 50 fisherfolk. 70% of participants reported tangible economic benefits as a result of the activity, which responded to stakeholder needs and the Fisheries Sector Action Plan. (page 57) AMI note: This has been added to the Grenadine Bank ER3 seascape section, as well as the achievements section in both the executive summary and main conclusions. Key challenges should include Caribsave’s withdrawal after two years of the project. Much of the ground work that Caribsave performed in terms of developing relationships with stakeholders planning activities had to be redone. AMI note: Added to the list of challenges a note saying that “TNC identified the withdrawal of Caribsave from the project as a challenge”. “CMBP activities implemented under Expected Result 3 in both St. Vincent and the Grenadines have achieved limited success to date and, in cases such as the lobster gear pilot project, resulted in failure” (page 58) A more accurate statement would be that the lobster gear activity needed more time to follow up on the results of the pilot. The idea always was to conduct a retrial (or second phase), incorporating key learnings. However, the team realized it would take at least two years to address key issues, such as a more effective pot design, an affordable and environmentally sound source of bait, and a larger supply of pots. Given conditions at the time, the team decided to discontinue the pilot and re-start it under another project if funding became available. AMI note: Minor modifications made to the report text based on this clarification. “Fishers no doubt have benefitted from CMBP trainings, and need more training and alternative livelihood opportunities, if threats to nearshore ecosystems are to be reduced. That said, many of the activities aimed at promoting sustainable fishing practices have either 264 failed or not yet proven successful. For example, fishers, though interested and open to try an alternative, abandoned the lobster pot project owning to the fact that the trap design did not perform well, and two pots per fishermen was also not enough to secure adequate catch.” (page 60) Further to the above comment, the reason only two pots were provided per fisher was because this was a pilot activity. More pots would have been provided in a second phase. As additional background information, fishers in Bequia (another Grenadine island, not far from Mayreau) successfully use this design of lobster pot, and fishers in Isle de Rhonde, with whom SUSGREN previously worked, found the pot design to work successfully. At least five of these fishers in Isle de Rhonde relinquished their destructive trammel nets and switched to lobster pots. The pilot project in Mayreau under CMBP accomplished its objectives. AMI note: Minor modifications made to the report text based on this clarification. Observations and Recommendations (page 61) “Document successful approach and lessons learned for application, as needed, in other CMBP seascapes” We agree with this general recommendation, though the reader might have a better understanding of the recommendation if some background information was included on CMBP’s mangrove restoration work and its impact on the surrounding community. Four previous restoration efforts were unsuccessful in Grenada until the change in method from PVC pipes to bamboo tubes (biodegradable and safe for environment) as conduits to help plantlets take root. Plantlets have now established themselves successfully and their survival rate is high compared to other efforts. AMI note: Minor modifications made to the report text based on this clarification. Communication and education (page 62) This section is incomplete, as there is no mention of an outreach officer hired for the SIOBMPA to increase community awareness of conservation issues. AMI note: Minor modifications made to the report text based on this clarification. Summary analysis of findings (page 63) “About half of the planned activities for the North Coast of Hispaniola Seascape have been completed, including conducting a feasibility assessment for the implementation of a pilot fishing gear swap program, and a rapid assessment of the legal framework for an enforcement unit for Three Bays National Park. Activities pending include wide distribution of the Fisheries Sector Action Plan (FSAP), development of a feasibility study for the implementation of a pilot oyster cultivation project, a pilot study to test reduction of juvenile catch using alternative fishing gear, and drafting a conservation agreement for the 265 designation of the first coral and fish reserve in Haiti. Partially completed activities include expansion of the apiculture project to other communities, and the establishment of a community enterprise for ecotourism services.” This statement is misleading and incorrect. It should be clarified the status of progress is for Year 4 and not the entire duration of the program. Also, there are no pending plans to further distribute the Fisheries Sector Action Plan (FSAP) or to develop a “feasibility study for the implementation of a pilot oyster cultivation Project”. For partially completed activities, it should be noted CMBP is establishing two community-based enterprises for ecotourism services and honey packing and marketing. AMI note: according to Annex B consolidated CMBP revised Workplan, this statement is true, and is relevant for Year 3. AMI did verify the accomplishment according to the progress reports but also through the interviews with FOPROBIM. Except for the establishing of two community-based enterprises for ecotourism services and honey packing and marketing, which were partially implemented, were not implemented. Other minor modifications were made to the report text based on this clarification Summary analysis of findings (page 63) “CMBP is unique in Haiti in that there are no other programs that provide technical and institutional assistance in marine biodiversity conservation.” This statement is not accurate. There are other programs developed by such partners as IDB, UNEP, UNDP, and Reef Check. AMI note: Minor modifications made to the report text based on this clarification. Progress in accomplishing CMBP performance indicators targets (page 64) “According to TNC, a feasibility analysis for the implementation of a fishing gear swap pilot program was conducted by CMBP. The feasibility analysis proposed a small, six-month pilot aimed at testing the reduction of juvenile catch using alternative fishing gear. That pilot test was planned for year 3 of the project; however, it has not started yet.” This statement is slightly misleading. The feasibility analysis proposed a small pilot aimed at testing the reduction of juvenile catch using alternative fishing gear. It did not recommend a particular period of time. CMBP was proposing a six-month pilot because that was the remaining time in Year 4. AMI note: Minor modifications made to the report text based on this clarification. Progress in accomplishing CMBP performance indicators targets (page 65) 266 “In fact, it is not yet clearly defined how CMBP will measure the reduction in threats to marine coastal diversity, how frequently it should be measured, and through which indicator.” A reduction in threats to marine and coastal biodiversity is not directly measured by the program. CMBP uses a combination of USAID standard and customized indicators to measure improved biodiversity and natural resource management (reference PIs 1, 2 and 5), which are also indicative of reduced threats. There are also other social indicators (ref PI 6 and 7) that indirectly bear evidence of reduced threats to the marine environment. AMI note: Minor modifications made to the report text based on this clarification. Strengthening effective fisheries policy and building a network of no-take fish sanctuaries (page 69) “The FSAP looked beyond the Pedro Cays to include the southwest coast.” This statement appears to suggest that the FSAP’s scope was too large. It covered the Jamaica seascape, which is defined as Pedro Bank and southwest coast of Jamaica (from Whitehouse in Savanna-la-Mar to Great Bay, St. Elizabeth). Also, this section should include information about the fish sanctuary network week-long training session held in August 2017. AMI note: The use of the phrase “looked beyond” was not meant as a negative, only to imply that the FSAP did not simply consider the Pedro Cays. Nonetheless, minor modifications made to the report text based on this clarification. Training and capacity building of C-CAM, BBFFS and fishers (page 70) “The focus group discussion with the Bluefield’s Bay Fishermen’s Friendly Society (BBFFS), which manages the Bluefield’s Bay SFCA, revealed that trainings have been well received, but have not yet led to the development of pilot programs or the creation of alternative livelihoods, with the notable exception of people hired as staff at BBFFS.” Since Year 3, a contractor has been working on the ground with the group in Bluefield’s in developing and formalizing alternative livelihood options in crafts and ecotourism. Training and capacity building of C-CAM, BBFFS and fishers (page 70) “The Evaluation Team did not observe any apiculture pilot projects; however, many individuals in the BBFFS focus group discussion expressed interest in this activity.” The apiculture project was started under a previous program in Bluefield’s. As described during the interview with the evaluation team, CMBP is providing assistance to members of BBFFS who had participated in this program and still have functioning hives. In Y4, 267 these individuals will receive additional trainings and bee boxes to expand honey production and further establish their alternative livelihoods. These planned trainings are mentioned in the final paragraph of this section, and this initiative is also included in the Y4 workplan. IV. Expected Result 4: Effective Management and Governance of Marine Protected Areas (page 85) Performance indicators “Indicator 3 (new laws, policies and plans) has produced mixed results; overachieved in Jamaica and fully achieved in Haiti, but half way in Grenadine Bank, and 0% of target completion in Samana Bay.” As explained above, this statement is incorrect. MPA management plans and action plans “The action plans for coral and marine mammal conservation in Samana Bay are of sub￾standard quality and require significant improvements.” As explained above, this statement is misleading. Overarching Conclusions (page 89) “Some MPA authorities and local sub-grantees are not using the METT correctly. Review of METT self-evaluation scores from multiple CMBP sites demonstrates that most are using the terrestrial versus the marine version of the METT. This is an important finding as many of the indicators in the terrestrial METT are not applicable to marine conservation. Moreover, interviews with sub-grantees and other MPA management authorities revealed few examples of how these organizations are analyzing results of their evaluations and subsequently incorporating new knowledge into management planning processes. Such analysis and renewed planning may be occurring, just to a limited degree. Finally, some sites would still benefit by integrating government and other stakeholder perspectives into the METT process.” We are not aware of marine vs. terrestrial versions of the METT. The WWF 2007 METT tool has questions that are specific to the management of “terrestrial only” protected areas, “marine only” protected areas, and protected areas that cover both types of ecological systems. Most questions in the tool used are quite general and applicable to marine and overall protected area management. CMBP METT assessments do not include components applicable only to terrestrial protected areas when calculating the final scores. While many CMBP seascapes cover only marine ecosystems within the protected area boundaries, some areas have both a terrestrial and marine component that are under various levels of management. Also, CMBP uses different versions of the WWF METT in some 268 countries based on previous methodologies used by government entities or cooperation partners/projects. This approach allows for continuity in information exchange with local authorities and decision makers. It would be disruptive for us to change this. A limitation of the METT tool is that it does not allow comparison between sites. However, the scoring system accurately tracks progress for one site over time. We agree with the finding that sub-grantees and MPA management authorities are not fully analyzing the METT results and incorporating knowledge into management planning processes. More attention could be placed on this during a cost extension. AMI note: Minor modifications made in the seascape sections, as well as the main conclusions and recommendations, to address this clarification. V. Sub-grantee Organizational Capacity Assessment (pages 74-85) “This section presents findings, conclusions and recommendations based on CMBP mid-term performance evaluation question 3: What is the capacity (financial, managerial, and programmatic/technical) of CMBP’s local implementing partners (i.e. sub-grantees)? What implications does sub-grantees’ overall capacity have on CMBP’s ability to achieve expected results? And What implications does sub-grantees’ capacity have for the sustainability of the activities implemented and results achieved by these organizations? As mentioned in the evaluation design, methods and limitations section, the Evaluation Team conducted an assessment of the organizational capacity of each CMBP sub-grantee. USAID’s Organizational Capacity Assessment (OCA) tool was applied and complemented with additional information obtained during interviews and focus group discussions with TNC, government agencies, stakeholders and the sub-grantees.” Jeniffer Calderón, the CMBP Senior Grants Specialist, was not interviewed or requested to provide information about this topic. Nevertheless, we would like to offer the observations below based on her monitoring of the sub-grantee compliance with the award. Center for the Conservation and Eco-Development of Samana Bay and its Surroundings “The Evaluation Team concludes that, with an overall average score of 3.35, CEBSE is at a Moderate Level of organizational capacity. Specific areas of Moderate Capacity include financial management and internal control, program management, and organizational management and sustainability. All other areas are considered at a Strong Capacity level. Under financial management and control, CEBSE could increase its capacity level for cost￾sharing, which is currently at Low Capacity level. Similarly, areas at the Basic Capacity level that CEBSE could improve include service delivery standards (under project performance management), strategic business planning, and external communications (both under organizational management and sustainability). “ 269 Compared with other small organizations, we agree CEBSE is a moderate level of organizational capacity. “CEBSE struggles to secure regular funding. Because CEBSE’s area of focus is limited to the Samana Bay region, it is difficult for the organization to secure funding proposals at the national level (CEBSE has not expanded its geographic scope in the last several decades). One key related negative impact of this challenge has been the organization’s inability to retain marine biologists or ecologists as permanent staff. Of note, CEBSE considers one of its strengths the capacity for collaboration with adept scientists, both local and international, as one of its strengths. So as an interim solution to deal with funding challenges, the organization hires scientists when needed. Finally, CEBSE staff also indicated lack of vehicles, equipment and technical assistance as particular challenges to achieving certain programmatic results associated with gaps in funding”. We do not believe it is a priority now for CEBSE to increase its geographic focus. Instead, CEBSE should develop fundraising strategies and capacity. The most recent report on TNC subrecipient monitoring indicates the following:  CEBSE’s board of directors is not active, nor are its members well positioned to provide guidance and exposure to the local industries, government, and conservation efforts.  CEBSE has a high dependency on TNC-provided funding. According to CEBSE’s managers, TNC represents more than 90% of their total funding. CEBSE’s sustainability is in jeopardy if they are not able to obtain additional funding before the end of the TNC￾funded project.  Board members do not include key members in the Samana economic or government environment. Although some board members are aligned with CEBSE’s mission, they are not well positioned to obtain funding for the organization. “As alluded to previously, CEBSE has limited capacity for financial management and internal control systems. Staff indicated one of its greatest challenges is the organization’s inadequate ability to consolidate funding from different donors in order to manage administrative expenses. The organization also lacks a full-time accountant, and has at times in the past had some issues meeting donor cost-share requirements. However, it is important to recognize that CEBSE does produce an annual project budget that reflects individual project budgets, includes multi-year projections of the organization’s revenues and expenditures, indirect cost rates for projects, and necessary line items to meet donor requirements.” CEBSE’s limited capacity for cost share is related to challenges in fundraising. They have appropriate controls in place to track match, but they do not have the funds. Relevant information on most recent TNC subrecipient monitoring report indicates that CEBSE’s accounting system is deficient and shows a weak system of controls. However, CEBSE has recently provided evidence of improvements in the system. CEBSE´s budget process is deficient because they do not have secured income sources and have difficulty in developing projections. 270 “In summary, CEBSE has the organizational staying power, staff capacity, and appropriate level of programmatic organization to achieve present and future expected results under CMBP. That said, the organization still requires oversight and financial support of an implementing partner, such as TNC, to achieve successes, and is not yet ready to operate without that support. CEBSE could strongly benefit from increased administrative support, primarily cost-sharing capacities, in order for the organization to move towards independence, and sustain activities that have been implemented under CMBP.” We agree with this summary. Sustainable Grenadines, Inc. “The Evaluation Team concludes that, with an overall average score of 3.62, SUSGREN is at a Strong Level of organizational capacity. The only capacity area where SUSGREN is at Moderate Capacity is financial management and internal control, particularly cost-sharing. All other organizational capacity areas are at the Strong Capacity level.” In our opinion, SUSGREN is at a moderate level organizational capacity. Similar to CEBSE, they have fundraising issues. “Historically, SUSGREN hires staff on a per-project basis, which makes staff retention nearly impossible. Once a project or grant contract is complete, that staff position disappears. This does not allow for continuity of staff, building of staff skill sets, or consistent organizational presence in the community. With CMBP funds, SUSGREN has been providing project support in both areas MPA capacity building and management and sustainable livelihoods. SUSGREN recognizes the critical need to secure more permanent funding sources for staff positions, specifically identifying the need for a development director. Staff are developing a five-year strategic plan that is expected to be complete in spring 2018.” Staff retention is a major problem for SUSGREN. Lack of secure, on-going finance is one of the main reasons SUSGREN has been unable to retain staff members. Seven SUSGREN staff positions are currently funded at least 50% by CMBP and five positions are 100% funded by CMBP. SUSGREN has virtually zero funding secured for these staff positions post-CMBP. This is a major vulnerability for the organization and a threat to the longevity of CMBP results. SUSGREN recognizes the critical need to secure more permanent funding sources for staff positions and has identified actions to develop such funding sources within a five-year financing plan. Seed funding will be required to establish the planned sustainable financing mechanism(s). The financing plan will lay out how SUSGREN’s five-year strategic plan will be financed. Both documents are currently in development, scheduled for completion in May 2018. 271 SUSGREN´s leads (James and Orisha) have the capacity and expertise to do fundraising; however, they are dedicated to implementing projects, which leaves almost no room for fundraising efforts. Regarding annual work plans (organizational management and sustainability point 7.2), SUSGREN currently does not have an annual work plan, which is a weakness not reflected in the score provided. Likewise, the ratings of 4 for change management (point 7.3) and 4 for knowledge management and external linkages (point 7.4) are overestimates as SUSGREN does not have structured systems in place for these areas. “SUSGREN and TNC both agree the sub-grantee has potential for significant growth and independent sustainability. Each concludes that SUSGREN’s work in the region will continue after CMBP is complete. That said, SUSGREN requires training to hire additional staff to do fundraising and business development efforts. The strategic plan can and perhaps will include comprehensive annual budgets, in contrast to how SUSGREN currently reviews finances on an individual project basis. SUSGREN also requires capacity improvements in succession planning and management, and improved IT systems for information sharing, additional issues that could be addressed in the strategic plan. Of note, SUSGREN’s relationship with TNC is both strong and beneficial.” Similar to CEBSE, SUSGREN still requires oversight and financial support of an implementing partner, such as TNC, to achieve expected results, and is not yet ready to sustain results. SUSGREN needs to secure funding for an Executive Director that does not directly implement projects, but who leads the organization’s fundraising and business development efforts. Also, SUSGREN needs to secure funds for a financial/administrative assistant with appropriate knowledge to use their financial system and document transactions. Foundation for the Protection of Marine Biodiversity “The Evaluation Team concludes that, with an overall average score of 3.65, FOPROBIM is at a Strong Level of organizational capacity. Similar to SUSGREN, the only capacity area where FOPROBIM is at a moderate level is financial management and internal control. All other organizational capacity areas are at the Strong Capacity level. Also, similar to SUSGREN, the main capacity sub-section which needs strengthening is cost-sharing (under financial management and control). All other sub-sections are at the Moderate to Strong capacity level.” In our opinion, FOPROBIM is also at a moderate level in terms of organizational capacity. The lack of an accounting system makes this organization vulnerable in financial management. Also, FOPROBIM does not have policies and procedures in place, and they adjust to donor requirements on a case-by-case basis. 272 “To sustain CMBP activities in the future, both ANAP and TNC indicate that more staff support is needed to develop the organization and implement activities. Of note, CMBP funding nearly doubled FOPROBIM staff capacity, and the organization hopes it will retain the ability to implement activities, perhaps at a lower level, once CMBP funding expires.” Similar to CEBSE and SUSGREN, FOPROBIM still requires oversight and financial support of an implementing partner, such as TNC, to achieve expected results, and is not yet ready to sustain results. Caribbean Coastal Area Management Foundation “The Evaluation Team concludes that, with an overall average score of 3.71, C-CAM is at a Strong Level of organizational capacity. Similar to SUSGREN and FOPROBIM, C-CAM only capacity area at a Moderate Capacity level is financial management and internal control. All other capacity areas are at the Strong Capacity level. Also, similar to SUSGREN and FOPROBIM, the main organizational capacity sub-section which needs strengthening is cost￾sharing (under financial management and control). All other sub-sections are at the Moderate to Strong capacity level.” In our opinion, CCAM is also at a moderate level in terms of organizational capacity. Financial management and administration still need some improvements and oversight. “CMBP and its partners may wish to put together a plan for strengthening sub-grantee cost sharing capacities, as this is a common weakness for each organization. The cost sharing sub￾section assesses whether the organization has systems to track, report, and document cost sharing and meet the cost sharing requirement in agreement with various donor regulations.” We agree there should be a plan to strengthen partners capacities, but we believe the focus should be fundraising and the development of policies, procedures manuals, and accounting systems. Once they demonstrate effective fundraising and stronger administrative and financial systems, they will be a better position to provide cost sharing. “From interviews, it is apparent that sub-grantees have either not had any donor cost￾sharing requirements or had prior donor cost-sharing requirements that were not met, have no written policies and procedures for recording and reporting on cost-sharing, nor written policies and procedures on cost sharing that are inadequate and require substantial changes. More generally, each sub-grantee may benefit by focusing on diversifying funding, strengthening project design and management capacity, and becoming self-sufficient organizations.” We agree. TNC did not ask partners to provide match because we knew about their capacity limitations and believed they would not be able to accept USAID funding if it depended on a cost-sharing requirement. 273 VI. Recommendations and Suggested Actions to be Completed by August 31 Expected Result 4 Samana Bay Consultant Recommendations CMBP Comments CMBP should consider counting the resolution that creates six no-take zones in Samana Bay, as a policy under CMBP monitoring indicator 3 As stated above, we already have included the resolution as a policy under CMBP monitoring indicator 3 In the future CMBP should use the marine version of the METT to measure changes in MPA management The METT version is provided by the Environment Ministry, and the only version approved. Management plans for the six no-take zones should be developed CMBP will develop the management plans for the no-take zones, but they will require approval of the Ministry of Environment Jurisdiction and competencies between MARENA and CODOPESCA should be clarified to ensure appropriate management of fisheries resources in Samana Bay CMBP has already conducted a legal research to identify possible overlapping competencies between MARENA and CODOPESCA Expected Result 3 Samana Bay Consultant Recommendations CMBP Comments Roundtable meetings between fishers and authorities should continue to receive support from CMBP Roundtables are scheduled on a regular basis until August Future trials with new fishing gear should be conducted side-by-side with traditional gear, to demonstrate the value of the former in comparison to the latter. This is also likely to increase acceptance of the new gear by fishers Agreed. This is the approach CMBP is taking to allow comparison between the traditional Bottom Trawl Net and the suripera net Expected Result 3 Grenadine Bank Recommendations CMBP Comments  Prioritize gender mainstreaming for remaining CMBP training courses  CMBP will attempt to increase female participants in the remaining HACCP courses this year.  Document successful approach and lessons learned for application, as needed, in other CMBP seascapes  CMBP will document the successful approach and lessons learned for the mangrove restoration activity. Expected Result 4 Three Bays Recommendations CMBP Comments CMBP should work with ANAP to identify the support it requires to implement the 3BNP Management Plan CMBP has already arranged meetings with ANAP and USAID, UNDP, IDB and the Canadian Embassy in Haiti to discuss the priorities of the Management Plan. During the next few months we will organize 274 another meeting with ANAP to identify the support it requires to implement the 3BNP Management Plan. CMBP should seek ways for accelerating the procurement process for supplies and equipment required for day-to-day park activities We have been working with USAID on this, and the supplies should be delivered in the next few weeks. The Three Bays National Park management plan could be enhanced and complemented with a detailed operational plan and a realistic budget that focuses on targeted actions for marine areas in the park CMBP prepared a draft operational plan and budget and submitted it to ANAP for feedback. ANAP has not yet responded. We will follow up again. Expected Result 3 Three Bays Recommendations CMBP Comments The pilot test for fishing gear swap should be implemented as soon as possible, to allow for learning and improving CMBP’s implementation effectiveness and impact We will implement the pilot program in Year 4. CMBP should make efforts to ensure ANAP hires park wardens under the Ministry’s payroll when the CMBP ends We will continue to encourage ANAP to hire the park wardens under its payroll. They have previously agreed to this in principle. Expected Results 4 Jamaica Recommendations Comment CMBP prioritize support for C-CAM to further advance plan development CCAM is expected to complete South West Cay Special Fishery Conservation Area management plan by August 31. CMBP should continue building capacity for patrolling, identifying weaknesses and gaps, and strengthening emerging enforcement programs Building capacity for patrolling and monitoring will continue in Year 4 USAID should follow-up with TNC to assess if the new staffing approach is proving effective The sanctuary manager is already showing better management, reporting and implementation capacity and increased achievements by C-CAM Linked to the effort to secure plan approval, and request NEPA to clarify and broadly disseminate its leadership roles and responsibilities TNC will contact NEPA and make this request. The draft National Cays Management Policy recommends that NEPA be the chief entity responsible for Cays, but it has not been finalized. Prioritize support for a MOU between C-CAM and the Marine Police, and share lessons from partnership with other fish sanctuary management organizations CMBP will share lessons learned with other sanctuaries. However, the MOU between CCAM and the Marine Police is no longer being pursued in view of USAID’s concerns about enforcement. 275 Expected Results 3 Jamaica Recommendations Comments As planned, facilitate pilot ecotourism activities in Bluefield’s and apiculture training in remaining CMBP time Integrate the marketing plan that BBFFS developed into any ecotourism pilot CMBP is now conducting apiculture training, and tour guide training (a part of the ecotourism activities) will take place before the end of the year. However, there are no further plans this year for the ecotourism pilot. Revisit and prioritize gender mainstreaming efforts Noted. Revisit and determine the most effective communication methods relative to the supporting agencies and the roles these agencies needs to play in CMBP Agreed. 276 Annex VIII – Table of monitoring indicators The following indicators were scrutinized: CMBP Indicator 3 Number of new laws, policies, strategies, plans, or regulations addressing biodiversity conservation officially proposed or adopted as a result of USG assistance. CMBP Indicator 5 Number of hectares of biological significance and/or natural resources under improved natural resource management as a result of USG assistance. CMBP Indicator 6 Number of people with tangible and/or perceived increases in economic benefits derived from improved natural resource management and conservation as a result of USG assistance. CMBP Indicator 7 Number of fishers and others who have applied new technologies or management practices as a result of USG assistance CMBP Indicator 8 Number of person hours of training in natural resources management and/or biodiversity conservation supported by USG assistance Expected Result 4. Indicators for effective management and governance of MPAs/MMAs • CMBP Indicator 3. • CMBP Indicator 5. Expected Result 3: Indicators for more sustainable fisheries sector • CMBP Indicator 6. • CMBP Indicator 7. Cross-cutting indicator for Expected Result 3 and 4:  CMBP Indicator 8.