FINAL EVALUATION OF THE U.S. CORAL TRIANGLE INITIATIVE SUPPORT PROGRAM Evaluation Report October 2013 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Doug Baker, Miriam Balgos, Mike Duthie and Ninnette Lasola of Social Impact, Inc. and partner Management Systems International. FINAL EVALUATION OF THE U.S. CORAL TRIANGLE INITIATIVE (US CTI) PROGRAM EVALUATION REPORT October 17, 2013 Contract Number: RAN-I-00-09-00019 Order Number: AID-486-TO-13-00004 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. US CTI Evaluation Report iii CONTENTS Contents ........................................................................................................................................ iii Figures and Tables ....................................................................................................................... vi Acronyms ......................................................................................................................................vii Executive Summary ...................................................................................................................... 1 Evaluation Purpose and Evaluation Questions .......................................................................................................1 Program Background ....................................................................................................................................................1 Evaluation Design, Methods and Limitations ..........................................................................................................2 Most Important Findings ..............................................................................................................................................3 Most Important Conclusions ......................................................................................................................................5 Primary Recommendations .........................................................................................................................................6 I. Evaluation Purpose & Evaluation Questions ........................................................................... 8 Evaluation Purpose ........................................................................................................................................................8 Evaluation Questions ....................................................................................................................................................8 Gender ........................................................................................................................................................................9 II. Program Background ............................................................................................................. 10 Program Objective ......................................................................................................................................................10 Program Structure ......................................................................................................................................................11 Program Implementation Status ..............................................................................................................................11 III. Evaluation Methods & Limitations ....................................................................................... 12 Evaluation Design ........................................................................................................................................................12 Data Collection Methods ..........................................................................................................................................12 KII Sampling ..................................................................................................................................................................15 Survey Sampling ...........................................................................................................................................................15 Integration of Gender into Data Analysis .........................................................................................................18 Limitations and Risks ..................................................................................................................................................18 Attribution and Generalizability ..........................................................................................................................18 Selection Bias ...........................................................................................................................................................19 Response Bias ..........................................................................................................................................................19 Logistical Constraints ............................................................................................................................................19 IV. Findings & Conclusions ......................................................................................................... 20 Evaluation Question One—Concept .....................................................................................................................20 Regional Approach Context ................................................................................................................................20 Findings on the Overall Approach of Support to the CTI-CFF as a Regional Organization ................20 Conclusions on the Overall Approach of Support to the CTI-CFF as a Regional Organization ........21 iv US CTI Evaluation Report Findings on Specific Actions Supporting the Regional Approach ................................................................21 Conclusions on Specific Actions Supporting the Regional Approach ........................................................24 Findings on Vertical Integration ..........................................................................................................................24 Conclusions on Vertical Integration ..................................................................................................................25 Findings on Horizontal Integration .....................................................................................................................25 Conclusions on Horizontal Integration .............................................................................................................25 Evaluation Question Two—Design ........................................................................................................................26 Findings on Overall Development Hypothesis and Results Framework ...................................................26 Conclusion on Overall Development Hypothesis and Results Framework .............................................28 Findings on Subordinate Results Frameworks (Results Areas) ...................................................................28 Conclusion on Subordinate Results Frameworks (Results Areas) .............................................................28 Findings on Gaps in the Results Framework ....................................................................................................29 Conclusion on Gaps in the Results Framework .............................................................................................30 Evaluation Question Three—Implementation .....................................................................................................31 Overall Findings on Effectiveness of Intermediate Results ...........................................................................31 Overall Conclusions on Effectiveness of Intermediate Results ...................................................................31 Findings on Result Area 1: Regional Platforms ................................................................................................31 Conclusions on Result Area 1: Regional Platforms ........................................................................................32 Findings on Result Area 2: EAFM .......................................................................................................................32 Conclusions on Result Area 2: EAFM ...............................................................................................................32 Findings on Result Area 3: MPA ..........................................................................................................................32 Conclusions on Result Area 3: MPA ..................................................................................................................33 Findings on Result Area 4: CCA .........................................................................................................................33 Conclusions on Result Area 4: CCA.................................................................................................................33 Evaluation Question Four—CTI-CFF.....................................................................................................................34 Findings on Consonance of US CTI with CTI-CFF Goals ............................................................................34 Conclusions on Consonance of US CTI with CTI-CFF Goals ....................................................................34 Evaluation Question Five—Lessons Learned and Future ..................................................................................34 Findings on the Implementation Context .........................................................................................................35 Conclusions on the Implementation Context ................................................................................................. 35 Findings on Using Multiple Mechanisms ............................................................................................................35 Conclusion on Using Multiple Mechanisms ......................................................................................................36 Findings on Consortium Approach ....................................................................................................................36 Conclusion on Consortium Approach ..............................................................................................................37 Findings on M&E as a Management Challenge ................................................................................................. 38 Conclusion on M&E as a Management Challenge ...........................................................................................38 US CTI Evaluation Report v Findings on Other Implementation Challenges ...............................................................................................38 Conclusion on Other Implementation Challenges .........................................................................................39 Findings on Innovative Practices .........................................................................................................................39 Conclusion on Innovative Practices ...................................................................................................................39 Findings and Conclusions on Gender .....................................................................................................................39 Findings on Gender ................................................................................................................................................39 Conclusions on Gender ........................................................................................................................................40 V. Recommendations .................................................................................................................. 41 Urgent Recommendations ........................................................................................................................................41 Regional-Level Recommendations ..........................................................................................................................42 Country-Level Recommendations ..........................................................................................................................43 Assistance Model Recommendations .....................................................................................................................44 Project Design, Implementation and Management Recommendations ..........................................................44 Other Recommendations ..........................................................................................................................................45 Annexes ........................................................................................................................................ 46 Annex 1: Evaluation Statement of Work ..............................................................................................................47 Annex II: Data Collection Instruments ..................................................................................................................60 Annex III: Evaluation Design Matrix ........................................................................................................................70 Annex IV: Sources of Information ...........................................................................................................................75 Documents Reviewed............................................................................................................................................75 Informants Interviewed .........................................................................................................................................81 Annex V: Disclosure of Any Conflicts of Interest ...............................................................................................89 vi US CTI Evaluation Report FIGURES AND TABLES Figures Figure 1: CTI-CFF Implementation Area....................................................................................................................10 Figure 2: Simplified US CTI Results Framework ......................................................................................................11 Figure 3: US CTI Evaluation Team Travel [Blue signifies Sub-Team A; Green signifies Sub-Team B; Red signifies team that traveled to the Regional Priorities Conference] ..........................................................13 Figure 4: Survey Respondents by Sector ....................................................................................................................17 Figure 5: Survey Respondent Partnerships ................................................................................................................17 Figure 6: Survey Respondent Involvement ................................................................................................................18 Figure 7: The establishment and enhancement of regional and national platforms receive a higher effectiveness rating from survey informants than ratings for EAFM, MPA, CCA and the overall Strategic Objective. ................................................................................................................................................22 Figure 8: Percentage of respondents by country who feel the regional approach contributed to achievement of program results .........................................................................................................................22 Figure 9: Final Working Draft of the US CTI Support Program Consolidated Results Framework, Program Areas and Indicators .............................................................................................................................27 Figure 10: How a unified US CTI Support Program could contribute to the CTI Vision ..............................28 Tables Table 1: Sites Visited During US CTI Performance Evaluation .............................................................................14 Table 2: Number of Key Informants By Country ....................................................................................................15 Table 3: Sources of Survey Contacts ..........................................................................................................................16 Table 4: Survey Respondent Information ...................................................................................................................16 US CTI Evaluation Report vii ACRONYMS APEC Asia Pacific Economic Cooperation ASEAN Association of Southeast Asian Nations CA Cooperative Agreement CCA Climate Change Adaptation CT Coral Triangle CT6 The Six Coral Triangle Countries CTC Coral Triangle Center CTI-CFF Coral Triangle Initiative on Corel Reefs, Fisheries and Food Security CTMPAS Coral Triangle Marine Protected Area System CTSP Coral Triangle Support Partnership E3 USAID Bureau for Economic Growth, Education and the Environment EAFM Ecosystems Approach to Fisheries Management GMCA Gizo Marine Conservation Area Management Committee IFADs Inshore Fishery Aggregating Devices IMACS Indonesia Marine and Climate Support Project IR Intermediate Results IRS Interim Regional Secretariat IUU Illegal, Unreported and Unregulated (fishing) KII Key Informant Interview LEAP Local Early Action Plan (Climate Change) LRFFT Live Reef Food Fish Trade M&E Monitoring and Evaluation MEAT Marine Protected Area Effectiveness Assessment Tools MoU Memorandum of Understanding MPA Marine Protected Areas MSI Management Systems International NCC National Coordinating Committee NOAA National Oceanic and Atmospheric Administration NPOA National Plans of Action PI Program Integrator PMP Performance Management Plan PNG CLMA PNG Center for Locally Managed Areas PNG Papua New Guinea PPP Public-Private Partnerships RDMA Regional Development Mission for Asia USAID REAP Region-wide Early Action Plan viii US CTI Evaluation Report REAP-CCA Region-wide Early Action Plan for Climate Change Adaptation for the Nearshore Marine and Coastal Environment and Small Island Ecosystems REO Regional Environment Office, USAID/RDMA REX Regional Exchanges RF Results Framework RHM Reef Health Monitoring RIG Regional Inspector General RPO Regional Program Office RPOA Regional Plan of Actions RS Regional Secretariat SDS-SEA Sustainable Development Strategy for the Seas of East Asia SEAFDEC Southeast Asian Fisheries Development Center SI Social Impact, Inc. SO Strategic Objective SOW Statement of Work TA Technical Assistance TNC The Nature Conservancy TT Tetra Tech ARD TWG Technical Working Group UPMSI University of the Philippines Marine Science Institute US CTI U.S. Coral Triangle Initiative Support Program USAID U.S. Agency for International Development USDOI U.S. Department of the Interior USG U.S. Government WWF World Wildlife Fund US CTI Evaluation Report 1 EXECUTIVE SUMMARY EVALUATION PURPOSE AND EVALUATION QUESTIONS This document reports on the evaluation of the performance of the United States Coral Triangle Initiative (US CTI) program, as well as on the validity of the foundational assumptions inherent in the US CTI Results Framework (RF). Specifically, the evaluation is designed to investigate the effectiveness of the program’s key elements—the Results and Intermediate Results—in achieving the Strategic Objective (SO) of the RF and the Coral Triangle Initiative on Coral Reefs, Fisheries and Food Security (CTI-CFF) Goals. The results of the evaluation will be used primarily by the United States Agency for International Development (USAID) Bureau for Economic Growth, Education and Environment (E3), Asia regional and bilateral missions and other U.S. government (USG) agencies such as the National Oceanic and Atmospheric Administration (NOAA), the U.S. Department of the Interior (USDOI), CTI-CFF Stakeholders and CTI-CFF Development Partners to inform the development of ongoing and future support to help accomplish CTI-CFF goals and targets. A team of experts from Social Impact, Inc. (SI) and Management Systems International (MSI) was formed with limited participation from staff members of the USAID Regional Development Mission for Asia (RDMA) itself. The evaluation statement of work (SOW) designed by RDMA (see Annex 1) set out five questions. The primary question was: How effective has the US CTI’s regional collaboration and integration approach been in achieving its Strategic Objective of “improving management of biologically and economically important coastal and marine resources in the Coral Triangle” as applied in the US CTI Results Framework? The evaluation was also mandated to address Secondary Questions related to conceptual and design aspects, as well as performance aspects, including gender equality issues. These questions included:  What specific lessons have been learned about the validity of the program’s development hypothesis and results frameworks?  How effective have US CTI actions under Intermediate Results been in achieving corresponding Results?  How effective have US CTI results been in contributing to the accomplishment of corresponding CTI￾CFF Goals, strategic objectives and targets and the US CTI Strategic Objective?  What key challenges, either technical or managerial, have been encountered and innovative practices applied in achieving the US CTI Strategic Objective, thereby contributing to the CTI-CFF Goals? PROGRAM BACKGROUND The Coral Triangle (CT) is an area encompassing almost four million square miles of ocean and coastal waters in the Southeast Asian and Pacific nations of Indonesia, Malaysia, Papua New Guinea, the Philippines, the Solomon Islands and Timor-Leste, called the Coral Triangle 6 (CT6). The area is home to some 363 million people and the biological resources of the CT directly sustain more than 120 million people living within this area and benefit millions more worldwide. The CT is at risk due to a range of factors, including overfishing, destructive fishing practices, land- and ocean-based sources of 2 US CTI Evaluation Report pollution and climate change. These have a negative impact on food security, employment opportunities and living standards of people dependent on the resources. Further, it is the poor who are most dependent on the threatened marine resources. Each of the six countries has its own local resource management problems and responsibilities. However, environmental problems also have regional international political and economic aspects— because each of the CT6 countries has an economic zone in which it has exclusive rights to marine resources, but those fish and marine resources are not confined to any one zone in breadth or over time. As far back as the year 2000, the international NGOs working in the CT area realized that there could be value in creating a joint regional program to address these problems and began to work with the CT6 countries to realize this vision. In 2007 President Yudhoyono of Indonesia officially launched the process. In 2008 the US CTI Program of Support was begun by USAID, and in 2009 the CT6 countries formed the regional CTI-CFF. The US CTI program helped bring into existence the CTI-CFF organization among the six countries and continues to support it. The CTI-CFF is intended to be a formally ratified treaty organization that sets policies and carries out actions regarding marine and coastal resources through a formal structure, including a Regional Secretariat (RS). At present, the process of official ratification of the RS by the countries continues and so there are interim structures. The overall goal of the US CTI is to improve the management of biologically and economically important coastal-marine resources and associated ecosystems that support the livelihoods of peoples and economies in the CT. The US CTI program works at multiple levels but is organized as a regional program with an intentionally regional impact. Activities promote information exchange, galvanize international cooperation to address transnational or regional issues and share lessons among CT6 countries. These were intended to integrate with national activities to support policy reform and institutional capacity development that provide foundations for effective regional cooperation and with local activities designed to demonstrate best practices in priority geographies and specific sites defined as areas where biological and socioeconomic factors are of such importance that they merit focused management attention. Similar sets of activities in priority geographical areas would roll up to national and regional activities that generate regional results. The USAID RDMA funds and oversees US CTI, provides overall program direction and works closely with bilateral USAID Missions in the region. To implement this program RDMA uses three mechanisms:  The Coral Triangle Support Partnership (CTSP), a five-year cooperative agreement awarded to the World Wildlife Fund (WWF), which serves as the prime for an NGO consortium that includes The Nature Conservancy (TNC) and Conservation International (CI);  The Program Integrator (PI) contract, led by Tetra Tech/Associates in Rural Development (TT), which provides regional support, technical assistance and small grants and supports the U.S. program team and the CTI-CFF Interim RS; and  The U.S. NOAA Inter-Agency Agreement, which provides capacity building, technical assistance and training to support CTI goals. EVALUATION DESIGN, METHODS AND LIMITATIONS To address the evaluation purpose and questions, the Evaluation SOW (see Annex 1) mandated a triangulated mixed methods performance evaluation, drawing on both quantitative and qualitative data from document reviews, individual and group key informant interviews (KII), site visits and a survey. The majority of information for the evaluation was derived from KIIs, which consisted of both individual and group semi-structured interviews—more than 100 in total with approximately 200 people in the region US CTI Evaluation Report 3 and the United States. Given the restricted field schedule in the SOW, which limited the amount of time in each country to approximately one week, an electronic survey was also developed to increase the number and type of respondents. The celebration of Ramadan complicated interview availability in Indonesia, Malaysia and Timor-Leste, and communications were limited in Papua New Guinea (PNG) and Timor-Leste outside of Dili. Despite this, representatives were interviewed from all organizations identified and prioritized at the inception of fieldwork plus others from additional organizations that were identified over the course of the evaluation. MOST IMPORTANT FINDINGS  What may be transformative in US CTI is the creation of a regional system of ocean governance that incorporates the four results areas, which shows potential for a collaborative mechanism that motivates the CT6 countries to work together despite differences in capacity and resources, and could be effective given the opportunity (would take more than five years) and resources (of which relying on only donor money would be insufficient).  Of the four thematic areas in the RF, the regional/national platform work was the most effective at achieving more tangible results versus the other three areas (Ecosystems Approach to Fisheries Management [EAFM], Marine Protected Areas [MPA] and Climate Change Adaptation [CCA]) in the RF.  Achievement of the US CTI RF Intermediate Results (IRs) that call for “policies developed,” “constituencies engaged,” “networks strengthened,” “capacity increased” and similar was generally high. Those IRs that called for more complete actions such as “sustainable financing mobilized,” “EAFM applied,” “MPA effectiveness improved” and “climate adaptation strategies applied” were generally at a more incipient stage.  While the CTI-CFF has begun to address some of the most important threats in the CT with US CTI support, decisions were also made not to address some important issues. The USG decided that US CTI would not address tuna fishing, for example, as this is considered a “trade” issue and better handled by other USG organs. Many respondents brought up the fact that the CTI-CFF was not yet addressing some of these larger issues and questioned whether such larger issues might now or even ultimately be “too big for CTI-CFF to handle” now— because of its stage of development, and later because it only includes these six countries— and whether they would be more appropriately addressed by other already existing regional bodies such as Asia Pacific Economic Cooperation (APEC), Association of Southeast Asian Nations (ASEAN) or others. Among the CTI-CFF principles in the RPOA (#4, #5) are to work with the existing bodies and the decision to agree with the Southeast Asian Fisheries Development Center (SEAFDEC) on their leadership in Live Reef Food Fish Trade (LRFFT) was mentioned by several as possible recognition of this.  The long-term value of the significant achievements is somewhat tempered by the incomplete stage of the work/effort. The fact that the CTI-CFF depends largely on the whole constellation of US CTI actions including policies, frameworks, Regional Exchanges (REX), Technical Working Groups (TWGs) and training means that the CTI-CFF is not yet sustainable. The RS is not yet ratified, though most stakeholders consider it likely at some point, and, though most of the CT6 countries have committed financial support, the non￾operationalization of the RS means that the funds are not yet actually applied. The exception to this is Indonesia, which has been supporting the Interim RS located in its Fisheries Ministry. Countries do provide substantial financial support for their programs and for some regional activities such as meetings. The future of the CTI-CFF remains uncertain as the USG, through 4 US CTI Evaluation Report the US CTI, has been the principal donor, indeed, as everyone acknowledges, the cause of its existence.  Gender was not considered a primary matter in the original US CTI program design. There was no explicit mandate for gender mainstreaming in the original program design or in subsequent re-design. Direction from program officials was to incorporate gender into activities, and there are specific instances of success working with women’s groups, for example.  There was little vertical integration through the three levels of regional, national and local; these could have been two separate programs (regional and national/local). The greatest disconnect has been at the national level where the National Coordinating Committees (NCCs) were conceived to pass regional plans to local. In a number of ways, the desired vertical integration involving all three levels was not achieved in part as regional level exchanges were attended by higher-level government officials often not sufficiently connected to the ground. What was lacking was a coordinating mechanism to link national to local implementation, involving local and other subnational government and non-government institutions (not just CTSP NGOs).  The NCCs in each country were designed to be responsible for all five CTI-CFF Goals, but in practice most operated primarily in response to the three goals addressed by the US CTI, and there was significant variation in ability to function.  Generally, NGO implementers used this as a program to continue their work at sites— though improved with the use of regional studies, guidelines, frameworks and tools and with some knowledge exchange. This brings with it some comparative advantages in terms of relevant prior experience and a base from which to achieve on-the-ground results in a shorter timeframe but also can limit the ability to implement new types of activities. The USCTI was instrumental in promoting in CTI-CFF the concept of horizontal integration across sectors. Beyond the conceptual, and in an incipient stage of implementation, there are examples of an expanded MPA scope to cover climate change and fisheries goals, and expanded MPA planning and implementation but not of horizontally integrated programs across six countries.  Despite the stated SO of the US CTI program, a common logic and vision were not well developed and communicated across program implementers and stakeholders. The level of familiarity with the purpose or SO of the US CTI and with the entire RF decreased markedly the closer the stakeholder was to the ground and further from program management as, for instance, in villages or on sites. This was expected, but unexpected was the degree of unfamiliarity even among those more responsible for program management. The RF was unfamiliar beyond the fact that it covered the four areas, much as in the simple diagram (see Figure 2), and was considered more suited to communicating the program’s broad intention than of serving as the detailed organizing framework. As an innovative regional approach reaching down to all levels, a logical concept of the CTI and what it represents may be equally important for local implementers and the community, as for regional and national players.  From the beginning, the program sustainability strategy for the CTI-CFF depended on the ratification of the RS and the maturing of the CTI-CFF supported by the CT6 which has not yet happened. While there was an “endgame” or close-out strategy for the US CTI, this does not represent a strategy for sustaining activities such as the NGO site work. US CTI reports that large funding streams are already lined up in most all countries and for most all sites. Other sustainability strategies, such as mainstreaming CTI-CFF in national and subnational US CTI Evaluation Report 5 development plans and institutionalizing coordinating mechanisms (vertical and horizontal) independent of external funding were also not observed.  With respect to sustainability in the field, development of alternative or sustainable livelihoods was an important area where individual successes were noted, but without a policy for the identification, selection and development of alternative enterprise opportunities.  Early lack of clarity on roles and responsibilities among implementers of the different US CTI mechanisms generated initial confusion among partners which, although lessened over time, generated inefficiencies in coordination.  There was a slow program start-up and inefficiencies in key activities such as work planning. Relationships, communication and coordination among consortium members have become more efficient over time as the internal capacities of country teams improved. The PI and CTSP made robust efforts to minimize the impact of the challenges on implementation. There were also changes in the emphasis required by RDMA on matters such as the degree of regionality versus country programs in early years.  There was near-universal appreciation for the quality of training and other technical work of NOAA and as valuable expert members of TWGs. For example, NOAA worked in or near (the proposed) Tun Mustapha Park in Malaysia and on the north shore of Timor-Leste in inventorying fish stocks as a baseline. Some respondents in Timor-Leste however questioned how much local capacity was built for sustainability and setting up and using a system for data management. In general, there was little understanding among participants of the way in which the NOAA work was integrated into the rest of the US CTI program. MOST IMPORTANT CONCLUSIONS  On the whole, evidence shows—and stakeholders agree—that the US CTI was effective at supporting establishment of a new regional paradigm regarding coastal and marine resources in the CT6 countries. The six countries had never before cooperated on these matters, at this scale, or in this depth. The common policies, frameworks and guidelines developed were invaluable, and the support for the REX meant that potential colleagues in the six countries could realize this potentiality and begin to collaborate.  Demands for CTI-CFF consolidation may be suffering from unrealistic expectations about what is possible in five years since most formal international organizations have taken many more years to be consolidated.  At a conceptual level, the program was to have implemented all three thematic areas (EAFM, MPA, CCA) in each of the sites in each of the six countries. For this they were called integration sites or demonstration sites. However, in practice, the site work tended to be in only one thematic area—primarily in MPA, which is the best-understood and best-developed area of knowledge. The reason for this may be the local capacity, experience, funding and time to develop, but it is also the thematic area with which the implementing NGOs are most familiar.  The three programs, or work streams, of EAFM, MPA and CCA have each proven to be effective to some degree in achieving the related Regional Plan of Action (RPOA) goals. Each of these is at different stages on the ground but with completion of the EBM integration tool kit, integration of the three is feasible in the future. As work on these is at different stages on the ground, their integration is yet further away, and the effectiveness of an integrated 6 US CTI Evaluation Report approach as proposed to be carried out through the US CTI, as yet, neither proven nor disproven.  US CTI actions toward achievement of MPA effectiveness has been positive through supporting the development and endorsement of the Coral Triangle Marine Protected Area System (CTMPAS) framework, the support for the development CT Atlas and the development and adoption of the MEAT. These and training have contributed in improving management of the MPAs. Thus the MPA work also contributed to meeting the relevant CTI￾CFF RPOA goal.  While there were no specific pro-gender policies in the original US CTI design, specific US CTI actors did implement pro-active gender action including the PI and Regional Program Office (RPO) both in their own hiring and in directions for inclusiveness in regional exchanges. Noted is the presence of women in leadership of the CT6 NCCs in the recent Manado Regional Priorities Conference.  When USAID/RDMA is not able to assume the internal coordination function, ensuring clear roles, responsibilities and relationships from the beginning and actively managing those relationships can help ensure productive coordination. (See Recommendations below.) PRIMARY RECOMMENDATIONS  USAID should urgently identify, develop and implement an interim mechanism for continuation of certain functions performed by the PI including, specifically, the CTI-CFF donor coordination function. USAID should urgently identify, develop and implement an interim continuation program for certain activities of the NGO-led program so as not to lose the gains already achieved on the ground. USAID should immediately inform people in the field about these plans for continuation.  USAID should support a time-limited project that focuses on the CTI-CFF as an organization, providing transitional assistance to the interim RS and to the permanent RS when created.  A follow-on program should include direct technical assistance to the NCCs to help them become viable nationally owned mechanisms that implement and oversee their national programs.  The new program should support exchanges of technical and experiential information among the country programs and sites to capitalize on the value of networking (link to country programs), TWGs, national and regional learning networks, etc.  NOAA should be more fully engaged in country and regional work plans and the training and services it provides should be explicitly tied to country and site needs and better communicated. NOAA should also consider using more local/regional expertise in providing its training.  USAID should implement fieldwork in the CTI countries through the bilateral Missions to carry out complementary programs rather than through RDMA. RDMA should support the coordination of these programs.  In any subsequent program, USAID should allow a broader range of interventions and result areas in these country programs, including alternative or sustainable livelihoods according to local needs and designs. Programs should be periodically re-analyzed and emerging priorities or needs addressed and resources reallocated to these. US CTI Evaluation Report 7  For country programs, USAID should consider a wider range of vehicles/mechanisms beyond a consortium of international NGOs (as structured in US CTI). Such other mechanisms could include contracts, agreements with NGOs not in a consortium or with a consortium with clearer structure.  In a subsequent program, USAID should ensure a sound development hypothesis, a clear and logical RF, a comprehensive Monitoring and Evaluation (M&E) Plan and an efficient system for work planning, budgeting and coordination. USAID should also ensure that projects have and devote a sufficient share of resources, including personnel, to carry this out. 8 US CTI Evaluation Report I. EVALUATION PURPOSE & EVALUATION QUESTIONS EVALUATION PURPOSE The purpose of this evaluation of the US CTI program is to examine the performance of the program, as well as test the foundational assumptions inherent in the US CTI RF. Specifically, the evaluation is meant to investigate the effectiveness of the program’s key elements—the Results and Intermediate Results— in achieving the SO of the Results Framework and the CTI-CFF Goals. The evaluation’s findings, conclusions and recommendations provide insights to guide the development of future USAID programs and the design of effective results frameworks to maximize development outcomes. The results of the evaluation will be used primarily by USAID/E3, Asia regional and bilateral missions and other USG agencies such as NOAA and USDOI, CTI-CFF stakeholders and CTI-CFF Development Partners to inform the development of ongoing and future support to help accomplish CTI-CFF Goals and targets. During the evaluation In-Brief with the RDMA, greater interest was shown in obtaining stakeholder views on future coastal marine–focused programs in the region. The Evaluation Team included questions to elicit this information while recognizing and making clear to respondents that this evaluation was not an effort to actually design any specific future programs, and that the evaluation would be contributory to RDMA thinking but not determinative. Respondents were interviewed on regional and local priority issues or problems and potential solutions with or without a USG role, regional actions and multiple country engagement among the CT6. EVALUATION QUESTIONS The evaluation SOW set out as the primary question: 1. How effective has the US CTI’s regional collaboration and integration approach been in achieving its Strategic Objective of “improving management of biologically and economically important coastal and marine resources in the Coral Triangle” as applied in the US CTI Results Framework? (Approach to Concept) The evaluation was also mandated to address the following secondary questions related to conceptual and design aspects of US CTI, as well as performance aspects, including gender equality issues: 2. What specific lessons have been learned about the validity of the program’s development hypothesis and results frameworks? (Effective Program Design) 3. How effective have US CTI actions under Intermediate Results been in achieving corresponding Results? (Effective Implementation) 4. How effective have US CTI results been in contributing to the accomplishment of corresponding CTI-CFF Goals, strategic objectives and targets and the US CTI Strategic Objective? (Program Relationship with CTI-CFF) 5. What key challenges, either technical or managerial, have been encountered and innovative practices applied in achieving the US CTI Strategic Objective, thereby contributing to the CTI￾CFF Goals? (Lessons Learned and Future) US CTI Evaluation Report 9 Each of the five questions was explored in the interviews through a series of follow-up questions that clarified and deepened the discussion. For example, the first question centers on the approach used to achieve the SO, and follow-up questions were employed to assess the regionality, integrative aspects and effectiveness of the approach. All follow-up questions are included in the Interview Protocols in Annex II. Gender While there was no specific question related to gender in the evaluation SOW, the Evaluation Team was instructed to “use these questions to help examine the gender equality issue in the program design and implementation.” Attention was given to ascertaining whether gender was taken into consideration in the design and implementation of the US CTI program’s activities, and in particular, whether gender equality was among the principles applied in the design of the CTSP or NGO country programs. To determine this, the Evaluation Team designed its data collection instruments and procedures to proactively obtain inputs from males and females, especially in social or economic situations where female voices are less easily heard, and employed culturally sensitive and appropriate survey techniques to obtain this part of the spectrum of input. Further, the question of whether or not information fully represents the different effects and opinions of both genders was taken into consideration in the analysis of the data. 10 US CTI Evaluation Report II. PROGRAM BACKGROUND PROGRAM OBJECTIVE The regional CTI-CFF brings together the six countries of Indonesia, Malaysia, PNG, the Philippines, the Solomon Islands and Timor-Leste to preserve the threatened marine, coastal and small island ecosystems of the CT region (see Figure 1). The CTI-CFF is intended to be a formally ratified treaty organization that sets policies and carries out certain actions regarding marine and coastal resources through a formal structure including an RS. The US CTI supports the CTI-CFF and, in fact, in these formative years of the CTI-CFF, the U.S. support has been more than fundamental. The US CTI activities have been practically synonymous with the CTI-CFF. The overall goal of the US CTI is to improve the management of biologically and economically important coastal-marine resources and associated ecosystems that support the livelihoods of peoples and economies in the CT. USAID/RDMA funds and oversees US CTI, provides overall program direction and works closely with bilateral USAID Missions in the region. Figure 1: CTI-CFF Implementation Area The CTI-CFF Regional Plan of Action has five goals relating to: 1) Seascapes; 2) Ecosystem Approach to Fisheries Management; 3) Marine Protected Areas; 4) Climate Change Adaptation; and 5) Threatened Species. The simplified version of the US CTI RF (see Figure 2) shows four principal areas of US CTI actions and their congruence with three of the regional goals of the CTI-CFF. The framework shows that management improvement, capacity improvement, regional collaboration and integration of measures across program areas of EAFM, MPA and CCA will result in the overall improved management of all resources in the region. US CTI Evaluation Report 11 Figure 2: Simplified US CTI Results Framework The congruence of the US CTI EAFM, MPA and CCA goals with the corresponding CTI-CFF goals 2–4 is unsurprising since the two (US CTI and CTI-CFF) were essentially being planned at the same time, by the same people and organizations, and were intended to be similar, at least in the areas in which both operated. For each of these areas there is a further specific subordinate RF explained further in Section IV, Evaluation Question Two, which also presents the complete and detailed RF. PROGRAM STRUCTURE The US CTI is implemented through three mechanisms under the RDMA but without a hierarchy among the mechanisms: 1. Cooperative Agreement: The CTSP, a five-year cooperative agreement awarded to the WWF, which serves as the prime for an NGO consortium that includes TNC and CI. The CTSP implements activities that align with the National Plan of Action (NPOA) through one or more of the NGOs in each of the six CTI-CFF countries; 2. Contract: TT serves as a PI, providing regional support, technical assistance and small grants and supporting the U.S. program team and the CTI-CFF Interim RS; 3. Inter-Agency Agreement: NOAA provides capacity building, technical assistance and training to support CTI goals. PROGRAM IMPLEMENTATION STATUS US CTI is in its fifth and final year of implementation. It is expected that there will be short no-cost extensions to some parts of the program beyond the 30 September 2013 contract end date. These are not new activities or vehicles, and the program as a whole will not be extended. The NOAA program will continue through 2014, and RDMA will continue activities into 2014 through an agreement with the USDOI in a program outside of the focus of this evaluation. The Indonesia Mission marine program (Marine Protected Area Governance and Indonesia Marine and Climate Support Project [IMACS]), which is related to and is an offspring in the same family as the US CTI, also continues through 2014. 12 US CTI Evaluation Report III. EVALUATION METHODS & LIMITATIONS The US CTI Evaluation Team consisted of four experts from SI and MSI, who were supported by staff members from RDMA who participated for limited times. The team included:  Team Leader: Mr. Doug Baker  Deputy Team Leader: Dr. Miriam Balgos  Evaluation Specialist: Mr. Mike Duthie  Local Evaluation Specialist: Dr. Ninnette Lasola  USAID/RDMA Natural Resources Management Officer: Ms. Danielle Tedesco  USAID/RDMA Monitoring and Evaluation Corp: Mr. Samuel Turano EVALUATION DESIGN USAID set the design of the evaluation as a mixed methods performance evaluation, drawing on both quantitative and qualitative data from a document review, individual and group KIIs, site visits and a survey. By using multiple methods of data collection drawing on both quantitative and qualitative data, the evaluation was able to limit bias inherent in any individual data collection method. Baseline data was not available, so analysis of change over time relied on recall and respondent perception. To mitigate recall and response bias, multiple data sources and methods were used to triangulate results. Specific data collection approaches and key sub-evaluation questions were identified for each evaluation question and laid out in an Evaluation Design Matrix (Annex III). The US CTI program covers various dimensions and units of analysis, including geographic level (regional, national and site), country (each of the CT6 countries), result area (regional/national platforms, EAFM, MPA and CCA), mechanism (CTSP, PI and NOAA) and consortium member (WWF, TNC and CI). While some questions focus on specific dimensions and units (e.g., Question 1 focuses on the regional level), the evaluation generally attempts to aggregate across dimensions in its conclusions, but significant variations across units are noted. Given the evaluation’s emphasis on looking forward, the Evaluation Team deliberately included respondents with less direct engagement with US CTI, but substantial knowledge and involvement in the CTI-CFF or key result areas, to identify additional insights into possible new or different approaches to achieving the US CTI objectives. DATA COLLECTION METHODS The US CTI evaluation began in June 2013 with the preparation of the Evaluation Work Plan. The Team convened in Bangkok in mid-July for one full week to meet with RDMA and to present the Inception Report and In-Brief before deploying to the field for data collection. The PI maintains its office in Bangkok, so the Evaluation Team consulted with the PI team during the first week in Bangkok. US CTI Evaluation Report 13 Figure 3: US CTI Evaluation Team Travel [Blue signifies Sub-Team A; Green signifies Sub-Team B; Red signifies team that traveled to the Regional Priorities Conference] As designed in the SOW, in order to visit the maximum number of countries the team broke into two sub-teams during data collection (see Figure 3 for a map of their travels). In each of the countries, the sub-team consulted with the relevant ministries or other government officials as well as the relevant NGO staff. Additionally, one or more site visits were made to the locations of field activities in each CT country (except for PNG). The sub-teams devoted approximately one week in each country to allow for the sometimes significant travel from capital cities to outer regions. The Team then reconvened in Bangkok for one week of analysis and preparation and presentation of the Out-Brief to RDMA. Finally, three members of the Team then attended the Second Regional Priorities Conference in Manado, Indonesia, on August 20-22. At the workshop the Team continued their work (analysis and report writing) and conducted additional interviews with officials from sites or programs with which they had been unable to meet because of logistical or access difficulties. This was also an opportunity to engage USAID in more extended discussion of conclusions and recommendations. The data collection methods used in the evaluation include: Document Review: The document review provided critical background and contextual information and fed directly into the development of instruments for other data collection methods and provided a solid base of understanding for the Team before going to the field. The Evaluation Team obtained 230 documents electronically from the PI and CTSP, which were grouped into background documents; activity reports; task orders; work plans; action plans/strategies; meeting minutes and associated documents (including decisions); progress/annual reports; evaluation reports; and others (see Annex IV for full list). Each document was systematically reviewed by the Evaluation Team using a protocol to prioritize, review, analyze and summarize. Additional documents were received and similarly analyzed throughout the evaluation implementation. Key Informant Interviews: The majority of data for the evaluation was derived from KIIs, which consisted of both individual and group semi-structured interviews—more than 100 interviews in total with approximately 200 people, including 85 women representing 44 percent of the respondents (see 14 US CTI Evaluation Report Table 2). Based on the Evaluation Design Matrix (Annex III) and sub-questions, a specific interview protocol, which included both open- and close-ended questions, was developed reflecting each major evaluation question (see Annex III). Given the broad range of stakeholder types and their experience with US CTI, three variations of the interview protocol were developed to focus on questions most relevant to regional/national level stakeholders, site level stakeholders (e.g., sub-national government or implementer staff in field offices) and program beneficiaries (e.g., community members in areas near MPAs). These protocols were used as guides and were typically implemented with filters to skip unrelated questions and allow for opportunities to probe important or unclear responses. Nearly all interviews were conducted in person, with the exception of approximately 10 respondents who were interviewed by phone due to logistical conflicts or the fact that they operate out of Hawaii, for example. Details on sampling and responses are included below. Electronic Survey: Given the restricted field schedule in the SOW, which limited the amount of time in each country to approximately a week and prohibited visiting all program implementation sites, an electronic survey was developed to increase the number and type of respondents providing information to the Evaluation Team. The survey consisted of primarily close-ended questions but also included four open-ended questions for respondents to provide more detailed or nuanced responses (see Annex II for the survey protocol). The survey was kept brief to maximize responses and focused more directly on respondent perceptions related to the evaluation questions. It also included limited demographic information, including sex, age, sector and location, to permit disaggregated analysis. The survey was implemented through SurveyMonkey and invitations were sent out to potential respondents during the initial stages of fieldwork so that the Evaluation Team could encourage responses during site visits. The survey was left open for approximately four weeks, through the end of fieldwork. Site Visits: The Evaluation Team conducted semi-structured observations at priority geographic sites in five of the six program countries to observe program implementation and effects (see Table 1). A site visit in PNG was not possible due to time and security constraints. During site observations, the Team visited assisted communities, gazetted or planned MPAs, marine ranger posts and park stations to identify evidence of program implementation and outcomes. The Team included experts with extensive field experience in Southeast Asia as well as other countries with similar conditions, which provided an expert base for comparison and judgment. Table 1: Sites Visited During US CTI Performance Evaluation Country Site Visited Indonesia Nusa Penida MPA Malaysia Tun Mustapha (proposed NP) Papua New Guinea N/A Philippines Taytay Solomon Islands Gizo Timor-Leste Nino Konis Santana NP US CTI Evaluation Report 15 KII SAMPLING To identify KII respondents, the Evaluation Team first relied on a list of individuals who had been involved in US CTI, provided by the PI. The PI and CTSP identified between 7–16 priority respondents in each country based on their involvement with US CTI activities. The Evaluation Team used this as a starting point, adding others to the sample, both from the list and from recommendations of respondents, a method known as snowball sampling. The Team interviewed 192 individuals. (See Table 2 for details.) Table 2: Number of Key Informants By Country Country Number of Informants Males Females Indonesia 29 21 9 Malaysia 27 12 15 Papua New Guinea 17 4 13 Philippines 37 21 16 Solomon Islands 29 17 12 Thailand 12 6 6 Timor-Leste 23 20 4 United States 16 6 10 TOTAL 192 107 85 Key informants included representatives from USAID, each program implementer (at their headquarters and staff in regional, national and field offices where applicable), international agencies (such as the Asian Development Bank [ADB]), national and sub-national government, national and sub-national ministry or agency staff, local NGOs, private sector and other beneficiaries at the site level. Beneficiary respondents included local fishing populations, community leaders, merchants, tourism operators and park/MPA rangers. Annex IV provides a full list of informants. SURVEY SAMPLING The electronic survey was sent to all stakeholders and participants identified by the PI, which included 579 unique email addresses. Table 3 provides a breakdown for the contact sources. Throughout fieldwork, and through suggestions provided by survey respondents, 45 additional email contacts were identified. Since many email addresses were incorrect, this corresponded to 514 valid contacts, from which 205 survey responses were recorded. Of these, 15 did not respond to any of the substantive questions, leaving a total sample of 190. 16 US CTI Evaluation Report Table 3: Sources of Survey Contacts Source No. of Contacts Program Partners 152 Indonesia 104 Malaysia 122 Papua New Guinea 27 Philippines 101 Solomon Islands 42 Timor-Leste 31 TOTAL 579 As shown in Table 4, the survey respondents were highly educated: 79 percent held a graduate degree and only 3 respondents had less than an undergraduate degree. Although most respondents (83 percent) spent less than 50 percent of their time on US CTI tasks, they had on average nearly 3 years of involvement with US CTI, and more than 80 percent rated themselves as having at least moderate knowledge of US CTI. There is a statistically significant positive correlation between self-rated knowledge of US CTI and both length of involvement (p=0.001) and percentage of work related to the program (p<0.001), but no correlation between knowledge and sex, age or education. Table 4: Survey Respondent Information Variable Result Response Rate 37% Number of Responses 190 % Female 41% % With Graduate Degree 79% Mean Years Involved with CTI 2.97 % Spending <50% time on CTI 83% % who received US CTI funding 44% US CTI Evaluation Report 17 Figure 4: Survey Respondents by Sector As shown in Figure 4, survey respondents represent all contacted sectors, with most representation from national government, which includes national level ministry staff, followed by NGOs. The majority of respondents (72 percent) reported working with CTSP, while only about 20 percent had worked with NOAA and 22 percent had worked with the PI, as shown in Figure 5. Figure 5: Survey Respondent Partnerships 18 US CTI Evaluation Report Respondents represent each of the CT6 countries. However, 76 percent come from Indonesia, Malaysia or the Philippines, the most populous countries, while only 14 percent come from PNG, the Solomon Islands or Timor-Leste, but on a per capita basis (per 100,000) the response rate was greater for the smaller countries. Figure 6 shows that respondents had been involved at fairly similar rates across the main program activities, with the exception of support to the RS. Figure 6: Survey Respondent Involvement Integration of Gender into Data Analysis In light of the critical livelihood issues addressed by the program and the important influence these have on gender equity, norms, roles and dynamics, the evaluation included questions in the interview protocols and document review related to how gender was integrated into program design and implementation. The Evaluation Team also recorded the sex of all interview and survey respondents to be able to analyze whether men and women had different perspectives on program implementation and effectiveness. Moreover, the Evaluation Team sought to include both men and women as respondents at all levels and received survey responses from 77 women (41 percent of the total sample). In analysis, the Team specifically sought to determine if there were differing effects for men and women. LIMITATIONS AND RISKS Attribution and Generalizability The most significant limitation in the evaluation relates to attribution. In the absence of baseline data or a valid comparison group at each program level, findings and conclusions related to program effectiveness rely almost exclusively on stakeholder perceptions. This is further complicated by the fact that CTSP consortium-member NGOs had been working in most program sites for years prior to initiating the US CTI program. Most also had additional non–US CTI funds supporting work in the same areas (a requirement of the award). This limitation was understood from the initiation of the evaluation, and the evaluation questions were accordingly focused less on attribution and more on implementation and the program approach. US CTI Evaluation Report 19 In addition, while the Evaluation Team sought to collect data from as many respondents and from as many locations and stakeholders as possible, due to time constraints, the Evaluation Team was only able to reach approximately 200 respondents through interviews and only able to visit field sites in five of the six countries. Given that much of the data collected in this evaluation was qualitative and collected from a convenience sample, the Evaluation Team is somewhat limited in its ability to generalize findings beyond direct respondents, although these respondents represent nearly all prioritized stakeholders. Selection Bias The electronic survey is intended to help mitigate the issue of generalizability, allowing for data collection from a wider group, yet there is still evidence of some selection bias given the very high education level and predominance of respondents from Southeast Asia in the survey sample. The snowball sampling technique was used to minimize the potential for selection bias, including the possibility that the implementers may have, even unintentionally, directed the Evaluation Team to the most favorable respondents, and in practice, the Evaluation Team found that the vast majority of possible respondents identified through the snowball sampling technique were already on the prioritized sample list. Moreover, the Evaluation Team did not identify any marked differences in responses from the initial and snowball sample respondents. Response Bias A third potential limitation relates to response bias that may occur if respondents think that responding in a certain way may lead to additional funding, or hesitate or self-censor in interviews in which USAID staff participate. Through source triangulation, the Evaluation Team minimized this limitation, and found, in practice, that interviews with USAID staff present were not substantively different in the type of information presented. In fact, it was during interviews with USAID participation that some of the most unexpected and even negative/critical results were revealed. A related type of interviewer or response bias could occur if questions are asked in a way that leads respondents to certain responses. Interview protocols and survey questions and response options were designed to be as neutral as possible to mitigate this concern. Logistical Constraints An additional time constraint related to holiday schedules. Ramadan occurred during fieldwork, making some respondents difficult to reach, although the Team planned fieldwork around this. Communication difficulties, due to poor phone and internet infrastructure, particularly in PNG and anywhere outside of the capital in Timor-Leste, led to some delays in interview scheduling. Nevertheless, the Team was able to meet with representatives from all prioritized organizations plus additional organizations that were identified. Accordingly, the Evaluation Team was able to successfully mitigate the most significant potential limitations and is confident that these limitations had minimal impact on results. 20 US CTI Evaluation Report IV. FINDINGS & CONCLUSIONS EVALUATION QUESTION ONE—CONCEPT How effective has the US CTI’s regional collaboration and integration approach been in achieving its Strategic Objective of “improving management of biologically and economically important coastal and marine resources in the Coral Triangle” as applied in the US CTI Results Framework? Regional Approach Context This evaluation assessed the effectiveness of the regional collaboration and integration approach (“regional approach”) that the US CTI has taken to support the establishment and implementation of the CTI-CFF as a regional ocean governance regime in the CT. The regional approach is comprised of the following: 1) coordination mechanisms that sought to bring together coastal and ocean sectors and different levels of government and various stakeholders in the management process (regional, national and sub-national platforms of collaboration and integration); 2) results areas that address specific thematic issues of concern in the CT (EAFM, MPAs and CCA); and 3) desirable values, principles and cross-sectoral approaches to management (e.g., ecosystem-based management, gender and social equity, stewardship, participation, capacity development and public outreach), which are all depicted in the US CTI RF. The regional collaboration approach that the US CTI followed in its support of CTI was assessed in terms of the extent to which it eliminates or alleviates the issues of concern addressed by the CTI; the extent to which policies have been instituted, programs have been initiated and rules are complied with; the extent to which the CTI promotes desirable values, principles and cross-sectoral approaches to management (e.g., ecosystem-based management, gender and social equity, stewardship, participation, capacity development and public outreach); and the extent to which the CTI causes changes in behavior, interest and performance among various implementers and stakeholders and their institutions, all of which together aims to contribute to the achievement of the US CTI SO. Findings on the Overall Approach of Support to the CTI-CFF as a Regional Organization In 2009 the US CTI affirmed that they set out to undertake activities at the regional, national and prioritized geographic areas that were intended to produce results that were either catalytic (effective only at the regional level), strategic/foundational (cannot be done by a single country or provides regional elements that benefit national implementation), or additive (represents the sum of country efforts).1 Based on interviews and document analysis, this regional approach is generally considered successful. The CTI-CFF is the most important emerging regional ocean governance mechanism in the region. Interviewees and survey respondents said: Firstly I think the recognition of the existence of the CTI and the necessity of international collaborations to manage is an important achievement. 1 US CTI Support Program Coordination Meeting Report, July 27–29, 2009, Denpasar, Indonesia US CTI Evaluation Report 21 Getting all CTI countries to sit together to address the issues is a major achievement. The creation of CTI as a political platform provided a way of these 6 countries working together. More importantly, the CTI is unique as it provided a political platform for Pacific Islands and SE Asia countries to come together to work on protecting the seafood basket of the world. It also rallied neighboring countries to support CTI and engaged ADB, World Bank, IFC to broaden its horizon in looking at a regional perspective to have greater impact, particularly on shared natural assets. Convincing CT6 countries of the need to approach conservation issues in a collective and integrated manner, the formation of regional platforms to address the 5 goals of the RPOA was an innovative approach to make individual nations realize the importance of trans-boundary issues that can only be addressed at regional level. I think the most notable achievement created was the formation of networks of institutions and people. There has been a great deal of knowledge exchange and learning from these networks that would have been difficult or would have taken longer to achieve without the support of the US CTI. The amount of resources (financial and technical) poured in facilitated these exchanges, and helped increase the capacity of people and institutions in the NCCs and the CTSP. The networks formed are quite strong, and I believe that these networks would continue to strengthen their partnerships, share knowledge and information and help each other whenever possible. Conclusions on the Overall Approach of Support to the CTI-CFF as a Regional Organization In terms of resolving the issues of concern in the CT that the CTI aimed to address, it would be difficult to establish on-the-ground impacts of interventions after only five years of CTI implementation. However, the abovementioned achievements, despite some challenges, all contributed toward the attainment, though not yet complete, of the US CTI SO, which is aligned with the goals of the CTI. On the whole, evidence shows—and stakeholders agree—that the US CTI was effective at supporting establishment of a new regional paradigm regarding coastal and marine resources in these Southeast Asian countries. The six countries had never before cooperated on these matters at this scale or in this depth. The common policies, frameworks, guidelines and similar that were developed were invaluable, and the support for REX meant that colleagues in the six countries could realize this potential and begin to collaborate. Findings on Specific Actions Supporting the Regional Approach Informants were consistent in indicating appreciation for the utility of the US CTI regional and national platforms, which the survey results also confirmed (see Figure 7). Interviews of regional implementers (PI staff, CTSP RPO and technical experts, Interim RS) and national implementers (CTSP national staff, NCC members) showed that the establishment and strengthening of regional platforms (regional TWGs, regional exchanges, support for the RS, Coordination Mechanisms Working Group, partner coordination support, CT Atlas) and national platforms (NCCs, except in PNG, where the NCC has not functioned effectively due to various factors not related to US CTI), as well as the development of regional frameworks for EAFM, MPAs and CCA constituted the regional approach put in place to catalyze and sustain improved management of coastal and marine resources in the CT. Interviews at regional and national levels were consistent in recognizing the good work of the PI in organizing and carrying out the regional activities of the US CTI. USAID has envisioned that a permanent RS would eventually take over the role of the PI. For all countries, CTSP was a primary supporter of the development of an NPOA, providing in most cases consultant and technical experts selected by the countries themselves, not by consortium members. 22 US CTI Evaluation Report Figure 7: The establishment and enhancement of regional and national platforms receive a higher effectiveness rating from survey informants than ratings for EAFM, MPA, CCA and the overall Strategic Objective. Results from the survey show that the program was rated more effective in the development of stronger regional and national platforms or networks for marine and coastal management than it was in the other US CTI objectives on EAFM and CCA, and to a lesser extent MPAs. As shown in Figure 8, while there was significant variation by country, a total of 94 percent of survey respondents felt the regional approach had contributed to achieving results. Figure 8: Percentage of respondents by country who feel the regional approach contributed to achievement of program results As one CTSP regional implementer stated, the US CTI covers countries which vary in capacity and use different methodologies in the management of coastal and marine resources, and the regional approach provides an opportunity to standardize methodologies across countries. In addition, the informant reported that the use of regional frameworks and tools made policy making and conduct of activities at the national and local level potentially more efficient, brought incentive to improve management, and provided an opportunity for peer pressure and learning between and among the CT6 countries. A national implementer from Malaysia expressed that the regional work is beneficial for pushing work at the national/sub-national level, e.g., dedicated funding has been allocated to support work on EAFM and 0 10 20 30 40 50 60 70 80 1 2 3 4 5 Number of Respondents Rating of Effectiveness for Each Result Area Platforms EAFM MPA CCA Strategic Objective 75% 80% 85% 90% 95% 100% US CTI Evaluation Report 23 “US CTI has kept the momentum… I cannot imagine if there were no US CTI in last five years – many things happened because they are there.” Indonesian official MPAs in Sabah, Malaysia and the Philippines, respectively. Moreover, EAFM is now a national policy in Malaysia. The regional approach aimed to develop and foster an enabling environment for the management of coastal and marine resources in each of the CT6 countries, though it is not there yet. Awareness and recognition of issues has increased at the national level and in priority geography sites, but capacity to address issues in an integrated manner following the ecosystem approach is still limited. Enabling conditions have been put in place, but enforcement and implementation remain an issue, with lack of resources at the local level as a major factor. Much more work is needed on various fronts to effect behavioral change. It has also been pointed out that it might be more practical to work at the sub￾regional level where opportunities and problems are more closely linked for fewer countries, e.g., in the case of the LRFFT. Feedback from the interviews at all levels also revealed that the informants consider the issues addressed by US CTI as largely national and local in nature, and despite the recognition of the contribution of the regional approach, 58 percent of survey respondents thought independent national or sub-national programs could have achieved similar or better results with the same resources. Moreover, while the US CTI has begun to address some of the most important threats in the CT—such as Illegal, unreported and unregulated (IUU) fishing, decreasing fishing yields, overcapacity, fishing subsidies, deteriorating conditions of coral reefs and climate change impacts—informants at the regional and national levels questioned whether some issues—e.g., encroachment by foreign fishing vessels and other IUU fishing, overcapacity and management of straddling/migratory fish stocks—may be too big for CTI-CFF to handle. They felt these issues may be more appropriate or more cost-effective for other regional bodies such as APEC, ASEAN or Southeast Asian Fisheries Development Center (SEAFDEC) to address. In fact, CTI-CFF Principles 4 and 5 suggest that the CFF use existing and future forums to promote implementation and be aligned with international and regional commitments. US CTI provided catalytic opportunities for CTI-CFF to establish these links formally and informally in various formats, e.g., LRFFT, RPOA/IUU, PEMSEA events, ASEAN MARKET. US CTI did provide the first intergovernmental contact between CT6 countries and Hong Kong, particularly Indonesia, Malaysia and the Philippines, which are the primary source countries. The work done so far within the context of the CTI-CFF and supported by the US CTI and other development partners has motivated CT6 countries’ commitment to addressing CTI issues. Support and leadership of national government agencies for the work of the regional TWGs and NCCs, and the decision to establish an RS with financial support from the CT6 countries, attest to this, although its ratification in five of the six countries (Malaysia is the only country that has ratified) has met common procedural roadblocks. Most of these regional mechanisms, however, could be deemed only somewhat effective in themselves or not yet effective since they have not resulted in systematic implementation due to a lack of connection to national and site-level activities, and they have not led to changes in management on the ground. Moreover, for the integration sites that could be considered partially effective in demonstrating effectiveness in terms of management interventions in any one of the three thematic results areas, there could only be partial attribution to US CTI regional interventions since the sites selected had benefited from pre-existing work by the NGOs. Finally, there are doubts due to the lack of strategies established to sustain these sites after the termination of the US CTI. 24 US CTI Evaluation Report Conclusions on Specific Actions Supporting the Regional Approach The program was effective in the implementation of regional activities under the US CTI, primarily those that were carried out in establishing regional frameworks and developing regional tools and plans that need to be further supported in order to keep the momentum of the collaborative initiative, including the continued functioning of the TWGs, NCCs and other regional and national platforms. A permanent functioning RS could support these. Given this, findings also suggest that coastal and marine issues such as those addressed by the US CTI—which are mostly national and local—are largely common to most CT6 countries and that there could be benefits to a program of on-the-ground action supported by information and technical exchange between and among countries. Consistent with the RPOA guiding principles, for certain issues, especially extra-regional or transboundary ones, working with other existing organizations or mechanisms, such as APEC, ASEAN, PEMSEA or SEAFDEC, may be appropriate and effective. Findings on Vertical Integration The US CTI was envisioned as a consolidated regional program to support CTI-CFF, with similar sets of activities in targeted geographical areas rolling up to national and regional activities that generate regional results. At the same time, regional results and lessons learned were intended to support national and site-level activities in targeted geographic areas. The coordinating mechanisms through which vertical integration as envisioned through this program is supposed to happen, include the regional technical working groups, the regional exchanges, regional prioritization workshops, senior officials meeting and ministerial meetings. There were instances cited by informants, however, that indicate that particular coordinating mechanisms were not working as they should, e.g., participants to the regional exchanges were not always chosen based on appropriate selection criteria; activities carried out by the two NGOs working in the priority geography sites on Manus Island in PNG are not coordinated, and the approaches of the NGOs were different. However, interviews at the local level and site visits consistently indicated that although the work of US CTI at the regional level had some connection to the national level through NCCs, e.g., the development of national/local (in the case of CCA) frameworks based on regional frameworks on EAFM, MPAs (CTMPAS) and CCA (Region-wide Early Action Plan [REAP]), activities at the local level were not well coordinated and linked to the national level. Regional linkages and direct technical support from regional teams was provided for EAFM and MPAs in Indonesia, Malaysia, the Philippines, the Solomon Islands and Timor-Leste. Experts and other regional-level staff provided training, hands-on mentoring and technical assistance in such areas as mapping, MPA network design, EAFM integration and climate change. Only a regional framework and local guidance for CCA activities have been developed. One regional informant mentioned that this was a deliberate move to expedite the development of local early action plans. This may well be justified by the need for site-specific vulnerability assessment and adaptation planning. However, this might prove to be a case of missed opportunities in that national efforts such as the National Adaptation Programmes of Action and National Plans of Action and associated funding mechanisms could provide broader institutionalization and sustainability to local adaptation. There were conflicting reports at the field level in terms of the local impacts of the regional approach of the US CTI program. Regional mechanisms sought to improve management at the local level, such as the Region-Wide Early Action Plan for CCA (REAP-CCA) that led to Local Early Action Plans (LEAPs) and the establishment of the Mayors Roundtable that spun off the formation of municipal and provincial officials leagues in the Indonesia, the Philippines, PNG and the Solomon Islands, to mention a few. However, it is too early to tell whether the regional approach could actually result in improved management of coastal and marine resources at the local level. US CTI Evaluation Report 25 Conclusions on Vertical Integration There was little vertical integration through the three levels; these could have been two separate programs (regional and national/local). Generally, CTSP implementers used this as a program to continue their work at sites—though improved with the use of regional studies, guidelines, frameworks and tools as well as some knowledge exchange—but this program as implemented was not completely effective in carrying out integrated holistic programs in six countries (i.e., not a truly novel or transformative program in that respect), although the establishment of functional regional TWGs and NCCs could be considered innovative. This may be explained by the fact that the regional, national and local mechanisms put in place by the US CTI have not been given enough time and resources to produce the desired transformative results at all levels. Given enough time and resources to address implementation challenges, the regional, national and local mechanisms put in place by the US CTI could produce the desired transformative results at all levels. Findings on Horizontal Integration Site visits and interviews of regional, national and local implementers as well as observations of TWG sessions during the second regional prioritization workshop in Manado revealed that horizontal integration of EAFM, MPA and CCA programs is weak overall and even at the “integration” sites. One informant calls it “working in silos.” Although some informants at integration sites stated that integration occurs organically since activities on EAFM, MPAs and CCA serve mutual objectives (e.g., MPAs aim to protect coral reefs from further degradation, conserve habitat for spawning aggregations to sustain fish stocks and preserve physical structures that serve as buffers against coastal erosion, storm surges and flooding), it is evident from work plans that work on EAFM, MPAs and CCA under the US CTI is still performed as very separate programs. The development of the EBM integration guide, however, is a good indicator of intent, though there is no indication of actual application. The USCTI was instrumental in promoting in CTI-CFF the concept of horizontal integration across sectors. Beyond the conceptual, and in an incipient stage of implementation, there are examples of an expanded MPA scope to cover climate change and fisheries goals, and expanded MPA planning and implementation but not of horizontally integrated programs across six countries. Eventually, the effectiveness of the regional approach has to be established in terms of connection to national and site-level activities, connection between and among national implementers and changes in management on the ground. In terms of connection between and among national implementers, the US CTI has to be credited with supporting the coordination mechanisms that allow formal and informal interactions not previously observed among those players before the CTI. Informants frequently mentioned new friendships and working relationships formed as a result of CTI. However, for the integration sites deemed successful in demonstrating effectiveness in terms of management interventions in any one of the three thematic results areas, there could only be partial attribution to US CTI regional interventions since the sites selected had benefited from pre-existing work by the NGOs, something that was anticipated and therefore is not given overdue consideration in this evaluation. Interviewees in the communities expressed the fear that if activities at the sites were suddenly stopped, then the communities would go back to earlier exploitative use, and they would also be vulnerable to re-encroachment. Note that, for people in the sites, the end of a project is a “sudden stop” since they are not involved in contracts and donor relationships. Conclusions on Horizontal Integration The three programs, or work streams, of EAFM, MPA and CCA have each proven to be effective to some degree in achieving the related RPOA goals. Each of these is at different stages on the ground but with completion of the EBM integration tool kit, integration of the three is feasible in the future. As work on these is at different stages on the ground, their integration is yet further away, and the 26 US CTI Evaluation Report effectiveness of an integrated approach as proposed to be carried out through the US CTI, as yet, neither proven nor disproven. EVALUATION QUESTION TWO—DESIGN What specific lessons have been learned about the validity of the program’s development hypothesis and results frameworks? Findings on Overall Development Hypothesis and Results Framework The RF, which consists of four results statements that capture the expected outcomes and impacts of program activities over the life of the program (See Figure 9), was developed as a management tool to provide the basis for formulating the consolidated work plan and performance management plan of the US CTI Support Program. IR1.1 Policies developed and advanced IR1.2 Institutional capacity and collaboration strengthened IR1.3 Learning and information networks strengthened IR1.4 Public and private sector constituencies engaged IR1.5 Sustainable financing mobilized CTI Goal 4 CC adaptation measures achieved Strategic Objective Improved Management of Biologically and Economically Important Coastal and Marine Resources and its Associated Ecosystems that Support the Livelihoods of Peoples and Economies in the Coral Triangle CTI Goal 2 EAFM and other marine resources fully applied R2. Ecosystem approach to fisheries management improved in CT countries IR2.1 EAFM framework developed and endorsed IR2.2 Fisheries management capacity increased IR2.3 Enforcement capacity increased IR2.4 EAFM applied in priority geographies CTI Goal 3 MPAs established and effectively managed R3. Marine protected area management improved in CT countries IR3.1 MPA System framework developed and endorsed IR3.2 MPA management capacity increased IR3.3 MPA effectiveness improved in priority geographies IR4.1 Capacity to apply climate change adaptation strategies increased IR4.2 Climate adaptation strategies applied in priority geographies R4. Capacity to adapt to climate change improved in CT countries R1. Regional and national platforms strengthened to catalyze and sustain integrated marine and coastal management in the Coral Triangle Program Indicators PM1. No. of hectares in areas of biological significance under improved management PM2. No. of hectares under improved natural resource management as a result of USG assistance PM3. No. of policies, laws, agreements or regulations promoting sustainable natural resource management and conservation that are implemented PM4. No. of people receiving USG supported training in natural resources management and biodiversity conservation PM5. No. of laws, policies, agreements, or regulations addressing climate change proposed, adopted, or implemented as a result of USG assistance PM6. No. of public-private partnerships formed REGIONAL COLLABORATION INTEGRATION US CTI Evaluation Report 27 Figure 9: Final Working Draft of the US CTI Support Program Consolidated Results Framework, Program Areas and Indicators2 The RF directly supports three of the five goals under the CTI RPOA (ecosystem approach to fisheries management, marine protected area management and climate change adaptation, in addition to a broad crosscutting result—strengthening regional and national platforms to catalyze and sustain integrated marine and coastal management in the CT). As it is, the RF does not clearly convey the program logic and requires another logic model; Figure 10 below depicts how a unified US CTI Support Program could contribute to the CTI Vision to explain how regional collaboration and integration are to occur. From the desk study, it can be deduced that the RF is based on a common understanding and commitment as laid out in the Leaders’ Declaration launching the CTI at the World Ocean Summit (May 2009), as follows:  To establish a cooperative arrangement to sustainably manage the marine, coastal and small island ecosystems in the CT region;  To ensure that these efforts contribute effectively to strengthening food security, increasing resiliency and adaptation to climate change. However, the above do not constitute a clear development hypothesis on which the RF was based. The documentation contains no formulated statement, such as: “If regional and national policies are improved and harmonized, and, if site-level work is undertaken to demonstrate the utility of these, then …” or “If comparable integrated local work is undertaken at a range of sites and conditions, then an integrated model can be tested and made applicable for wider implementation…” The RF, as with the rest of the work planning and future coordination of the US CTI and other development partners, was based on the CTI RPOA, CT6 Senior Officials and TWG Meeting decisions, roadmaps and outcomes, yet it still does not provide a logical basis for understanding how the program components work together to achieve the overall SO. Moreover, it is unclear whether Result Areas 2–4 are expected to, through regional collaboration and integration, achieve Result Area 1. Public education and outreach, which could have contributed to the achievement of the program’s SO, were not an integral part of the program design. Although a communication strategy was developed midterm, it was lacking in a deliberate effort to inform individuals, communities, governments and other stakeholders as to the uniqueness and importance of the CT and the CTI, as well as the goals and objectives of the initiative. 2 In Year 2 of the program, an additional IR was added under Result Area 4 relating to the development of a CCA framework 28 US CTI Evaluation Report Figure 10: How a unified US CTI Support Program could contribute to the CTI Vision Conclusion on Overall Development Hypothesis and Results Framework Despite the stated SO of the program, a common logic and vision were not well developed and communicated across program implementers and stakeholders. This is one of the important conditions of preparedness in program implementation that should have been adequately addressed at program start. Findings on Subordinate Results Frameworks (Results Areas) The frameworks developed in each of the three thematic results areas (EAFM, MPAs and CCA) are based on previously established frameworks and approaches (EAFM is mainly derived from the Food and Agriculture Organization’s EAF, MPAs from current logic models and body of knowledge and experience in Asia and elsewhere, and CCA from current prescriptions and guidelines developed on vulnerability assessment and adaptation planning) and are not novel or groundbreaking. Yet these frameworks are technically sound (as are the source frameworks) and worthwhile and could potentially be the foundation for robust policy making in these results areas. In the RF, each of these subordinate RFs is not shown as a framework of logical elements but rather a collation of viable worthwhile actions that could contribute to the Result. In other words, none of these is a “program” supported by a clear logic model. What may be transformative, however, is the sum of these parts, i.e., the creation of a regional system of ocean governance that incorporates the four results areas. Such a system has the potential for a collaborative mechanism that motivates the CT6 countries to work together despite differences in capacity and resources and could be effective given ample opportunity (it would take more than five years) and resources (of which relying on only donor money would be insufficient). Conclusion on Subordinate Results Frameworks (Results Areas) As with the overall RF, individual subordinate RFs are not logically internally consistent. The IRs in each do not “add up” or lead to the Result, in the sense that the elements of one level of an RF should be necessary and sufficient for achievement of the next level. While this may have resulted from expected US CTI Evaluation Report 29 diversity in activities or strategies across the highly diverse program contexts, this lack of a clearly defined, logical approach at the overall and subordinate RF levels lessens the likelihood of consistent implementation and integration across sites. Because of the program’s explicit desire to implement an integrated approach across divergent contexts, a clear program strategy, even if it requires flexibility or modification based on context, is particularly important to maximize coordination. Findings on Gaps in the Results Framework The program design is lacking in the following aspects: Except in the case of the PI, which has a gender plan as required by contract and has applied elements consistently, e.g., in staff hiring ratios, and in minimum female participation in regional exchanges, gender has not been considered consistently in the US CTI program design or in program implementation. There was no explicit mandate for gender mainstreaming neither in the original program design nor in subsequent direction from program officials. (See section on gender for further explanation.) From the beginning, the US CTI program did not have a well-designed transition or sustainability strategy, though this is in part because the US CTI pre-dated the CTI-CFF RPOA. While the establishment of the Permanent Secretariat is central to the CTI-CFF sustainability strategy, this should not have precluded the inclusion of sustainability strategies at the national and subnational levels, especially considering that the process of establishing a permanent secretariat is known to involve factors outside the control of the US CTI and that it might not happen within the duration of the program. The endgame activities and targets developed in Year 3 were intended to make certain that the US CTI program would have tangible outcomes from its activities and were not intended to lead directly to a sustainable program. The program reports that it has trained or increased the capacity of over 10,000 people according to its Performance Management Plan (PMP). Training activities did not necessarily translate into implementation on the ground; a more robust capacity development effort and application in the field were needed. For example, in a village in the Solomon Islands, selected community members were trained on a standard monitoring procedure for MPAs but have not done any monitoring of their MPA site as yet. Although Malaysia and the Philippines—two very capable countries already—were successful in establishing national programs that can sustain initial efforts in EAFM and MPAs through US CTI, this has not been achieved by the other four countries. Absence of an approach/strategy to strengthen the local level institutions (government and civil), which are tasked with management of coastal and marine resources (inshore), was critical. This is a common finding based on the interviews in the CT6 countries. CTSP implementers shared that they find themselves bypassing provincial or municipal involvement in critical steps in specific interventions in order to do their job based on work plans. Generally, the consortium of NGOs did not involve local indigenous NGOs in CTSP activities at the integration sites in a way that developed their capacity to operate effectively or independently. It is true that local NGOs are largely weaker and less capable of independently taking on the work but the involvement of and development of capacity of local NGOs was also not an explicit strategy. Despite this, there were limited examples of involvement of local NGOs in specific situations, including MPA partners to TNC. Lack of focus on the development of alternative or sustainable livelihoods was consistently identified by respondents as a problem, especially at the local level, and the lack of emphasis on the development of public-private partnerships and tangible involvement of other line ministries and sectoral representatives in the NCC were contributors to this. This lack may have led to lessened local support for CTSP work. Community members interviewed in the Solomon Islands described the program as “all stories but no action,” referring to the lack of livelihood interventions. 30 US CTI Evaluation Report It appears that the concept and practice of EAFM have not yet been firmly entrenched among national and local implementers, as indicated by the intention of at least four of the CT6 countries to deploy inshore fishery aggregating devices (IFADs) as an alternative livelihood, despite its potential contribution to overfishing in inshore and pelagic waters. This is another indication that greater regional cooperation in the CT did not necessarily lead to better substantive results in the issue area at the national level. Moreover, US CTI’s intervention regarding IFADs was limited to CTSP’s planned activities on IFADs for Years 4 and 5 in PNG. For Year 4, WWF was supposed to “Continue technical assistance to NFA (National Fisheries Authority) to finalize the community-based FAD deployment policy and best practice guidelines for FAD deployment and to support three LLGs in Manus Province in the application of the best practice guidelines” the outcomes of which were not found anywhere in PNG’s summary of achievements. For Year 5, WWF in PNG is supposed to work together with NFA to establish socio￾economic/livelihood monitoring tools to measure the impact of IFADs, which did not materialize because WWF US put a hold on WWF PNG Year 5 activities due to financial management and reporting issues. Although current intended deployments, which are mostly government initiatives not supported by US CTI, may have a sound basis from a WorldFish Center initiative on IFADs, both national implementers and local governments noted that government ministries will be unlikely to have the capacity to strictly adhere to guidelines on deployment and management, which are prerequisites to sustainable IFADs. Moreover, in some of the sites, IFADs seem to eclipse MPAs and CCA in terms of community interest and support since the IFADs represent a tangible delivery of livelihood, even if not yet implemented. Regional business forums did not sufficiently address development of alternative livelihood; lack of tangible success in the establishment of PPPs is also a factor (Manus Fishing Cooperatives as a hub for community fisheries marketing, learning and development is one of the few successes). The report of the Second Regional Business Forum merely reiterated the need for it, mentioned some potential innovations and contained broad recommendations, but it did not present concrete actions or strategies for moving forward in the development of alternative livelihoods. It should be noted that while the livelihood component is essential to the success of management programs, the US CTI is a biodiversity￾funded program following USAID guidelines in utilization of biodiversity funds. Conclusion on Gaps in the Results Framework The weaknesses identified in program design have to be taken into consideration in terms of their potential impact on desired transformation at the local level although not the main focus of the regional collaborative approach. The weaknesses may have contributed to decreased local level support and perhaps engagement in other program activities, which in the long term could lead to relatively lesser on-the-ground impacts. Without concrete involvement of local institutions, the program is unlikely to attain its expected impacts. The growing popularity of IFADs as an alternative livelihood in the CT could have provided an object study in which the elements of the regional collaborative approach could be applied: 1) the precautionary principle in EAFM should have been applied, given its potential contribution to overfishing in inshore and pelagic waters; 2) it should have taken into consideration what positive and negative biological and ecological impacts IFADs can pose on MPAs; 3) the US CTI could have been proactive in establishing guidelines and criteria for the selection and development of alternative livelihoods, which could have been used in decision-making by countries in the deployment of IFADs, although the development of alternative livelihoods is outside the scope of the current program, except for those that could directly affect biodiversity; and 4) IFADs could have been the focus of an integrated activity and closer interaction between and among the three thematic TWGs. This evaluation was not designed to assess whether US CTI promoted the deployment of IFADs as an alternative livelihood practice despite their controversial nature. Rather, it was to establish whether the regional framework and US CTI Evaluation Report 31 national policies on EAFM put in place, as well as the intended technical assistance to PNG with support from US CTI, were able to support national decision making on the matter. EVALUATION QUESTION THREE—IMPLEMENTATION How effective have US CTI actions under Intermediate Results been in achieving corresponding Results? Overall Findings on Effectiveness of Intermediate Results Achievement of the US CTI RF IRs that call for “policies developed,” “constituencies engaged,” “networks strengthened,” “capacity increased” and similar was generally high. There was actual achievement of the target on putting frameworks in place, in particular, the RPOA, EAFM Framework, CTMPAS Framework and Action Plan, REAP-CCA and LEAP Guide. Those IRs that called for more complete actions such as “sustainable financing mobilized,” “EAFM applied,” “MPA effectiveness improved” and “climate adaptation strategies applied” were generally at a more incipient stage. Overall Conclusions on Effectiveness of Intermediate Results It would be expected that, in a longer time period, the achievements to date in the development of frameworks, building of networks and establishment of relationships would result in achievements in implementation, but at this point there is not clear evidence of systematic changes in management across result areas. Findings on Result Area 1: Regional Platforms US CTI initiated forming and supporting regional and national platforms including the Interim Regional Secretariat (IRS), the NCCs and the TWGs and promoted awareness of the need for conservation of resources. Activities for the development and advancement of policies and frameworks as well as strengthening of institutional capacity and collaboration were mentioned by the national and government respondents, including the CTSP implementers, as an achievement worth highlighting in the US CTI program. Results of the electronic survey showed that the respondents perceived the regional/national platforms as the most successful component. This supports the findings of the Evaluation Team in the field interviews. Activities related to information and learning networks, IR 1.3, were limited and were conducted through the provision of grants to the Coral Triangle Center (CTC) and the University of the Philippines Marine Science Institute (UPMSI) to help initiate the formation of regional learning networks on MPAs and CCA through mentoring and developing capacities of other local institutions and organizations. Activities for IR 1.4—public and private sector engagement—mainly consisted of conducting business forums. In Nusa Penida, private sector constituency engagement consisted of small livelihood endeavors, which included community mangrove ecotourism, seaweed farming ventures and support services such as simple restaurants. Elsewhere, this included income for women and wives of fishermen through handicrafts, as in Maliangin Island and Banggi for weaving, which resulted in a contract between Banggi and the Shangri-la Tanjung Aru resort. There was not a program-wide effort toward mobilization of sustainable financing, IR 1.5. Although at present they remain mainly livelihood activities, they may, when enhanced and sustained, be the triggers in mobilizing a stable and diverse sustainable financing for the MPA. Awareness of results related to the activities of the information and learning networks was limited even among those interviewed at the national level and more so at the local level. This was because initial grant activities were done only in Indonesia for the CTC and the Philippines for the UPMSI. Activities 32 US CTI Evaluation Report for IRs 1.3-1.5 have to yet to gain sufficient traction for them to contribute to the achievement of the expected results. Conclusions on Result Area 1: Regional Platforms Activities in the formation and support of regional and national platforms—including the IRS, NCC and TWGs and the resultant formation of teams—that succeeded in coming up with outputs for each of the thematic areas, despite bureaucratic and administrative challenges, built trust and confidence among the CT6 countries. The years of working together as a team resolving issues and concerns have brought about the realization of the need and importance of collaboration. The regional exchanges that supported the technical meetings, workshops, site visits and the trainings involving government, project partners and local and community partners provided the venue for development of awareness, understanding and appreciation of the socioeconomic, cultural and political realities affecting the management of resources, not only of individual countries but the entire CT region. Although much has yet to be done in terms of catalyzing and sustaining marine and coastal management in the CT, the initial achievements in IR 1.1 and 1.2 positively contributed to achieving Result 1 in the RF. Findings on Result Area 2: EAFM In EAFM, there was an active TWG for the development of the RF, a 10-year road map, a number of regional exchanges and technical publications. A recent decision to make SEAFDEC the IRS for LRFFT through an MoU will enable the program to benefit from the expertise and experience of SEAFDEC. Another output was the publication of an integrated guide to ecosystem-based coastal areas and fisheries management in the CT. Nine key informants expressed the value of this guide in providing the much-needed guidance to boost EAFM activities at the sites. Based on the KIIs of the three thematic areas (EAFM, MPAs and CCA), EAFM was considered the second most effective US CTI action in achieving its objective, behind MPA. This result was consistent with the electronic survey conducted. Conclusions on Result Area 2: EAFM Activities in EAFM revolved around the development of the draft regional framework, the 10-year road map, a number of regional exchanges and publication of EAFM technical products, but implementation and application at the priority geographies is just beginning. Fisheries management (IR 2.2) and enforcement capacity (IR 2.3) have been addressed through trainings and workshops, yet application of EAFM in priority geographies (IR 2.4) is lagging. It was noted, though the indications of the encouraging effects of earlier training and workshops had been witnessed by key informants, particularly in the Philippines through the use of US CTI lecture slides by a fisheries government official in a lecture that promotes the concept of EAFM. Of the four IRs under Result 2 in the RF, only IR 1.1 had been fully achieved through the development of an Integrated Guide. Since the Guide was completed and disseminated in August 2013, its effect in influencing countries to take action on the implementation of EAFM and finally contributing to the achievement of Result 2 is not apparent. Findings on Result Area 3: MPA In MPAs, there were regional exchanges that included a significant number of technical meetings, forums and training and establishment of the TWG that led in the development of the CTMPAS Framework and Action Plan, the first joint policy product on MPAs for CTI-CFF. Other important achievements were the development MPA section in the CT Atlas Website that can offer useful and accessible information for design and planning of MPA and MPA Networks for the CT6 governments, researchers and other practitioners in the region, although it has yet to succeed in aggregating other useful information relevant to the improvement of MPAs and MPA management, as well as the marine resources in general. Among the three thematic areas in the US CTI RF, MPA was ranked by KII and electronic survey respondents to be the most effective US CTI action for achieving its objective. Existing advanced MPA US CTI Evaluation Report 33 activities and successes in some CT6 countries boosted the early development of action plans, inspired information sharing and promoted appreciation of the importance of MPAs in improving the management of marine resources. The MPA Management Effectiveness Assessment Tool (MEAT), which was developed in the Philippines and has attracted interest from Malaysia and Indonesia, also represents a significant advance in this area. Although, based on the KII responses, countries like Indonesia and Malaysia intend to adopt the MEAT with modifications to suit local conditions, the development of the MEAT had been a big step toward a systematic and standardized assessment and determination of MPA effectiveness. However, at least 11 informants noted that delay in the legalization/gazetting of MPAs by national governments (e.g., Malaysia, the Solomon Islands and Timor-Leste, and even Bali, Indonesia) is a factor that hinders full implementation of plans, framework and tools. This delay in legalization has occurred for a number of reasons, including government processes and actions unrelated to US CTI program production of information or plans. Conclusions on Result Area 3: MPA US CTI actions toward achievement of IR 3.1 by supporting the development and endorsement of the CTMPAS framework, achievement in part of IR 3.2 via the support for the development of the CT Atlas and the achievement in part of IR 3.3 through the development and adoption of the MEAT created tools and frameworks to assist in improving MPA management. However, the Evaluation Team found little evidence, including through stakeholder interviews, that management of MPAs had significantly improved, since the capacity and resources for management have yet to be fully realized at the site level. Findings on Result Area 4: CCA In CCA, regional exchanges, trainings, workshops and technical assistance were conducted to assist the countries in developing the REAP-CCA that led directly to the LEAP. A final version of a more concise and short version of a LEAP Guide was also produced. Informants expressed, and the survey showed, that the CCA theme is considered the least effective US CTI action of the three thematic areas. It must be noted, though, that this perception by the respondents who were directly or indirectly involved in CCA activities may be due to the fact that, among others, it took some time for the REAP to be developed since it started through the first CCA regional exchange in Jakarta in October 2010 and was followed up by another in the Solomon Islands in April 2011. As a result, early steps had been taken in most CT6 countries to establish teams under their NCCs with multi-agency and sectoral representation, including representation from NGOs. Steps to build capacity for implementation had also been undertaken, with a focus on the local government that will be on the frontline in experiencing climate change. Nevertheless, it is apparent that even with these efforts there is still a need to increase efforts to effectively catalyze more support and enhance understanding of climate change impacts. Limited implementation of explicit CCA activities was observed at the site level. Explicit CCA activities in most countries and sites consisted mostly of studies, assessments and piloting the LEAP guide. The most commonly reported activity was mangrove replanting, which is a form of reducing “non-climate” stressors. This internationally recognized approach for addressing CCA in marine ecosystems was reinforced in CCA training. In general, CCA activities lag behind MPA and EAFM. Conclusions on Result Area 4: CCA Under Result 4, major outputs were the development of CTI REAP-CCA as policy instrument to guide the CT6, support for the development and integration of CCA action plans in local government development plans, the development of the final version of the LEAP guide and its adoption, the establishment of the CCA Centers of Excellence and the conduct of regional exchanges including provision of technical assistance. The conduct of the various CCA activities increased the level of awareness of the CT6 countries on the issues of climate change, yet IRs 4.1 and 4.2 have yet to be 34 US CTI Evaluation Report achieved. However, it is recognized that funds for CCA work were markedly lower than for the other thematic areas. EVALUATION QUESTION FOUR—CTI-CFF How effective have US CTI results been in contributing to the accomplishment of corresponding CTI-CFF Goals, Strategic Objectives and Targets and the US CTI Strategic Objective? Findings on Consonance of US CTI with CTI-CFF Goals The US CTI RF directly supports three of the five goals under the CTI-CFF RPOA (EAFM improved in CT countries, MPA management improved in CT countries and CCA improved in CT countries). The objectives of the US CTI under the three results areas (EAFM, MPA and CCA) were fully consonant with the corresponding RPOA Goals 2–4. This is unsurprising since the activities were planned together while US CTI was starting in 2008 and the CTI-CFF in 2009, based on the US CTI, and with the same participants. The focus on the three CTI RPOA goals has helped apply the program’s resources to a set of manageable goals in a targeted way. Effort was also made by RDMA to identify additional USG support (e.g., bilateral) or other development partners that have contributed to the achievement of the CTI RPOA Goals but are not specifically a part of the US CTI. These additional resources are not necessarily limited to support for RPOA Goals 2–4. RPOA Goals 1 (Seascapes) and 5 (Threatened Species) were partially covered by activities under US CTI and, in some countries, explicitly included by the NCCs in their work on EAFM or MPA. Examples include, in the Philippines, work by CI in the Western Philippines Sea; in Indonesia, work by CI in the Bird’s Head Seascape; and in the Philippines, Indonesia and Malaysia, work by WWF in the Sulu-Sulawesi Marine Ecoregion. However, when interviewees were specifically asked about aspects of the other two RPOA goals not covered, the level of knowledge of RPOA Goals 1 and 5 was low. Knowledge of those RPOA Targets not explicitly covered in US CTI, such as COASTFISH, and National Centers of Excellence on CCA, was low among US CTI participants. Even though the NCCs in each country were conceived as being responsible for all five RPOA Goals, in practice there was significant variation in ability or function, and most operated primarily in response to the three goals addressed by the US CTI. Conclusions on Consonance of US CTI with CTI-CFF Goals The US CTI has directed its resources toward the accomplishment of RPOA goals 2–4, aided by additional resources from bilateral programs and efforts by other development partners. Success achieved in these areas was significant although limited to those actions that set up the enabling environment more than actions on the ground (see above). Though focus on RPOA goals 2–4 was explicit in the program, US CTI efforts also contributed, to a limited extent, to RPOA goals 1 and 5. To this extent, however, the work is still at a stage of consolidation. The US CTI has been effective at supporting the three CTI-CFF RPOA goals on which it explicitly worked. EVALUATION QUESTION FIVE—LESSONS LEARNED AND FUTURE What key challenges, either technical or managerial, have been encountered and innovative practices applied in achieving the US CTI Strategic Objective, thereby contributing to the CTI-CFF Goals? US CTI Evaluation Report 35 Findings on the Implementation Context The US CTI program was implemented in six countries that represented a wide range of marine conservation experience, resources and human capacity, as well as physical and communication infrastructure. Not only was there vast variance in these factors across countries, there was also large variation within countries, particularly between the capitals and implementation sites. PNG, the Solomon Islands and Timor-Leste all had significantly less resources and human capacity than the Philippines, Malaysia and Indonesia, leading to different challenges, national priorities and approaches across these countries. Moreover, the communal approach to land ownership in PNG and the Solomon Islands requires a different approach to MPAs particularly. Finally, poor infrastructure and high operating costs at the country level in PNG, the Solomon Islands and to a lesser extent Timor-Leste, complicated implementation. Indeed, the Evaluation Team experienced a more than 24-hour Internet outage while in Port Moresby, and CTSP reported regular, much longer outages. It is important to keep this implementation context in mind when considering the technical and managerial challenges in US CTI implementation. Conclusions on the Implementation Context It is clear that the CT6 countries represent a wide diversity of experience, capacity and resources, significantly complicating efforts for a regional or integrated approach to implementation. Any approach in such a diversity of contexts must allow for flexibility, yet it is under precisely these types of circumstance that strategic planning and effective M&E are most beneficial since they strive to provide an adaptive, yet targeted, management approach. Given these circumstances, US CTI’s progress— particularly in the development of regional platforms, frameworks and relationships—is remarkable. However, as discussed in more detail below, improved coordination, a more responsive M&E system and a clearer RF could have contributed to further success. Findings on Using Multiple Mechanisms USAID/RDMA employed three mechanisms to achieve the US CTI objectives: a cooperative agreement (CA) for CTSP; a contract with the PI; and an interagency agreement with NOAA. This was intended to leverage the complementary skillsets of each implementer in a coordinated fashion to better achieve the initiative’s objectives. The PI was seen as a critical component of a coordinated response. Indeed, the purpose of the PI contract was to be “mainly responsible for coordinating inputs from various USG agencies and partners and for facilitating a unified USG response to the CTI.” The Evaluation Team found, however, that particularly initially, USAID/RDMA, the PI and CTSP all noted a lack of clarity and direction on roles and responsibilities for the PI, particularly vis-à-vis CTSP. Although part of the PI’s expected function included developing annual consolidated national and regional work plans through consultation with the US CTI partners, the existence and role of the PI was not recognized in the CTSP CA, which called for many of the same tasks and functions as described in the PI contract, nor was the PI role made clear to CTSP, according to both the PI and CTSP, upon program initiation. Given that the internal coordination and reporting functions outlined in the PI contract are typically undertaken by the primary implementer (CTSP in this case), representatives from both the PI and CTSP noted that the introduction of the PI generated tension around the ownership and leadership of certain management processes, including work planning and reporting. This affected accomplishments in the early years of the program. Both the PI and CTSP, along with USAID/RDMA, acknowledged that this tension dissipated and the work planning and reporting functions ran more smoothly as the project moved on. Although not all external stakeholders, particularly those focused on national or site-level activities, were aware of the activities of NOAA or the PI, those that were typically placed a high value on their work, particularly the function of coordinating external stakeholders and donors performed by the PI. Indeed, the PI’s role in donor coordination was one US CTI program activity that multiple external stakeholders cited as a priority function that should be continued. 36 US CTI Evaluation Report The Evaluation Team noted that although the role of other donors in supporting the CTI-CFF was less than the USG, both the ADB and the Government of Australia have been providing significant resources to the RS and to the ratification process respectively. Donor coordination is especially important after the 2013 Manado Workshop, where the donors reaffirmed a unified position on ratification. Conclusion on Using Multiple Mechanisms The lack of clarity on roles and responsibilities among the implementers of the different US CTI mechanisms generated initial confusion among partners which, although they lessened over time, generated inefficiencies in coordination and limited the ability of the PI to implement its intended coordinating functions. This may have also influenced the finding that US CTI was unable to develop an M&E system that could be, or was, used for performance management, although the Team cannot definitively attribute this to the lack of clarity on roles. Should USAID consider using multiple mechanisms in the future, in such cases it might consider assuming the internal coordination function itself. When USAID is unable to assume the internal coordination function, ensuring clear roles, responsibilities and relationships from the beginning and actively managing those relationships can help ensure productive coordination. (See Recommendations below.) Findings on Consortium Approach USAID/RDMA awarded a cooperative agreement to WWF, which serves as the prime of a consortium of NGOs that also includes TNC and CI. These NGOs were already operating in the CT6 countries. Involving these three NGOs allowed access to networks, technical resources, community relationships and experience that would have been impossible to find in a single organization. Moreover, by utilizing one mechanism to access these three partners, USAID/RDMA was able to reduce initial procurement, contractual and, and at least in theory, management transaction costs over time. However, while USAID/RDMA anticipated that working through a consortium of large, USAID-experienced NGOs would reduce management burden, the Evaluation Team found that the consortium approach resulted in significant management challenges. Each consortium member brought its own reporting systems and structures. This was even true for WWF country offices which were independent from WWF-US (the CA prime) and represented either independent country offices or were affiliated with other WWF offices. Because of this, once budgeting, work planning and reporting passed from the RPO to the country offices of the partners, according to each of the country offices, all had to move through different parallel structures before coming back to the RPO for consolidation. Different formats and timelines then often led to delays in consolidation. Most country offices also reported that funding envelopes assigned to their country office often changed after submission of a draft work plan and budget to the RPO, resulting in a lengthy revision process and delays in implementation. These delays can at least in part be attributed to consortium-member country offices submitting initial budgets over their allotment or otherwise not following prescribed guidance. CTSP also reported that delays in USAID approval of work plans also contributed to implementation delays. One implementer country office reported that they typically lose three months per year due to delays in work planning, and that one year they had to develop five iterations of their work plan and budget, again, likely in part due to not following prescribed guidance. They also reported that funds are typically disbursed to them in December or January (one year not until March), resulting in significant difficulties for maintaining program continuity. These issues were supported by the other implementer in the country and are consistent with responses from implementers in other countries also. This was at least partially attributed to the parallel structures (i.e., each consortium member had its own structure for project planning and approvals) and lack of clarity on how country office budgets are aligned or consolidated. It should also be noted that delays were also caused by lengthy internal processes within consortium-member NGOs, as reported by an implementer in PNG as well as the RPO. US CTI Evaluation Report 37 Each of the CTSP consortium members acknowledged that the process for allocating budgets to each organization and country office was opaque. Eventually, to minimize overt competition for resources, some allocation decisions were made by the RPO based on factors such as Chief of Party judgment on absorption capacity, staffing levels and government support. Even so, the lack of objective criteria for the decisions or predetermined funding levels still led to reported low-level competition for resources, rather than complete coordination. The three NGOs making up the consortium have independently defined missions both generally and for their work in the CT6 countries, which informed their programs preceding US CTI. They all also have funds from other donors supporting their work in these countries. National-level stakeholders repeatedly reported that the activities of the NGO partners were not always aligned internally or with NCC priorities. Although the NCCs represent multiple organizations and multiple potential priorities (which may also change over time), this view was shared across NCC member and chair organizations. For example, some NCC representatives in PNG reported that priority sites were determined more by consortium-member experience than government priority, and NCC representatives in the Solomon Islands noted that it “was a pity” that the program focused so exclusively on Gizo. In fact, most US CTI sites were selected because of ongoing NGO work there, which balances the benefit of a solid base on which to build and the likelihood of being able to accomplish something in five years while also lessening the likelihood of truly innovative programs or complete coordination with government priority actions. The somewhat independent objectives as well as outside funding of individual NGOs (which certainly provided the opportunity for leverage and was a requirement) often led to confusion among stakeholders at all levels, although particularly at the site level, about which activities or programs were part of US CTI. Accordingly, many stakeholders could not definitively say what activities or effects they would attribute to USAID, US CTI support or even to CTSP since they better identified the activities with the NGO implementing them. Moreover, in countries with more than one implementer, there was no centralized US CTI or CTSP country-level coordinator to ensure coordination, communication and integration of activities. Although CTSP attempted to establish country coordinators from existing staff of consortium-member country offices, these were, with one exception, reported to be relatively unsuccessful due to lack of line management through, and direct accountability to, CTSP. Despite these challenges, external stakeholders generally had positive views of program implementation, with only 10 percent of survey respondents saying that the program was “poorly” or “very poorly” implemented (and 42 percent saying the program was “well” or “very well” implemented). Moreover, while CTSP implementers in each country gave examples of how difficulties in coordination among consortium members affected implementation, very few external stakeholders explicitly noted that internal program coordination or management challenges limited implementation. Conclusion on Consortium Approach Although relationships, communication and coordination between consortium members have become more efficient over time, and CTSP made strong efforts to minimize the impact of the challenges on implementation, the mostly unanticipated management challenges, alongside delays from USAID and the very complex regional program context, contributed to slow program start-up and inefficiencies in key activities such as work planning, which resulted in regular delays in project activities and suboptimal allocation of resources. While implementation through a consortium likely provided better coordination than would have been expected through independent programs in each country, implementation through three large organizations, unsurprisingly, resulted in management inefficiencies relative to implementation through a single organization. Having three independent NGOs with histories and ongoing activities outside of US CTI in the CT6 countries leveraged a unique set of resources and relationships but also generated slightly different, independent objectives for each organization’s work. These objectives were not always completely 38 US CTI Evaluation Report aligned with the overall US CTI objectives and required working through multiple, parallel structures, thereby reducing integration and coordination of activities at all levels. The lack of a country-level coordinator led to suboptimal communication and reporting between governments and CTSP, as well as among CTSP partners. Findings on M&E as a Management Challenge Under the direction of USAID/RDMA, the US CTI M&E system was initially based on a set of six performance indicators, yet, as reported in the Regional Inspector General (RIG) Review, US CTI “used standard indicators that did not measure progress toward the four main objectives.” Despite proposals to the contrary by implementers, USAID noted that they wanted to keep the M&E system relatively simple. However, since the program indicators were not aligned with project activities or the result areas, they were of minimal use for performance management, and indeed, the Evaluation Team found very little evidence of the use of indicators for program management. Moreover, USAID/RDMA reported low levels of knowledge among the consortium partners of the M&E system. Despite trainings and setting up online systems, CTSP reported that a lack of resources and capacity limited responsiveness on M&E issues, particularly in PNG, the Solomon Islands and Timor-Leste. The RIG Review recommended that US CTI “develop performance indicators that measure activity-based outputs and related results for each of the partnership’s four objectives,” which would provide clearer information on project performance at levels that could inform management decision making. Following the RIG Review, additional indicators and ‘sub-indicators’ were added, with the sub-indicators, which were custom indicators developed for five of the six original indicators, providing data more closely aligned to the project activities and result areas. However, in agreement with USAID/RDMA in response to the RIG Review, these were only reported starting in the year four annual report, which limited their utility for tracking performance over time and adjusting programming accordingly. A cursory review of the M&E system proposed for adoption by the CTI-CFF shows that it has a good deal of positive theoretical thinking about possibly useful indicators, but neither the indicators themselves are fully developed nor is the system to gather, manage or use them fully conceived. Conclusion on M&E as a Management Challenge Although US CTI was, from the beginning, understood to be a very complex program with long lines of communication and reporting, and USAID did not want to further complicate management systems, it is arguably in exactly such cases that a sophisticated M&E system is of highest utility. A well-designed M&E system can summarize and focus large amounts of information to reveal important trends across countries, partners and result areas and help identify possible management actions to maximize performance. The Team noted that the Indonesia marine program is emphasizing a common M&E system for their program. Also, as CTI-CFF works to finalize its M&E system, additional support and thought is required to refine and operationalize all indicator definitions as well as develop clear processes for collecting, managing and using monitoring data in order to provide a system useful for performance management. Findings on Other Implementation Challenges US CTI was designed to contribute to the RPOA and NPOAs, collaborating with CT6 governments and the NCCs. The Evaluation Team found that in many cases consortium-member country office priorities and activities were not directly aligned with government and NCC priorities. Yet CTSP and even government officials in Timor-Leste, PNG and the Solomon Islands, noted that there was sometimes a tension between getting government buy-in and support, which could often take significant time, and proceeding according to ambitious work plans and program targets. NCC representatives in nearly all countries noted that they had limited resources and capacity to fully support or engage in US CTI activities, particularly at the sub-national level where ministry staff are overburdened or nonexistent. For example, in the Philippines, the provincial fisheries officer for Taytay, who did not have a fisheries US CTI Evaluation Report 39 background, had been recently transferred at the time of the Evaluation Team’s visit, a situation similar to that in Timor-Leste, though the newly transferred official was otherwise competent and very interested in the program. Likewise, in the Solomon Islands, the provincial environment officer for Gizo had been transferred to tourism. Accordingly, particularly at the local level, government and ministry officials only had limited availability to engage with CTSP, restricting opportunity for coordination. Conclusion on Other Implementation Challenges CTSP cannot “push” the governments but also needs to be able to accomplish the work when not receiving full coordination, as is likely to happen particularly at the sub-national level. However, this emphasizes the need for internal coordination and regular reporting and communication with national￾level stakeholders. Without CTSP country-level coordinators, this only happened sporadically or not comprehensively. Although NCC representatives from the Solomon Islands still complained about not receiving complete reporting from CTSP implementers, the CTSP-NCC liaison role was noted by the NCC and CTSP to be highly valuable in helping to both coordinate the NCC activities as well as facilitate communication between US CTI and the NCC. Findings on Innovative Practices Generally, stakeholders did not characterize this as an “innovative” program, except possibly at the highest conceptual and political level of the CTI-CFF itself. Neither the frameworks nor the tools, while valid and up to date, nor the work on the ground, while useful and valuable, was considered as representing innovative practices. An exception could be the EBM integration tool kit as an innovative practice but exchanges of personnel (REX) in learning or joint trainings for regional programs are an accepted practice, and the PI was instituted based on previous successful USG post-tsunami experience in Asia and has also been employed in other regional USAID regional programs. The most commonly cited innovative approach taken by US CTI was the Happy Fish mobile platform for data collection, which was being piloted in the Solomon Islands, with CTSP reporting that there was interest in scaling up to other CT6 countries. Working with the Ministry of Fisheries, US CTI developed a survey on smartphones to be regularly conducted with market traders to monitor fish catch. While the application of smartphones for data collection is not novel, the application for measuring fish stocks at markets is innovative. Conclusion on Innovative Practices In part because the overall program approach was not considered innovative except at the highest level in bringing together the six countries, and because it tended to rely on conceptually valid, existing approaches, the Evaluation Team did not find many examples of innovative practices applied in actual program implementation. Although the Happy Fish application generated significant interest and has potential for scale-up into other countries, its rollout in the Solomon Islands was limited, and hence it is too early to conclude whether the approach is a viable strategy for measuring fish stocks. FINDINGS AND CONCLUSIONS ON GENDER Findings on Gender Gender has not been considered consistently in the US CTI program design or in program implementation at any level. There was no explicit mandate for gender mainstreaming in the original program design or in subsequent re-design. Direction from program officials was to incorporate gender into activities and there are specific instances of success working with women’s groups, for example. The US CTI did carry out gender analysis in a number of reports of activities, e.g., a regional workshop on establishing learning networks found a quarter of the participants were female, but no gender analysis 40 US CTI Evaluation Report of activities was carried out at the site level. At least one key US CTI product, the EAFM Plan, encourages the involvement of women in the beginning of the process of developing an EAFM plan that considers climate change and ocean change. In the Solomon Islands, where women are not customarily actively involved in decision making in the management of coastal and marine resources, despite the fact that their role in certain parts of the supply/value chain is vital, they were actually at the forefront of program implementation. For example, two women are running the NCC well there. Women are running the NCC, the PNG Center for Locally Managed Areas, the TNC work in Manus, the MECC and the CCA work in PNG. In the other CT6 countries, there are also women leaders working at various levels, prominently so at the national level: the NCCs of four out of the six CT6 countries were represented by women at the second Regional Prioritization Workshop in Manado in August 2013. Conclusions on Gender Although the participation of both sexes was evident in various levels of the hierarchy in US CTI, the lack of a systematic process of assessing the implications for women and men of any planned action, including policy making in all areas and at all levels, still needs to be addressed. US CTI Evaluation Report 41 V. RECOMMENDATIONS URGENT RECOMMENDATIONS 1. USAID should urgently identify, develop and implement an interim mechanism for continuation of certain functions performed by the PI including, specifically, the CTI donor coordination function. The PI presently provides important services to the countries and to the IRS and represents the continuation of USAID commitments to the CTI-CFF as an organization. At this critical juncture, this continuity of commitment is important to demonstrate. The donor coordination function, in particular, is among the most important, especially since the recent Regional Priorities Conference in Manado. Other important functions besides donor coordination include continuing regional information exchanges, TWGs, communications, website, etc. Transitional assistance should be provided to the CTI-CFF and RS with the object of moving them toward self-sustainability. The object should be to move assistance along the continuum of: 1. providing services to the IRS externally; 2. providing services to the IRS internally through staff seconded or embedded; 3. supporting IRS in carrying out services themselves; and 4. providing occasional ad hoc services, consulting or specific capacity to the IRS 5. providing support to the active TWGs 2. USAID should urgently identify, develop and implement an interim continuation program for certain activities of the NGO-led program so as not to lose the gains already achieved on the ground. The most important functions include those that affect people on the ground, such as continuing the community enforcement of protected-areas and reef-monitoring programs. This support could conceivably be done through bilateral missions, but continuing assistance from USAID is critical. Note that this does not apply to Indonesia, whose marine program continues through 2014. 3. USAID should communicate with field teams and other stakeholders about the plans for continuation. There is significant uncertainty in the field among the CTI-CFF governments, NGOs and donors about the intentions of the RDMA with respect to the continuation of the U.S. support of the program. At the time of the preparation of the Out-Brief presentation, there was discussion of this with RDMA in general, but the Evaluation Team had not seen in writing and announced the plan for continuation of programs by the bilateral missions. At this meeting, the most significant message from USAID was that future assistance would be conditioned on the progress toward ratification. Even after the Manado Regional Priorities Conference, this ratification message would not be undercut by an announcement that temporarily some activities would be continued. In fact, it was implied in discussions and might be taken as a good will gesture or as one of true continued interest on the part of USAID. 42 US CTI Evaluation Report “If what they needed and what was achieved was all at the regional level, then why did we fund NGOs so much at the site level?” Indonesian official REGIONAL-LEVEL RECOMMENDATIONS 4. USAID should support a time-limited project that focuses on the CTI-CFF as an organization, providing transitional assistance to the interim RS and the permanent RS when created. This builds on the important progress that has been made and recognizes the importance of the issues. The size and structure of this new program is dependent on the ratification of the CTI-CFF by the countries. The program could also employ a measure of conditionality with successive stages or levels of assistance dependent on ratification and provision of financial and other support by CTI countries. If the CTI-CFF is not ratified and RS not created, the project would help to transition the extra-regional CTI-CFF issues to other appropriate organizations, mechanisms or forums. Such a program would therefore be even more limited and focused. (Note that this is a direct follow-on from Recommendation 1.) 5. In any follow-on CTI-CFF Support Program, USAID should include direct technical assistance (TA) to the NCCs to help them become viable nationally owned mechanisms/organizations that implement and oversee their national programs. On the assumption that the members of the NCCs are already proficient in their technical areas (fisheries, forestry, environment, other), this capacity-building assistance should focus on organizational, program management, participatory planning and other areas that would help them carry out their coordinating and management functions. The NCCs themselves should also be encouraged to include more participation from local NGOs, communities, the private sector and others with interests and complementary expertise. The Evaluation Team believes that this TA would best be provided through a regional program since the types of TA needed by all of the NCCs will be more similar, and there would be value in having all of the NCCs receiving very similar training, perhaps at the regional level. Further, this would best be provided at the regional level by an organization or firm with experience in the areas of organization development, knowledge management, technical exchanges, capacity building, etc. since the expertise to provide this TA would likely not be best held by an environment NGO that works at the site level. 6. The new program should support exchanges of technical and experiential information among the country programs and sites to capitalize on the value of networking (link to country programs), TWGs, national and regional learning networks, etc. This recommendation actually crosses the interface between both the intercountry level (country to country, site to site), which the complementary bilateral programs would carry out, and the regional￾level responsibilities of the RS as assisted by the regional program. 7. NOAA should be more fully engaged in country and regional work plans, and the training and services it provides should be explicitly tied to country and site needs and be better communicated. NOAA contributions should be included in work plan development of each program from the design stage. NOAA should also consider using more local/regional expertise in providing its training. While there was near-universal appreciation for the quality of training and other technical work of NOAA and as valuable expert members of TWGs, there was little understanding among participants, or US CTI Evaluation Report 43 even by the Evaluation Team, of the way in which the NOAA work was integrated into the rest of the US CTI program. COUNTRY-LEVEL RECOMMENDATIONS 8. USAID should implement fieldwork in the CTI countries through the bilateral Missions to carry out complementary programs with coordination by RDMA with or without a regional program. While there may have been some theoretical advantages to having a single program across the CT6 countries at the country level, the Evaluation Team believes that they were ultimately outweighed by the management and overhead burden. The level of similarity of programs and expected direct comparability across sites, whether considered “integrated” or “demonstration,” was never achieved, with programs as different as PNG and Indonesia. This difference and the desire to have programs better reflect the realities of each country led to the withdrawal of Indonesia to set up its parallel marine program. We recommend that the Philippines and Timor-Leste Missions also be given control of their programs and that the new USAID Pacific Mission be given responsibility for the PNG and Solomon Island programs. Malaysia could be supported directly by RDMA. Commonalities should be maximized as much as possible from nomenclature (US CTI –PNG, etc.) to program activities, RFs, PMPs and indicators. (See Recommendations below.) 9. In subsequent country programs, USAID should allow a broader range of interventions and result areas to be carried out through the program, including other RPOA/NPOA goals (besides the three of US CTI 1), as long as the choices are arrived at through a consultative process with governments, NGOs and other stakeholders are evidence-based and aligned with the program's SOs and country’s priorities. Also, programs should be periodically re-analyzed and emerging priorities or needs addressed and resources reallocated to these. In any subsequent program, USAID should ensure that alternative (or sustainable) livelihood development is strongly considered among the areas of interventions from the time of program design. Such alternative or sustainable development work should also be linked to other USAID Economic Growth programs. 10. In sites where fieldwork is implemented to date, the goal should be promoting consolidation and demonstrating sustainability before expansion or scaling up to more sites. The Evaluation Team believes that the amount of resources available is unlikely to permit the expansion of the program to additional sites and that there would be significant loss of effectiveness by spreading these too thinly. Given that the programs at the existing sites are not “finished” (Nino Konis Santana is not protected, Tun Mustapha Park is not gazetted, Bali MPA Network is not complete, etc.) the first objective should be demonstrating actual improvements on the ground, building on the base of investment in US CTI rather than expanding to new areas. 44 US CTI Evaluation Report ASSISTANCE MODEL RECOMMENDATIONS 11. For country programs, USAID should consider a range of vehicles/mechanisms beyond only a consortium of international NGOs (as structured in US CTI 1). Use of the WWF-led NGO consortium in the US CTI Support Program did not provide anticipated efficiencies in management. Our observations are that there was neither an efficiency gained for RDMA management nor an efficiency for the NGOs themselves. While we do recommend not using a consortium of NGOs structured as in US CTI 1, we clarify that we do not recommend barring NGOS (or even consortia of NGOs) from future programs where they may be appropriate and most effective. NGOs are thought by most stakeholders to bring certain advantages because of their missions and their orientation, their long-term presence in the countries, their perceived non-political natures, etc. They are recognized for their experience and technical capabilities and may well be more effective for site-based work. Thus, NGOs may continue to have a central role but not as previously configured nor necessarily as the default choice for implementation. In several of the CTI countries, more than one international NGO is working on the ground. Should a coalition or consortium of NGOs or any other organizations be awarded the work in any country, the internal structures should be clearer than in US CTI. (See Recommendations on Design below). By extension, the use of contracts for country-level work should also be considered where appropriate. In particular, while they may be less effective than NGOs at sites (not conclusively proven but widely asserted), they may in fact be more effective at working with governments, NCCs and other stakeholders at other levels. Additionally, the nature of the contractual relationships with organizations will likely be more responsive to Mission design as well as more financially transparent. PROJECT DESIGN, IMPLEMENTATION AND MANAGEMENT RECOMMENDATIONS 12. In any subsequent program, USAID should ensure that the design is based on a sound development hypothesis that addresses the issues in the CT and that all program implementers and stakeholders share a consistent understanding of the goals, objectives and approach and that each implementing member be fully committed to the objectives of the project. Lack of an accepted, articulable development hypothesis in US CTI led to programs that were dissimilar and disconnected across countries and led implementers, stakeholders, participants and cooperators at all levels to be unable to say what the program was trying to accomplish. This lack of cohesion leads to deviation of attention and dispersal of resources into other less effective or important areas and may also lead to a lack of understanding and appreciation for the central role of USAID in countries and among beneficiaries. 13. In any subsequent program, USAID should require a clear and logical RF from future program implementers within 90 days of award date. The RF should have expected program outcomes and outputs, intermediate results and activities and, based on this RF, the implementers should also be required to develop a comprehensive M&E Plan, with indicators at each level and a clear plan for how monitoring data will be used for performance management, not just reporting to Washington. Further, USAID should accept that designing a robust and comprehensive M&E system requires expertise and resources, and allow a significant share of project funding to be devoted to this. Particularly for projects of this size, complexity and uncertainty, USAID should build evaluability into project design in order to facilitate more rigorous investigation of program effects and causal questions. US CTI Evaluation Report 45 It is true that programs for policy change, organizational strengthening and similar are less easily evaluated for impact than are those that aim for change on the ground since the indicators and baselines are less clearly applicable/easily developed. However, relevant indicators and their baselines should be created so that later qualitative and quantitative evaluation is better supported. Further, for the site-level work, relevant indicators must be developed. Physical changes in conditions such as fish stocks, species composition, reef cover, etc. are both long term and not susceptible to changes in a few short years and, further, they are influenced by a range of factors outside of the control of any five-year development program. Nevertheless, indicators for these should be developed and baselines set so that the work can eventually be evaluated more quantitatively. Moreover, IRs must be identified and tracked to facilitate performance management. Indicators for reporting to Washington (high-level F indicators) are rarely useful for management, and relevant management indicators must be employed. “Number of hectares (in millions)” may be appropriate for Washington, but for actual MPA implementation, “Number of mooring buoys placed or serviced per quarter,” for example, is more appropriate. 14. In any configuration, but especially when working through a consortium or other multi￾organization structure, USAID should ensure a clear, efficient system for work planning, budgeting and coordination. This is a must for any program but especially for one that is complicated such as US CTI. The simple process of decentralization from the regional RDMA to the bilateral Missions will ensure that each project will have fewer layers but in fact clarity and efficiency should be explicit goals of the design. The related experience gained by US CTI implementers would be of immense value in subsequent programs if fully applied. OTHER RECOMMENDATIONS 15. USAID should continue involving its staff in evaluation fieldwork. The Evaluation Team found clear benefits from the participation of USAID staff members, particularly when the staff had expertise in the technical area under study. Their contributions, inputs, discussions, perspectives and knowledge of the people involved in the sector were useful. Staff should have clearly defined roles, availability and commitment from the beginning to allow them to fully engage in the evaluation. At a minimum, participating technical staff should receive some training in evaluation and, to maximize speed of comprehension, should have sufficient time to prepare and familiarize themselves with the program under scrutiny. The Evaluation Team believes that, for the participating staff, the field experience gained will be of significant benefit in their preparation of SOWs for future evaluations as well as in setting appropriate expectations for products, timeframes and other aspects. 46 US CTI Evaluation Report ANNEXES US CTI Evaluation Report 47 ANNEX 1: EVALUATION STATEMENT OF WORK FINAL PERFORMANCE EVALUATION OF THE U.S. CORAL TRIANGLE INITIATIVE SUPPORT PROGRAM (US CTI) I. BACKGROUND INFORMATION A) Identifying Information Program: U.S. Coral Triangle Initiative Support Program (US CTI) Activity Title Coral Triangle Support Partnership (CTSP) US CTI Program Integrator (PI) US CTI NOAA PAPA Award Number CA-486-A-00-08-00042 TO-EPP-I-07-06-00008-00 486-T-00-09-00016-00 Award Dates September 30, 2008 to September 29, 2013 September 3, 2008 to September 2, 2013 April 28, 2009 to April 27, 2014 TEC/Funding $32,000,000 $10,125,852 $3,100,000 Implementer World Wildlife Fund Tetratech/ARD NOAA C/AOR/Alternate Rene Acosta/ Mikell O’Mealy Rene Acosta/ Mikell O’Mealy Rene Acosta/ Mikell O’Mealy USAID/RDMA leads the US CTI and provides overall program coordination, working closely with bilateral USAID Missions in the Asia-Pacific region. The program is implemented by the Coral Triangle Support Partnership (CTSP), a five-year cooperative agreement awarded to the World Wildlife Fund (WWF), which serves as the prime for a NGO consortium that includes The Nature Conservancy (TNC) and Conservation International (CI). The National Oceanic and Atmospheric Administration (NOAA) provides leadership on capacity building, technical assistance and training as part of US CTI implementation. The U.S. Department of State serves as the lead U.S. Government agency for advancing diplomatic interests in the CTI. TetraTech/Associates in Rural Development (TetraTech ARD) serve as a Program Integrator (PI) under a task order contract to provide regional support, technical assistance and small grants and to support the U.S. program team. B) Development Context 1. Background and USAID’s Response The Coral Triangle is an area encompassing almost 4 million square miles of ocean and coastal waters in Southeast Asia and the Pacific surrounding Indonesia, Malaysia, Papua New Guinea, the Philippines, Timor-Leste, and the Solomon Islands (Figure 1). The area is home to some 363 million people and encompasses economic zones in which each country has exclusive rights to marine resources. Recognized as the global center of marine biological diversity, the region serves as the spawning and juvenile growth area for five species of tuna, comprising the world’s largest tuna fisheries. The biological resources of the Coral Triangle directly sustain more than 120 million people living within this area and benefit millions more worldwide. The natural environment of the Coral Triangle and the many essential goods and services it provides is at risk, however, due to a range of factors, including over-fishing, destructive fishing practices, land and ocean based sources of pollution, and climate change. These factors have a negative impact on the food security, employment opportunities, and living standards of people dependent on marine and coastal resources for their livelihoods. 48 US CTI Evaluation Report Figure 1. The Coral Triangle provides resources that directly sustain 120 million people living within the area and millions more worldwide. In May 2009, the leaders of the six Coral Triangle (CT6) countries met for a summit in Manado, Indonesia and signed the declaration launching the Coral Triangle Initiative on Coral Reefs, Fisheries and Food Security (CTI-CFF) and endorsed its Regional Plan of Action (RPOA). The event marked the culmination of a process launched by Indonesian President Yudhoyono in 2007 and supported by the United States to bring the CT6 countries together to preserve the threatened marine, coastal, and small island ecosystems of the Coral Triangle region. The CT6 countries have committed to implement the five goals of the CTI Regional and National Plans of Action (NPOAs), which present clear objectives and targets for achieving local, national and regional outcomes within 10 to 15 years. These five CTI goals are: Goal 1: Priority Seascapes Designated and Effectively Managed Goal 2: Ecosystem Approach to Fisheries Management (EAFM) and Other Marine Resources Fully Applied Goal 3: Marine Protected Areas (MPAs) Established and Effectively Managed Goal 4: Climate Change Adaptation Measures Achieved Goal 5: Threatened Species Status Improving Regional commitment and cooperation occurring within the context of the CTI has been expected to catalyze, expand, and sustain national and local-level implementation efforts through the implementation of the RPOA. Within the broad framework of the RPOA, the CT6 countries have developed national strategies and action plans, as well as worked together to identify and implement actions that require regional cooperation. The CTI thus encompasses a distinctively regional approach, building on country-driven priorities and actions. Management of the CTI is led by a Regional Secretariat based in Manado, Indonesia and is expected to be made permanent in 2013. An interim Regional Secretariat based in Jakarta, Indonesia is currently leading CTI activities. US CTI Evaluation Report 49 The overall goal of the US CTI is to improve the management of biologically and economically important coastal-marine resources and associated ecosystems that support the livelihoods of peoples and economies in the Coral Triangle. 2. Target Areas and Groups Activity levels: The US CTI Team works at multiple activity levels (Figure 2) and in priority geographies throughout the Coral Triangle region. Activities at both regional and national levels mutually reinforce each other. Regional activities are designed to promote information exchange, galvanize international cooperation to address transnational or regional issues, and share lessons among CT6 countries. National activities are focused on the implementation of NPOAs. They support policy reform and institutional capacity development that provide foundations for effective regional cooperation. The US CTI is not a collection of national programs, but rather a regional program designed to address regional priorities and achieve impacts at the regional level. Figure 2. US CTI activities at both regional and national levels reinforce each other. Local activities are designed to demonstrate best practices in priority geographies and specific sites. Priority geographies are defined as areas where biological and socio-economic factors are of such importance that they merit focused marine management attention. Similar sets of activities in priority geographical areas roll up to national and regional activities that generate regional results. The results achieved at the regional level, including increased technical capacity and learning, will support national and site-level activities in targeted geographic areas of priority value. The overriding vision is to achieve a fully integrated approach to coastal and marine management that delivers multiple benefits. Priority geographies: The US CTI is designed to achieve results on the ground. Priority geographies (Figure 3) were delineated at a scale at which the US CTI can have a measurable, positive impact on marine resource management. As such, the geographies may encompass seascapes, corridors, networks of MPAs, local government jurisdictions or individual MPAs. Priority geographies were considered across the Exclusive Economic Zones (EEZs) of each country while emphasizing biodiversity considerations. The selection of priority geographies was also guided by criteria used to consider the existing condition of the resource base, the foundation of enabling factors, and the feasibility of conservation work in candidate locations. Typically, priority geographies include one or more specific sites, which are smaller in scale to focus targeted field efforts to improve management and resource conditions. The US CTI established at least one target site among its priority 50 US CTI Evaluation Report geographies in each CT6 country to demonstrate integrated learning and achieve outcomes across all results areas, in partnership with local and national governments and stakeholders. Financial and human resources are focused on these target sites to maximize program impacts. Figure 3. Priority geographies of the US CTI. C) Approach and Intended Results US CTI implementation is guided by a Results Framework (Figure 4) aligned to the goals of the CTI RPOA. The framework was developed to focus and align the activities of all the US CTI partners on measurable development outcomes that can be achieved with available program resources. US CTI Evaluation Report 51 Figure 4. Priority geographies of the US CTI. The US CTI Results Framework consists of four results statements that capture the outcomes and impacts of program activities over the life of the program. Each result includes a set of more specific Intermediate Results (IRs) that, when implemented, will collectively contribute to meeting the broader result. By focusing on these results, the US CTI is strategic in targeting its resources both geographically and programmatically. The two CTI RPOA goals not specifically addressed by the US CTI include Goal 1 (Priority Seascapes Designated and Effectively Managed) and Goal 5 (Threatened Species Status Improving), although elements of these two goals are addressed at the country level. In Year 4, by applying adaptive management and focusing on activities with the highest potential for impact, USAID/RDMA and US CTI implementing partners created an “Endgame Strategy” to guide US CTI mechanisms in Years 4 and 5. This Strategy specifies the outcomes that US CTI aims to achieve by 2013 for each result statement, as well as the US CTI lead and supporting partners for each outcome to enable efficient resource allocation and integration among partners. The Endgame Strategy outcomes, lead, and partner roles are detailed in Table 1 below. The CTSP was also subject to a USAID Regional Office of Inspector General (RIG) Performance Review in Year 4. Recommendations from that Review focused primarily on Monitoring and Evaluation and are being applied across the CTSP program. IR1.1 Policies developed and advanced IR1.2 Institutional capacity and collaboration strengthened IR1.3 Learning and information networks strengthened IR1.4 Public and private sector constituencies engaged IR1.5 Sustainable financing mobilized CTI Goal 4 CC adaptation measures achieved Strategic Objective Improved Management of Biologically and Economically Important Coastal and Marine Resources and its Associated Ecosystems that Support the Livelihoods of Peoples and Economies in the Coral Triangle CTI Goal 2 EAFM and other marine resources fully applied R2. Ecosystem approach to fisheries management improved in CT countries IR2.1 EAFM framework developed and endorsed IR2.2 Fisheries management capacity increased IR2.3 Enforcement capacity increased IR2.4 EAFM applied in priority geographies CTI Goal 3 MPAs established and effectively managed R3. Marine protected area management improved in CT countries IR3.1 MPA System framework developed and endorsed IR3.2 MPA management capacity increased IR3.3 MPA effectiveness improved in priority geographies IR4.1 Capacity to apply climate change adaptation strategies increased IR4.2 Climate adaptation strategies applied in priority geographies R4. Capacity to adapt to climate change improved in CT countries R1. Regional and national platforms strengthened to catalyze and sustain integrated marine and coastal management in the Coral Triangle Program Indicators PM1. No. of hectares in areas of biological significance under improved management PM2. No. of hectares under improved natural resource management as a result of USG assistance PM3. No. of policies, laws, agreements or regulations promoting sustainable natural resource management and conservation that are implemented PM4. No. of people receiving USG supported training in natural resources management and biodiversity conservation PM5. No. of laws, policies, agreements, or regulations addressing climate change proposed, adopted, or implemented as a result of USG assistance PM6. No. of public-private partnerships formed REGIONAL COLLABORATION INTEGRATION 52 US CTI Evaluation Report Table 1: US CTI Endgame Strategy. CTI Secretariat and Coordinating Structures Established to Sustain Impact  Limited support provided to the CTI Regional Coordinator (Program Integrator (PI))  Support provided to evaluate progress towards the goals identified in the Regional Priorities Workshop (PI)  NCCs and Thematic Working Groups (TWGs) (e.g., MPAs, EAFM, CCA, Monitoring and Evaluation & Sustainable Finance) mobilized and strengthened (PI, CTSP)  Coordination support provided to US Government Agencies and CTI Development Partners (PI)  Communications support provided to the CTI Secretariat (PI)  Writing support provided for State of the Coral Triangle Report and CTI Annual Report (PI, CTSP)  Regional learning network established, initially through the Coral Triangle Center (i.e., work with TWGs and other institutions to collect and share tools, case studies, curricula, and other knowledge products through an interactive online portal) (PI)  CTI Local Governance Alliance mobilized to catalyze local government political will (PI, CTSP) EAFM Regional Framework Adopted and Guidelines Developed for National Operationalization  EAFM regional framework and 10-year road map formulated (CTSP, PI, NOAA)  National EAFM operational guidelines developed (CTSP, PI, NOAA)  Legal analysis for EAFM and Illegal Unreported, and Unregulated Fisheries completed and shared (CTSP, PI, NOAA)  Limited support provided to establish the Live Reef Fish Trade (LRFT) international standards (PI)  Regional exchange to inform design and build support for adoption of EAFM regional framework (PI, CTSP, NOAA)  Comprehensive integrated toolkit, case studies, and curricula for MPAs, CCA, and EAFM developed and disseminated and trainings delivered (i.e., CCA, Management Plan, Adaptive Management, Monitoring & Evaluation (M&E), and Enforcement) (CTSP, PI, NOAA)  Support EAFM TWG to get government adoption (CTSP, PI) Coral Triangle MPA System (CTMPAS) Developed and Adopted  Analysis of connectivity and climate impacts to inform MPA design completed (CTSP)  Regional workshops held for MPA TWG to design and adopt the CT MPA System (CTMPAS) (CTSP, PI, NOAA)  MPA Effectiveness Protocol developed and adopted to inform resilient MPA design (CTSP)  MPA integration sites finalized and linked to CTMPAS, and MPA M&E protocol implemented (CTSP) US CTI Evaluation Report 53  Regional exchange established to inform design and build support for adoption of the CTMPAS (PI, CTSP, NOAA)  Comprehensive integrated toolkit, case studies, and curricula for MPAs, CCA, and EAFM developed and disseminated and trainings delivered (CTSP, PI, NOAA) CCA Regional Early Action Plan (REAP) Adopted  Comprehensive integrated toolkit, case studies, and curricula for MPAs, CCA, and EAFM developed and disseminated and trainings delivered (PI, CTSP, NOAA)  Regional exchange established on REAP implementation and lessons learned, and to refine early actions (PI, CTSP)  Priority REAP Actions initiated (CTSP, PI, NOAA)  Local Early Action Plans (LEAP) developed to validate and apply REAP recommendations (CTSP)  Support CCA TWG to get government adoption D) Implementation Program Coordination: The US CTI places significant attention on coordinating its plans and activities with all stakeholders. Consultations with regional and national CTI bodies and CT6 countries occur regularly to ensure the program is directly supporting the CTI RPOA and NPOAs. Regular coordination also occurs with other partners providing support to the CTI to minimize overlap and highlight gaps in assistance that need to be addressed. These coordination mechanisms serve not only to optimize impact among all partners and stakeholders, but also to facilitate and ensure a unified U.S. government response to the CTI that is technically sound, culturally appropriate, and gender-sensitive. The US CTI program employs the following coordination tools and mechanisms:  Dialogue with regional and national CTI bodies and CT6 countries: The US CTI conducts regular consultations with regional and national CTI bodies and CT6 countries to ensure that program activities are aligned to support the achievement of CTI goals and targets.  Collaboration among U.S. government partners in the region: The US CTI has established coordination mechanisms designed to ensure U.S. partners are effectively communicating with each other and tracking the status of U.S. activities in the region. These includes monthly Program Coordination Conference Calls; bi-monthly Core Team Conference Calls; consolidated Annual Work Plans for US CTI team members; a consolidated Performance Management Plan; consolidated monthly, annual, and semi-annual Progress Reports; a US CTI Partner Portal; and an Annual U.S. Partner Coordination Meeting.  Coordination with non-U.S. CTI partners: Many CTI-related programs are being developed by organizations, such as the Asian Development Bank (ADB)/Global Environment Facility (GEF), Australian Government and country-specific bilateral donor agencies. Donor coordination efforts are conducted through meetings, dialogue and participation in regional and national donor forums, often linked to scheduled CTI government meetings. The Program Integrator maintains a Donor Project Matrix that tracks donor activities by country and by theme. The US CTI Partner Portal is used as a platform to support information sharing, online real-time planning, and collaboration among partners supporting the CTI. 54 US CTI Evaluation Report  Annual review and adaptation: The US CTI conducts annual dialogues with CTI bodies and CT6 countries as well as other non-U.S. partner programs to evaluate and adapt the five-year strategy and guide annual programming. Annual U.S. partner planning workshops are conducted to ensure a unified approach to implementation. Implementation methods are thoroughly detailed in the US CTI implementing partners’ task order, interagency agreement, cooperative agreement, annual work plans and semi-annual reports. E) Existing Resources and Documents The following resources are publicly available for review:  US CTI Website – www.uscti.org  CTI-CFF Official Website - www.coraltriangleinitiative.org The following documents will be made available upon award:  PI Task Order  CTSP Cooperative Agreement  NOAA Participating Agency Program Agreement  CTSP, NOAA, and PI Annual Work Plans  US CTI Performance Management Plans  US CTI Consolidated Work Plan  CTSP, NOAA, and PI Semi-Annual Reports  US CTI Midterm Evaluation Report  OIG Performance Audit Report  US CTI Quarterly and Weekly Updates  US CTI supported CTI-CFF Publications  US CTI/CTI-CFF Workshop and Conference Proceedings  Inventory of US CTI Tools and Products  Scientific Papers, Journals and Articles on CTI from non USAID sources  Documents for other CTI Development Partners, i.e. WWF, TNC, CI, ADB/GEF, UNDP, UNEP, FAO, GIZ, Australian Government II. EVALUATION RATIONALE Evaluation Purpose and Objective The Contractor must achieve the purpose of the final performance evaluation of the US CTI program which is to evaluate the performance of the program, as well as test the foundational assumptions inherent in the US CTI Results Framework (see Figure 4). Specifically, the Contractor must conduct the evaluation to investigate the effectiveness of the program’s key elements – the Results and Intermediate Results – in achieving the Strategic Objective of the Results Framework and the CTI-CFF Goals. The evaluation’s findings and conclusions will provide insights to guide the development future USAID programs and the design of effective results frameworks to maximize development outcomes. Evaluation Questions The Contractor must design the evaluation to answer this primary question:  How effective has the US CTI’s regional collaboration and integration approach been in achieving its Strategic Objective of “improving management of biologically and economically important coastal and marine resources in the Coral Triangle” as applied in the US CTI Results Framework? US CTI Evaluation Report 55 The Contractor must also address the sub-evaluation questions:  What specific lessons have been learned about the validity of the program’s development hypothesis and results frameworks?  How effective have US CTI actions under Intermediate Results been in achieving corresponding Results?  How effective have US CTI results been in contributing to the accomplishment of corresponding CTI-CFF Goals, strategic objectives and targets and the US CTI Strategic Objective?  What key challenges, either technical or managerial, have been encountered and innovative practices applied in achieving the US CTI Strategic Objective, thereby contributing to the CTI-CFF Goals? The Contractor should use these questions to help examine gender equality issue in the program design and implementation. Audience and Intended Uses The results of the evaluation will be used primarily by USAID Washington E3, Asia regional and bilateral missions and other U.S. government agencies such as NOAA, and USDOI, CTI-CFF Stakeholders and CTI-CFF Development Partners to inform the development of ongoing and future support to help accomplish CTI-CFF Goals and targets.3 III. EVALUATION DESIGN AND METHODOLOGY Evaluation Design The Contractor’s evaluation design must be non-experimental. Consistent with ADS 203.3.1.6 guidance on evaluation methodologies, a combination of quantitative and qualitative methods in data collection and analysis must be employed by the Contractor in the process. A triangulation method should be applied by the Contractor to increase level of validity in data collection and processing of results. Where appropriate and applicable, a purposive stratified sampling for data collection should be employed by the Contractor. The Contractor’s consultants is responsible for conducting the evaluation and in developing a detailed evaluation design including finalizing key and sub-questions, identifying sample size or the survey and interviews, and finalizing data collection and analysis methods to measure the efficacy of the US CTI Results Framework as applied in the context of a highly adaptive and complex nature of the CTI-CFF processes. The independent external consultants are to work in conjunction with other team members to plan and implement the proposed evaluation. USAID/RDMA and the full evaluation team will be involved with design, planning, and logistics, but the Contractor is required to provide significant and overall leadership and direction, as well as having the final responsibility for the major evaluation duties and deliverables. 3 See CTI-CFF Regional Plan of Action: www.coraltriangleinitiative.org/library/cti-regional-plan-action. 56 US CTI Evaluation Report Data Collection and Analysis Methods The Contractor is required to evaluate this multi-faceted program over a period of 15 weeks and across the CT six-country. Considering resources limitation, case study can be applied by using purposive approach to select one representative priority geography from each of CT6) to represent all geographic areas in each CT6 country. Data requirements, collection methods, and required analyses will be determined by the Contractor in collaboration with USAID/RDMA and under the direction of the Contractor’s team leader (not affiliated with USAID or the program) to reduce bias and promote a high quality evaluation under the USAID Evaluation Policy. Details on final datasets, collection methods (including interview questions and key informants to be interviewed), and analytical framework(s) will be approved by the TOCOR as part of inception report submission. The Contractor must disaggregate by sex, target country, and regional/national. Proposed data collection and analysis methods are provided in the table below. The Contractor must begin its data collection with a desk study of existing documents and information, followed by consultations with key stakeholders in Washington and Asia region to further refine the implementation approach. This will be followed by interviews of partners, stakeholders, and beneficiaries in the program’s target countries. Details on these illustrative approaches are provided in the table below. Data Collection Methods Analysis Methods Desk study: Review documents and databases listed in section 1.E. above. Work with USAID/RDMA to acquire additional documents as needed, and prioritize primary data collection where gaps remain. This method also serves as planning role. Documentation, Coding, Conceptualization, Categorization, Corroboration, Internal Consultations: Meet or conference call with key stakeholders in Washington and Southeast Asia for recommendations on specific areas of consideration. These may include but are not limited to: (a) USAID/RDMA, Asia Bureau, E3 Bureau, relevant USAID bilateral missions; and (b) USG Federal Agency partner staff including the NOAA, DOI and DOS. This is separate from the interview process by which data will be collected among some of the same stakeholders. This method also serves as planning role. Documentation, Coding, Conceptualization, Categorization, Corroboration, Survey: Develop an online questionnaires survey instrument in English that responds to the evaluation objectives. Distribute the survey widely (using email or Internet) for breadth and reach, especially to stakeholders that will not be visited by evaluation team members. Respondents will be 100% of strata or group universe of CTI-CFF contacts from email lists and database, e.g. a) internal and external to CTI-CFF and b) direct and indirect program participants from regional national, local/community covering CT6. These include but not limited to: staff and officials of the CTI Interim Regional Secretariat, government participants and representatives to the different thematic working groups, training participants from different countries in regional trainings and exchanges, National CTI Coordinating Committee members, local government officials in priority geographies, fisherfolks, communities in priority geographies, and donor partners collaborating with CTI. By the online survey service provider, or analyzed by any other appropriate software such as SPSS or MS-Excel. The data should be summarized and presented in graphic, descriptive, table and/or numeric formats. Key informant and focus group interviews: In-person and phone interviews should be conducted based on an interview guide developed explicitly for this evaluation. Stratified purposive sampling (regional, national and local) will be determined. Target informants include but not limit to representatives of the case study areas stakeholders, i.e. local and national government officials, NGO representatives, local beneficiaries (including women, youth, fisherfolks, and donor partners collaborating with CTI, etc.). Individuals and organizations in the CT6 countries should be prioritized based on missions and other stakeholder consultation, drawing from the types of stakeholder outlined above. The decision on whether to conduct an Transcription, Coding, Conceptualization, Categorization, Corroboration US CTI Evaluation Report 57 individual or group interview depends on a variety of factors including the type of questions and analyses planned, individual and cultural norms and preferences, and efficiency. A) Methodological Strengths and Limitations Methods Strengths Limitations Desk Study  Provide valuable information on substantive issues and generate a list of questions including key stakeholders that can be used in other methods.  Help to focus efforts and prioritize issues and gaps  Time consuming  Depends on resource availability  Lack of consistent data collection  Limited baseline data Consultations  Provide valuable information on substantive issues and generate a list of questions including key stakeholders that can be used in other methods.  Provide greater depth and insights and general surveys  Depends on availability of key stakeholders  Need to consider time zone difference.  Quality/reliability of data Survey  Cost and time effective  Can reach more respondents  Sample size and number of respondents may not be enough to statistically represent the whole population.  Limited reach if only conducted in English Key Informant Interviews  Potentially data rich, detailed answers  Might need to interview through translators (possible loss of meaning and data richness)  Might have informants’ bias Focus Group Interviews  Can generate broader ideas and responses.  Can include a greater number of participants in less time and result in rich discussion, if facilitated well  Might need to conduct discussion through translators (possible loss of meaning and data richness)  Some respondents may dominate in answering  Requires two interviewers IV. TEAM COMPOSITION The evaluation team must be comprised primarily of three (3) independent external consultants, as follows: 1) Team Leader (international consultant) 2) Assistant Team Leader/Evaluation Specialist (international or local consultant) 3) Evaluation Specialist/Administrative and Logistical Support (local consultant) The evaluation will be led by the “Team Leader” and supported by the abovementioned subject matter experts. The Team Leader with an Institutional and/or Regional Governance Specialist background and expertise will be primarily responsible for the overall implementation of the 58 US CTI Evaluation Report evaluation and ensuring that all expected tasks and deliverables are achieved on time and of high quality. S/he should have at least fifteen years (15) of professional experience coordinating similarly complex evaluations, and leading evaluation teams. The candidate must have exceptional organizational, analytical, writing and presentation skills. S/he must be fluent in English and must have a master’s level degree and/or technical knowledge and experience in a relevant analytical field (e.g. international development, organizational development, regional governance, natural resource management, coastal-marine and/or fisheries resources management), although doctorate level credentials are preferred. It would be highly desirable for the Team Leader candidate to have direct knowledge and/or experience working with USAID rules, regulations, and procedures, particularly requirements of Biodiversity conservation programs. S/he will oversee the overall drafting of the evaluation framework, including methodology determinations; organization of calendar/travel/ meetings; overseeing the desk study, interviews, and other data collection; and analyzing the data with input from team members and USAID/RDMA to draft the evaluation report. The Assistant Team Leader/Evaluation Specialist background will support the team leader in the implementation of the evaluation. S/he should have at least ten years (10) of professional experience implementing similarly complex evaluations involving multiple stakeholders. The candidate must have exceptional organizational, analytical, writing and presentation skills. S/he must be fluent in English and should have a master’s level degree and/or technical knowledge and experience in a relevant analytical field (e.g. international development, organizational development, regional governance, and coastal￾marine and/or fisheries resources management). It would be highly desirable to have knowledge and/or experience working with USAID rules, regulations, and procedures, particularly requirements of Biodiversity conservation programs. S/he will contribute to the overall drafting of the evaluation framework and participating in the desk study, interviews, and other data collection; and analyzing the data with input from team members and USAID/RDMA to draft the evaluation report. The Local Evaluation Specialist/Administrative and Logistical Support will provide additional technical support to the evaluation team as well as support administrative and logistical functions necessary to carry out the evaluation. S/he must be a national or local expert from the Coral Triangle region, and have strong organizational skills. S/he must have strong English speaking skills and a master’s level degree with at least 10 years of technical knowledge and experience in a relevant field (e.g. program management, project evaluation, natural resource management, international development, organizational development, regional governance, and coastal-marine and/or fisheries resources management). S/he will be responsible for assisting in coordinating the desk study, interviews, and other data collection, and providing overall administrative and logistical support to the team. The Contractor must field an evaluation team that provides complimentary skills and together possesses the technical, evaluation and managerial skills to submit high quality deliverables that meet the objectives of the task order without requiring significant revisions and substantive/significant input from the TOCOR and additional team members. If it is difficult to find team members who have both evaluation and technical skills/experience, then the Contractor may field a team composed of an experienced evaluator as team leader with technical experts on the team itself could be considered. The Contractor’s consultants will be supervised by the TOCOR, while working closely with the US CTI program’s Agreement Officer Representative (AOR) to gain in-depth information of the program activities. The TOCOR and/or alternate will provide strategic direction and guidance throughout the evaluation process, including the development of the work plan, any data collection tools, and evaluation report outline, approach, and content. These team members will provide complimentary technical assistance and assist in the overall evaluation implementation. US CTI Evaluation Report 59 In addition to the Contractor’s consultants, the evaluation team may be complemented by additional team members from USAID and other organizations as follows: 4) Sustainable Fisheries and Ecosystem Based Management Specialist (USAID/Washington or other USG Agency) 5) Integrated Coastal Resources Management Specialist (USAID/Washington or other USG Agency) 6) Monitoring and Evaluation Specialist and/or Program Development Specialist (USAID/RDMA) The exact participants will be confirmed with the evaluation team following award. It is expected that 2- 3 additional team members will be able to participate for a period of 2-3 weeks each, and focusing on assisting in conducting consultations and overall programmatic strategic review. V. EVALUATION MANAGEMENT The Contractor’s evaluation team is expected to visit a sampling of the CT member countries in which the US CTI Support Program is active, representing countries from both the eastern and western sides of the CT region. The evaluation team is also expected to visit the USAID/RDMA office in Bangkok, Thailand. The evaluation team will receive support from USAID/RDMA in selecting priority organizations and places to visit during the evaluation, and in gaining required country clearance. The evaluation team is expected to schedule interviews or other modes of data collection with key stakeholders, and USAID and US CTI team may provide contact information. The Contractor is responsible for making their own hotel, air travel, and local transportation arrangements in accordance with U.S. requirements for allowable carriers and per diems. Evaluation team members should have the necessary language skills for countries of focus, or engage local language interpreters to support interviews and reviews of local language documents and records. 60 US CTI Evaluation Report ANNEX II: DATA COLLECTION INSTRUMENTS PROTOCOLS I. LOCAL LEVEL LOCAL GOVERNMENT STAKEHOLDER KII MASTER PROTOCOL Overall Approach (Question 1)  What do you see as the primary issues being addressed through the CTI?  Do you think these issues are more effectively addressed in a regional or national/local manner?  Do you think the US CTI program has taken a regional approach to addressing these issues?  If so, in what ways has the approach been regional?  If so, have you noticed any challenges arising from this regional approach?  To what extent do feel that the program approach has been integrated, both vertically and horizontally?  NOTE: We may need to state how US CTI has defined integration first.  What do you see as the principal outcomes that US CTI seeks to achieve?  The US CTI Strategic Objective is “improving management of biologically and economically important coastal and marine resources in the Coral Triangle”. What do you think that means in practice? Or, how do you think that would be measured?  To what extent do you feel the approach taken by US CTI has been successful in achieving this strategic objective, on a scale of 1 to 5 (1=not at all, 2=minimally successful, 3=neither successful nor unsuccessful, 4= moderately successful, 5=very successful)?  To what extent do you think the level of success (or lack of success) can be attributed to the stated regional and integrated approach of US CTI?  Do you think the same level of effects could have been achieved through programs focused at the national or sub-national level?  Do you have any recommendations on how the program could improve its approach to regional collaboration or integration?  Project Design (Question 2)  What do you see as the main constraints to achieving better management of coastal and marine resources?  US-CTI has focused on 4 main results areas: regional and national networks and platforms, EAFM, MPA management, and climate change adaptation. Do you think these are sufficient to achieve better management of coastal and marine resources?  Are there other important areas that you think need to be addressed in addition to these?  On a scale of 1-5, to what extent do you think the US-CTI focus on these 4 result areas is adequate for achieving improved management (1=completely inadequate to 5=perfectly adequate)?  What do you think are the critical factors that need to be addressed in order to:  Develop stronger regional and national policy platforms or networks for marine and coastal management?  Improve EAFM practices?  Improve MPA management? US CTI Evaluation Report 61  Improve adaptability to climate change? Project Implementation (Question 3 and Question 5) (If respondent is not familiar with the CTI program this section or sub-questions can be skipped.)  On a scale of 1 to 5 (1 being not effective at all and 5 being very effective), how effective do you think US CTI actions have been towards the following objectives, and why?  Develop stronger regional and national policy platforms or networks for marine and coastal management?  Improve EAFM practices?  Improve MPA management?  Improve adaptability to climate change?  What have been the most notable achievements that you think were generated, at least in part, by US-CTI?  Which activities do you think have been most and least effective at achieving results at each of the following levels:  Local  National  Regional  Have you noted any specific examples of activities or approaches implemented in your country that were developed on the basis of activities in other CT6 countries?  If so, how did the idea or approach transfer from the other country (e.g. through a regional exchange or workshop or guidance document, etc.)?  What were some of the key challenges you noted in implementation of the US CTI?  Did you notice any steps taken to address each of these challenges? If so, what were they?  If these challenges are still relevant, how do you think they could be best addressed?  Were there any aspects of the US-CTI approach or implementation that you found particularly innovative? If so, what were they?  Do you have any suggestions for how CTI could innovate moving forward to more effectively achieve its objectives?  Were the member NGOs of the consortium perceived as partners by all the C6 countries? (Do you perceive the NGO consortium as partners?)  Do you think implementing the US-CTSP program through a consortium of NGOs was the right approach? Why or why not?  Would channeling the resources given to NGOs to the national governments be a better option?  What advantages or disadvantages do you think it would it have brought to achieving the objectives of the CTSP? Coordination with CTI-CFF Goals (Question 4)  The US CTI program had goals supporting RPOA Goals 2-4: were Goals 1 and 5 adequately addressed in other activities or are these of lower importance compared to 2-4?  In a scale of 1-5, with 1 as the lowest and 5 as the highest, what is the extent to which the US CTI program contributed to the following RPOA targets:  Strong legislative, policy and regulatory frameworks in place for achieving an ecosystem approach to fisheries management (EAFM)  How: 62 US CTI Evaluation Report  Improved income, livelihoods and food security in an increasingly significant number of coastal communities across the region through a new Sustainable Coastal Fisheries and Poverty Reduction Initiative (“COASTFISH”)  How:  A more effective management and more sustainable trade in live-reef fish and reef-based ornamentals achieved  How:  Establishment of a Region-wide Coral Triangle MPA System (CTMPAS)  How:  Development and implementation of a Region-wide Early Action Plan for Climate Change Adaption for the near-shore marine and coastal environment and small islands ecosystems  How:  Establishment and operation of Networked National Centers of Excellence on Climate Change Adaptation for marine and coastal environments  How: II. REGIONAL/NATIONAL STAKEHOLDER KII MASTER PROTOCOL Overall approach (Question 1)  What do you see as the primary issues being addressed through the CTI?  Do you think these issues are more effectively addressed in a regional or national/local manner?  Do you think the US CTI program has taken a regional approach to addressing these issues?  If so, in what ways has the approach been regional?  If so, have you noticed any challenges arising from this regional approach?  To what extent do you feel that the program approach has been integrated, both vertically and horizontally?  NOTE: We may need to state how US CTI has defined integration first.  The US CTI Strategic Objective is “improving management of biologically and economically important coastal and marine resources in the Coral Triangle”. What do you think that means in practice? Or, how do you think that would be measured?  To what extent do you feel the approach taken by US CTI has been successful in achieving this strategic outcome, on a scale of 1 to 5 (1=not at all, 2=minimally successful, 3=neither successful nor unsuccessful, 4= moderately successful, 5=very successful)?  To what extent do you think the level of success (or lack of success) can be attributed to the regional and integrated approach of US CTI?  Do you think the same level of effects could have been achieved through programs focused at the national or sub-national level?  Do you have any recommendations on how the program could improve its approach to regional collaboration or integration? Project Design (Question 2)  What do you see as the main constraints to achieving better management of coastal and marine resources?  US-CTI has focused on 4 main results areas: regional and national networks and platforms, EAFM, MPA management, and climate change adaptation. Do you think these are sufficient to achieve better management of coastal and marine resources? US CTI Evaluation Report 63  Are there other important areas that you think need to be addressed in addition to these?  On a scale of 1-5, to what extent do you think the US-CTI focus on these 4 result areas is adequate for achieving improved management (1=completely inadequate to 5=perfectly adequate)?  What do you think are the critical factors that need to be addressed in order to:  Develop stronger regional and national policy platforms or networks for marine and coastal management?  Improve EAFM practices?  Improve MPA management?  Improve adaptability to climate change? Project Implementation (Question 3 and Question 5) (If respondent is not familiar with the CTI program this section, or sub-questions can be skipped.)  On a scale of 1 to 5 (1 being not effective at all and 5 being very effective), how effective do you think US CTI actions have been towards the following objectives, and why?  Develop stronger regional and national policy platforms or networks for marine and coastal management?  Improve EAFM practices?  Improve MPA management?  Improve adaptability to climate change?  What have been the most notable achievements that you think were generated, at least in part, by US-CTI?  Which activities do you think have been most and least effective at achieving results at each of the following levels:  Local  National  Regional  Have you noted any specific examples of activities or approaches implemented in your country that were developed on the basis of activities in other CT6 countries?  If so, how did the idea or approach transfer from the other country (e.g. through a regional exchange or workshop or guidance document, etc.)?  What were some of the key challenges you noted in implementation of the US CTI?  Did you notice any steps taken to address each of these challenges? If so, what were they?  If these challenges are still relevant, how do you think they could be best addressed?  Were there any aspects of the US-CTI approach or implementation that you found particularly innovative? If so, what were they?  Do you have any suggestions for how CTI could innovate moving forward to more effectively achieve its objectives?  Were the member NGOs of the consortium perceived as partners by all the C6 countries? (Do you perceive the NGO consortium as partners?)  Do you think implementing the US-CTSP program through a consortium of NGOs was the right approach? Why or why not?  Would channeling the resources given to NGOs to the national governments be a better option?  What advantages or disadvantages do you think it would it have brought to achieving the objectives of the CTSP?  Are you familiar with the program integrator function of US-CTI provided by TT/ARD? 64 US CTI Evaluation Report  What role do you think the PI played in US-CTI?  On a scale of 1 to 5 (one being very little, 5 being very much), to what extent do you think the PI helped to improve coordination and collaboration for CTI?  Do you think that function would have been better achieved by another body, for example the USCTSP program or the Secretariat?  Looking ahead to once the Secretariat is formally established, what role, if any, do you think a PI function should play? Coordination with CTI-CFF Goals (Question 4)  The US CTI program had goals supporting RPOA Goals 2-4: were Goals 1 and 5 adequately addressed in other activities or are these of lower importance compared to 2-4?  In a scale of 1-5, with 1 as the lowest and 5 as the highest, what is the extent to which the US CTI program contributed to the following RPOA targets:  Strong legislative, policy and regulatory frameworks in place for achieving an ecosystem approach to fisheries management (EAFM)  How:  Improved income, livelihoods and food security in an increasingly significant number of coastal communities across the region through a new Sustainable Coastal Fisheries and Poverty Reduction Initiative (“COASTFISH”)  How:  A more effective management and more sustainable trade in live-reef fish and reef-based ornamentals achieved  How:  Establishment of a Region-wide Coral Triangle MPA System (CTMPAS)  How:  Development and implementation of a Region-wide Early Action Plan for Climate Change Adaption for the near-shore marine and coastal environment and small islands ecosystems  How:  Establishment and operation of Networked National Centers of Excellence on Climate Change Adaptation for marine and coastal environments  How: III. COMMUNITY STAKEHOLDER KII MASTER PROTOCOL What specific lessons have been learned about the validity of the program’s development hypothesis and results frameworks? (Question 2)  What do you see as the primary coastal and marine issues in the CT?  How are the issues addressed by the local CTI project/s?  Are these issues being addressed adequately by the local CTI project/s? If not, how should these be addressed or what improvements need to be made in the way the issues are addressed?  What are the obstacles in improving the management of coastal and marine ecosystems and resources in this area? What would you address these obstacles?  What are the obstacles in improving fisheries management? How would you address these obstacles?  What are the obstacles in improving MPA management? How would you address these obstacles? US CTI Evaluation Report 65  What are the obstacles in developing capacity to adapt to climate change? How would you address these obstacles?  Does the local CTI project/s address these obstacles? If yes, how well are they doing in addressing these obstacles?  With integration defined as [provide definition here], how is this approach being applied in the implementation of the local CTI projects? How does work on each of the three themes (EAFM, MPAs, CCA) affect work on the other themes?  How do higher levels of government assist in implementation of the local CTI project/s? How else should higher levels of government help work at the local level? How does/should local￾level implementation contribute to higher level implementation?  US-CTI has focused on 4 main results areas: regional and national networks and platforms, EAFM, MPA management, and climate change adaptation. Do you think these would be sufficient to achieve better management of coastal and marine resources?  Are there other important areas that you think need to be addressed in addition to these?  Are all of these necessary factors to achieving better management?  On a scale of 1-5, to what extent do you think the US-CTI focus on these 4 result areas is adequate for achieving improved management (1=completely inadequate to 5=perfectly adequate)? How effective have US CTI actions under Intermediate Results been in achieving corresponding Results? (Question 3)  What is your overall perception of the effectiveness of the local CTI projects?  On a scale of 1 to 5 (1 being not effective at all and 5 being very effective), how effective do you think US CTI actions have been towards the following objectives, and why?  Develop stronger regional and national policy platforms or networks for marine and coastal management?  Improve EAFM practices?  Improve MPA management?  Improve adaptability to climate change?  Has there been any improvement in the management of biologically and economically important coastal and marine resources in the Coral Triangle?  What are the specific indications that show that there have been improvements?  Do you think that the CTI program contributed to this change and how? What key challenges, either technical or managerial, have been encountered and innovative practices applied in achieving the US CTI Strategic Objective, thereby contributing to the CTI-CFF goals? (Question 5)  What have been the most notable achievements that you think were generated, at least in part, by the local CTI project/s?  Which activities do you think have been most and least effective at achieving results through the local CTI projects?  Are you aware of any specific examples of activities or approaches implemented in your locality through the local CTI project/s that were developed in other countries? How was the practice/technique/approach transferred? What do you think of these practices/techniques/approaches?  Were there any aspects of the local CTI project practices/techniques/approaches that you found particularly innovative? If so, what were they? 66 US CTI Evaluation Report  Do you have any suggestions for how the local CTI project/s could innovate moving forward to more effectively achieve its objectives?  Would channeling the resources given to NGO to the local governments be a better option?  What advantages or disadvantages do you think it would it have brought to achieving the objectives of the local CTI project/s?  What is the role of [name of NGO] in the local CTI projects? How well are they doing their job?  What is the role of the LGU in the local CTI projects? What assistance did the LGU receive to help it perform its role in the local CTI project? Was the assistance provided adequate? How well is it doing its job?  What other institutions are involved in the implementation of the local CTI projects? What are their roles? How well are they doing in the conduct of their responsibilities?  What is the informant’s role in the local CTI projects? What assistance did he/she receive to help him/her do his/her part in the local CTI project? IV. SURVEY PROTOCOL Dear _: I am writing on behalf of the US Coral Triangle Initiative’s evaluation team. Social Impact, on behalf of USAID, is carrying out a final evaluation of the US CTI program to better understand how USAID support has advanced the objectives of the CTI. Please kindly see the attached information from USAID Regional Development Mission for Asia for more detail on the evaluation. USAID or the US CTI implementers have identified you as a key participant or respondent. We think that you can provide valuable insight into the program as well as ideas for continued support to the Coral Triangle Initiative, so we kindly request that you complete a brief survey at the following address: [Insert survey link] We recognize that your time is valuable, so we expect the survey will take less than 10 minutes. Your answers will be kept completely confidential. We appreciate your help in shaping future support for the coral triangle! Sincerely, _ [Display the following message on the landing page] Purpose of the Survey The US Coral Triangle Initiative program works to support the overall Coral Triangle Initiative and has been implemented by the World Wildlife Fund, Conservation International, The Nature Conservancy, TetraTech/ARD, and NOAA through support from USAID. This survey can be completed in 15 minutes. Your responses to this survey will help us understand how to improve the US CTI program. Confidentiality Statement This survey guarantees respondent confidentiality and your participation in this survey is voluntary. All data will be used in an aggregate form that will make it impossible to determine the identity of the individual responses. Access to raw data will be tightly restricted to only those individuals directly involved in data analysis. [Start of questions] US CTI Evaluation Report 67 In which country do you primarily work?  East Timor  Indonesia  Malaysia  Papua New Guinea  Philippines  Solomon Islands  Thailand  United States  Other (Please Specify) Sex:  Male  Female Age:  __ years What is the highest level of education you have completed?  None  Primary  Secondary  Undergraduate degree  Graduate degree Which of the following best describes the type of organization or institution you represent?  National Government (including Ministries)  Regional or Local Government (including Ministries)  International Organization (i.e. UN, ADB, etc.)  Donor Agency  NGO  Private Sector  Academic  Other (Please specify) How long have you been involved with the US CTI program? __years What percentage of your work effort (job responsibilities) are directly related to the US CTI Support Program?  None  Between 0-25%  Between 26-50% 68 US CTI Evaluation Report  Between 51-75%  More than 75% Which US CTI counterparts have you worked with? (Mark all that apply)  Coral Triangle Support Partnership (WWF, TNC, CI)  Program Integrator (TetraTech/ARD)  NOAA  USAID  None of the above Have you received any type of funding from US CTI?  Yes  No  Don’t Know On a scale of 1 to 5, how much would you say you know about US CTI?  1 = Very little  2  3  4  5 = Very much Which of the following types of CTI activities have you been involved in? (Mark all that apply)  Policy development related to marine and coastal management  CTI related learning and information networks  Development of Ecosystem Approach to Fisheries Management, Marine Protected Area, or Climate Change Adaptation frameworks  Training or capacity building related to EAFM, MPA, or CCA  Meetings of Technical Working Groups  Regional exchanges  Other support to the Regional Secretariat  None of the above On a scale of 1 to 5, to what extent do you feel the US CTI activities were well implemented?  1 = very poorly implemented  2  3  4  5 = very well implemented On a scale of 1 to 5, to what extent do you think US CTI achieved the following objectives (1=very little; 5=very much): US CTI Evaluation Report 69 Result Level of Achievement Develop stronger regional and national platforms or networks for marine and coastal management 1 2 3 4 5 Don’t Know Improve EAFM practices 1 2 3 4 5 Don’t Know Improve MPA management 1 2 3 4 5 Don’t Know Improve capacity to adapt to climate change 1 2 3 4 5 Don’t Know Improve management of biologically and economically important coastal and marine resources in the Coral Triangle 1 2 3 4 5 Don’t Know On a scale of 1 to 5, to what extent do you think the regional nature of the US CTI program contributed to achieving positive results?  1 = No contribution  2  3 = Some contribution  4  5 = Significant contribution Do you think independent national or sub-national programs with similar resources could have achieved the same, or better, results?  Yes  No  Don’t Know Other Questions:  What have been the most notable achievements that you think were generated, at least in part, by US-CTI? (Open response)  Are there other activities not supported by US-CTI that you can think of that would significantly help in achieving the CTI objectives? (Open response)  Have you noticed any unintended results or effects from the US CTI program? (Open response)  Do you have any other suggestions for how support to the CTI could be improved? (Open response)  Are there any other respondents you think could provide helpful insights into the US CTI program? If so, can you please provide their contact information? (Open response) [After conclusion, display the following message] Thank you very much for your valuable input and for helping us to improve the US support to the CTI. If you have any questions, please contact us at _. 70 US CTI Evaluation Report ANNEX III: EVALUATION DESIGN MATRIX Evaluation Question Sub-evaluation Question Data Sources Specific Methods Data Analysis Methods 1. How effective has the US CTI’s regional collaboration and integration approach been in achieving its Strategic Objective of “improving management of biologically and economically important coastal and marine resources in the Coral Triangle” as applied in the US CTI Results Framework? N/A  Agreements  Work plans  Performance reports  Activity reports  Evaluation reports  Program documents (RPOAs, NPOAs, SOM and TWG documents)  Partner documents  Key informants (regional, national, project site)  Document analysis  Internal consultations  Survey  KIIs  Site visits  Content analysis  Frequency/trend/theme/pattern/cluster analysis  Cross-tabulations/matrix analysis  Comparative analyses (among levels/informant groups)  Cross-checking between/among methods  Sequential analyses (building a logical chain of evidence) 1.1 What is the regional collaboration aspect of the approach?  Agreements  Work plans  Performance reports  Activity reports  Evaluation reports  Program documents (RPOAs, NPOAs, SOM and TWG documents)  Partner documents  Key informants (regional, national, project site)  Document analysis  Internal consultations  Survey  KIIs  Site visits  Content analysis  Comparative analyses (among levels/informant groups)  Cross-checking between/among methods 1.2 What is the integration aspect of the approach?  Agreements  Work plans  Performance reports  Activity reports  Evaluation reports  Program documents (RPOAs, NPOAs, SOM and TWG documents)  Partner documents  Key informants (regional, national, project site)  Document analysis  Internal consultations  Survey  KIIs  Site visits  Content analysis  Comparative analyses (among levels/informant groups)  Cross-checking between/among methods US CTI Evaluation Report 71 Evaluation Question Sub-evaluation Question Data Sources Specific Methods Data Analysis Methods 1.3 What does “improving management” of the biologically and economically important coastal and marine resources mean?  Agreements  Work plans  Performance reports  Activity reports  Evaluation reports  Program documents (RPOAs, NPOAs, SOM and TWG documents)  Partner documents  Key informants (regional, national, project site)  Document analysis  Internal consultations  Survey  KIIs  Site visits  Content analysis  Comparative analyses (among levels/informant groups)  Cross-checking between/among methods 2. What specific lessons have been learned about the validity of the program’s development hypothesis and results frameworks? N/A  Agreements  Work plans  Performance reports  Activity reports  Evaluation reports  Program documents (RPOAs, NPOAs, SOM and TWG documents)  Partner documents  Key informants (regional, national, project site)  Document analysis  Internal consultations  Survey  KIIs  Site visits  Content analysis  Frequency/trend/theme/pattern/cluster analysis  Cross-tabulations/matrix analysis  Comparative analyses (among levels/informant groups)  Cross-checking between/among methods  Sequential analyses (building a logical chain of evidence) 2.1 To what extent are the results frameworks logically valid?  Agreements  Work plans  Program documents (RPOAs, NPOAs, SOM and TWG documents)  Partner documents  Key informants (regional, national, project site)  Document analysis  Internal consultations  KIIs  Content analysis 2.2 How has the application of cross-  Agreements  Work plans  Document analysis  Content analysis 72 US CTI Evaluation Report Evaluation Question Sub-evaluation Question Data Sources Specific Methods Data Analysis Methods cutting themes contributed to the validity of the results frameworks?  Program documents (RPOAs, NPOAs, SOM and TWG documents)  Partner documents  Key informants (regional, national, project site)  Internal consultations  KIIs 3: How effective have US CTI actions under Intermediate Results been in achieving corresponding Results? N/A  Work plans  Performance reports  Activity reports  Evaluation reports  Key informants (regional, national, project site)  Document analysis  Internal consultations  Survey  KIIs  Site visits  Content analysis  Frequency/trend/theme/pattern/cluster analysis  Cross-tabulations/matrix analysis  Comparative analyses (between/among levels, countries, priority geographies/sites)  Cross-checking between/among methods 3.1 What program results are visible at the local, national, and regional levels?  Work plans  Performance reports  Activity reports  Evaluation reports  Key informants (regional, national, project site)  Document analysis  Internal consultations  Survey  KIIs  Site visits  Content analysis  Frequency/trend/theme/pattern/cluster analysis  Cross-tabulations/matrix analysis  Comparative analyses (between/among levels, countries, priority geographies/sites)  Cross-checking between/among methods 3.2: To what extent have regional activities influenced local results and vice versa?  Work plans  Performance reports  Activity reports  Evaluation reports  Key informants (regional, national, project site)  Document analysis  Internal consultations  Survey  KIIs  Site visits  Content analysis  Frequency/trend/theme/pattern/cluster analysis  Cross-tabulations/matrix analysis  Comparative analyses (between/among levels, countries, priority geographies/sites)  Cross-checking between/among methods US CTI Evaluation Report 73 Evaluation Question Sub-evaluation Question Data Sources Specific Methods Data Analysis Methods 3.3 Have resources been adequate for achievement of program results?  Work plans  Performance reports  Activity reports  Evaluation reports  Key informants (regional, national, project site)  Document analysis  Internal consultations  Survey  KIIs  Site visits  Content analysis  Frequency/trend/theme/pattern/cluster analysis  Cross-tabulations/matrix analysis  Comparative analyses (between/among levels, countries, priority geographies/sites)  Cross-checking between/among methods 4: How effective have US CTI results been in contributing to the accomplishment of corresponding CTI-CFF goals, strategic objective and targets? N/A  Agreements  Work plans  Performance reports  Activity reports  Evaluation reports  Program documents (RPOAs, NPOAs)  Key informants (regional, national, project site)  Document analysis  Internal consultations  Survey  KIIs  Site visits  Content analysis  Comparative analyses (between/among results areas, countries, and priority geographies/sites) 4.1 To what extent have the US CTI results and CTI-CFF goals overlap?  Agreements  Work plans  Program documents (RPOAs, NPOAs)  Key informants (regional, national, project site)  Document analysis  Internal consultations  Survey  KIIs  Content analysis  Comparative analyses (between/among results areas, countries, and priority geographies/sites) 5: What key challenges, either technical or managerial, have been encountered and innovative practices applied in achieving the US CTI Strategic Objective, thereby contributing to the CTI￾CFF goals? N/A  Agreements  Work plans  Performance reports  Activity reports  Evaluation reports  Program documents (RPOAs, NPOAs, SOM and TWG documents)  Document analysis  Internal consultations  Survey  KIIs  Site visits  Content analysis  Frequency/trend/theme/pattern/cluster analysis  Cross-tabulations/matrix analysis  Cross-checking between/among methods 74 US CTI Evaluation Report Evaluation Question Sub-evaluation Question Data Sources Specific Methods Data Analysis Methods  Partner documents  Key informants (regional, national, project site) 5.1 Was a consortium of NGOs an effective structure for achieving the CTSP objectives?  Agreements  Work plans  Performance reports  Activity reports  Evaluation reports  Program documents (RPOAs, NPOAs, SOM and TWG documents)  Partner documents  Key informants (regional, national, project site)  Document analysis  Internal consultations  Survey  KIIs  Site visits  Content analysis  Frequency/trend/theme/pattern/cluster analysis  Cross-tabulations/matrix analysis  Cross-checking between/among methods 5.2 What was the value added in having a separate program integrator mechanism in this program?  Agreements  Work plans  Performance reports  Activity reports  Evaluation reports  Program documents (RPOAs, NPOAs, SOM and TWG documents)  Partner documents  Key informants (regional, national, project site)  Document analysis  Internal consultations  Survey  KIIs  Site visits  Content analysis  Frequency/trend/theme/pattern/cluster analysis  Cross-tabulations/matrix analysis  Cross-checking between/among methods US CTI Evaluation Report 75 ANNEX IV: SOURCES OF INFORMATION DOCUMENTS REVIEWED  Activity Report US CTI Launch CTI Development Partners Meeting Bangkok  Activity Report: CTI M&E Manual Development Workshop Manila April 2013  Activity Report: CTI Regional Learning Network Planning Meeting March 2011  Activity Report: CTI Regional Priorities and Actions May 2010  Albert, J., Teri, J., Schwarz, A-M and Luda, L. (2012) Nearshore FADs in Solomon Islands  ARD CTI TO Mod 1  ARD CTI TO Mod 2  ARD CTI TO Mod 3  ARD CTI TO Mod 4  ARD CTI TO Mod 5  ARD CTI TO Mod 6  Catalogue US CTI Regional Tools and Resource May 2013  CCA Activity Report: Regional Exchange on Climate Change Adaptation Tools for Action  CCA Activity Report: Regional Exchange on Policy and Action Planning for Climate Change Adaptation October 2010  CCA for Coral Triangle Communities A Guide for Vulnerability Assessment and Local Early Action Planning  Climate Change Adaptation for Coral Triangle Communities: A Guide for Vulnerability Assessment and Local Early Action Planning  Consolidated CTI Partner Update for SOM  Contextual Challenges in the Coral Triangle  Coral Triangle Marine Protected Area System (CTMPAS) Framework and Action Plan  Coral Triangle Regional Ecosystem Approach to Fisheries Management (EAFM) Guidelines  Crisis sentinel indicator Averting a potential meltdown in the Coral Triangle  CTI Annex 8 Decision Document on: Coordination Mechanisms 2012  CTI Malaysia NPOA  CTI Partner Coordination Meeting Report  CTI Partner Coordination Meeting Report Jan 2013 ANNEX A-E  CTI Regional Plan of Action  CTI-CFF Attachment 8-1: Roadmap to a Permanent Regional Secretariat of The Coral Triangle Initiative on Coral Reefs, Fisheries and Food Security, Key Actions, Milestones and Timeline  CTI-CFF Communications Strategy Design  CTI-CFF Interim Secretariat Operations Plan  CTI-CFF Regional Early Action Plan on Climate Change Adaptation  CTSP Year 1 Annual Report  CTSP Year 1 Semi-Annual Report  CTSP Year 1 Work Plan  CTSP Year 2 Annual Report  CTSP Year 2 Semi-Annual Report  CTSP Year 2 Work Plan  CTSP Year 3 Annual Report  CTSP Year 3 Semi-Annual Report  CTSP Year 3 Work Plan 76 US CTI Evaluation Report  CTSP Year 4 Annual Report  CTSP Year 4 SAR  CTSP Year 4 Work Plan  CTSP Year 5 Work Plan  Department of Environment and Natural Resources, Philippines. 2013. Sustainable Coral Reef Ecosystem Management Program. A National Program of the DENR. PAWB CMMO. Brochure.  Designing Marine Protected Area Networks to Achieve Fisheries, Biodiversity, and Climate Change Objectives in Tropical Ecosystems: A Practitioner Guide  Directorate General Marine, Coastal and Small Island, Ministry of Marine Affairs and Fisheries, Indonesia. No date. Disaster and Climate Change Resilient Coastal Village. Brochure  EAFM Activity Report: Enhancing Local Government and Stakeholder Capacity for Ecosystem￾based Fisheries Management  EAFM Activity Report: Live Reef Food Fish Trade Intergovernmental Forum  EAFM Activity Report: Regional Exchange on an Ecosystem Approach to Sustainable Live Reef Fish Food Trade in the Coral Triangle  EAFM Activity Report: Second CTI Regional Exchange and Policy Workshop on Ecosystem Approach to Fisheries Management  EAFM Activity Report: Third CTI Regional Exchange on the Implementation of EAFM in CT Countries  Final Report RIG CTSP Review  Food Security and the CTI  Guidelines for site selection, design and implementation. WorldFish report to Solomon Islands Ministry of Fisheries and Marine Resources  Incorporating Climate and Ocean Change into an Ecosystem Approach to Fisheries Management (EAFM) Plan  Indonesia NPOA Final  Indonesia US Support to CTI Achievements Summary  Institutional Entrepreneurs in the CTI – Rosen & Olsson  Kruijssen, F., Albert, J.A., Morgan, M., Boso, D., Siota, F., Sibiti, S., and Schwarz, A. J. (2013). Livelihoods, markets, and gender roles in Solomon Islands: case studies from Western and Isabel Provinces. CGIAR Research Program on Aquatic Agricultural Systems. Penang, Malaysia. Project Report: AAS-2013-22.  LGN Activity Report: CTI Local Government Network Forum and Executive Course December 2012  LGN Activity Report: CTI Mayors' Roundtable  Licuanan, W. Y. 2011. Impact of mass coral bleaching and crown-of-thorns sea stars on selected reefs in the Philippines. Project MIRROR  Malaysia US Support to CTI Achievements Summary  Manus E Ndras Tribal Development Network (MENDevNet)  MOD 1 WWF - CTI 486-A-00-08-00042-01  MOD 2 WWF RGN 486-A-00-08-00042-02  MOD 3 WWF RGN 486-A-00-08-00042-03  MOD 4 WWF RGN AID-486-A-00-08-00042-04  MOD 5 WWF RGN AID-486-A-00-08-00042-05  MOD 6 WWF RGN AID-486-A-00-08-00042-06  MOD 6 WWF RGN AID-486-A-00-08-00042-06  MOD 8 WWF RGN AID-486-A-00-08-00042-08  MOD 9 WWF RGN AID-486-A-00-08-00042-09 US CTI Evaluation Report 77  MPA Summary Report CTI Regional Exchange and Workshop on Monitoring and Evaluation for Improved MPA Management Effectiveness in the Coral Triangle May 2011  MPA Summary Report Regional Exchange on Designing and Operating MPA Networks and Systems in the Coral Triangle Countries June 2010  MPA Summary Report Regional Exchange on Designing and Supporting National and Regional MPA Systems in the Coral Triangle March 2012  MPA Summary Report Regional Exchange on Developing an MPA Management Effectiveness Protocol for the CTMPAS March 2013  NOAA CTI FY10-Q2 (Semi-annual) Progress Report  NOAA CTI FY10-Q3 Progress Report  NOAA CTI FY10-Q4 (Year End) Progress Report  NOAA CTI FY11-Q1 Progress Report  NOAA CTI FY11-Q2 (Semi-annual) Progress Report  NOAA CTI FY11-Q3 Progress Report  NOAA CTI FY11-Q4 (Year End) Progress Report  NOAA CTI FY12-Q1 Progress Report  NOAA CTI Region Year 2 Work Plan  NOAA Final Year 1 and 2 Work Plan  NOAA US CTI FY13 Consolidated Work Plan  Office of Inspector General (OIG). 2012. Review of USAID/Regional Development Missions for Asia’s Coral Triangle Support Partnership. Final Report.  Palla, H. P. No date. Fish Catch Monitoring In Taytay, Palawan. Western Philippines University￾Puerto Princesa Campus, Puerto Princesa City  Philippines NPOA Final  Philippines US Support to CTI Achievements Summary  PMP US CTI Support Program Year 2 Consolidated  PMP US CTI Support Program Year 3  PMP US CTI Support Program Year 4 Consolidated  PMP US CTI Support Program Year 5  PNG Marine National Plan of Action 2010-2013  PNG US Support to the CTI Achievements Summary  RBF Activity Report: Regional Exchange on Private Sector Engagement  RBF Activity Report: Second CTI Business Forum  Reed, T. No date. Achievements, Challenges, Next Steps, Lessons Learned (Indonesia, Malaysia, Papua New Guinea, Philippines, Solomon Islands, Timor-Leste).  Reefs at Risk Revisited Coral Triangle  Regional Framework for Legislation and Policy to Support an Ecosystem Approach to Fisheries Management (EAFM) in the Coral Triangle Initiative (CTI)  Regional US Support to CTI Achievements Summary  Region-wide Early Action Plan for Climate Change Adaptation for the Nearshore Marine and Coastal Environment and Small Island Ecosystems (May 2013)  RegPriorWS Report  SI-CTI NPOA 2010  Solomon Islands US Support to CTI Achievements Summary  SOM 1 Minister Numberi SOM Keynote  SOM 1 Ministerial Statement Final  SOM 2 Chairman's Summary  SOM 2 CTI Regional Plan of Action Manila Draft 78 US CTI Evaluation Report  SOM 2 Highlights  SOM 2 Manila Resolutions  SOM 3 Chairman's Summary final  SOM 3 DPM Keynote Address  SOM 3 Final Detailed Program  SOM 3 HLO Summit Prepcom1 Chairman's Summary  SOM 3 Joint Minist State  SOM 3 Minutes of the  SOM 3 Revised draft of Leaders Declaration Friends of Chair  SOM 4 Agenda  SOM 4 Attachment-1- coordination structure  SOM 4 Attachment-2 - TOR coordination structure  SOM 4 Attachment-5 - financial contribution  SOM 4 Attachment-6 - transition process  SOM 4 CMWG-KotaKinabalu-Preliminary Agenda  SOM 4 Decision Document - CoorMech  SOM 4 Decision Document SOM4 - Alignment  SOM 4 Decision Document SOM4 - Finance  SOM 4 Decision Document SOM4 - M And E  SOM 4 MEWG results on Monitoring Evaluation  SOM 5 Affirmed Decisions  SOM 5 FINAL - Annex-1 on Regional Secretariat  SOM 5 FINAL - Annex-1 on Regional Secretariat  SOM 5 FINAL - Annex-2 on CTI Roadmap  SOM 5 FINAL - Annex-3 on Draft of Joint Ministerial Statement  SOM 5 FINAL - Annex-5 on Draft Agenda MM2  SOM 5 FINAL - Joint Communique on Climate Change  SOM 5 FINAL - MM2 Agenda  SOM 5 FINAL - Regional Secretariat Structure  SOM 5 FINAL Annex 1 on Regional Secretariat  SOM 5 Final Chairman Summary  SOM 6 Agreement on the Establishment of regional secretariat  SOM 6 ANNEX 1 - Interim Regional Secretariat Report  SOM 6 ANNEX 10 - Decision on Country Report and SCTR  SOM 6 ANNEX 11 - Decision on Logo  SOM 6 ANNEX 12 - Decision on CTI-CFF Roadmap 2011  SOM 6 ANNEX 2 - Decision on Agreement on the Establishment of regional secretariat  SOM 6 ANNEX 3 - Decision on Staff Regulation  SOM 6 ANNEX 4 - Decision on Rules of Procedure  SOM 6 ANNEX 5 - Decision Financial Regulations  SOM 6 ANNEX 6 - Decision on Transition Period  SOM 6 ANNEX 7 - Decision on Monitoring and Evaluation WG  SOM 6 ANNEX 8 - Decision on Financial Resource Architecture  SOM 6 ANNEX 9 - Decision on Regional Priority Actions  SOM 6 Annotated Outline for the SCTR  SOM 6 Climate Change Adaptation Report to SOM6  SOM 6 Financial Regulations US CTI Evaluation Report 79  SOM 6 Indonesia Country Report  SOM 6 Logo for CTI-CFF (cover)  SOM 6 LRFT Workshop Report to SOM6 Yoga-Dar  SOM 6 Malaysia Country Report  SOM 6 Monitoring and Evaluation WG  SOM 6 Outline of the First CTI Annual Report  SOM 6 Philippines Country Report  SOM 6 PNG Country Report  SOM 6 Rules of Procedure  SOM 6 Solomon Islands Country Report  SOM 6 Timor-Leste Country Report  SOM 6 TOR Thematic Working Group  SOM 6 Transition to a permanent Secretariat  SOM 7 Chairman's Summary SOM7  SOM 7 Chairmanship of CTI Council of Ministers  SOM 7 Country Progress Reports  SOM 7 Cross Cutting Activities  SOM 7 CTI-CFF Annual Report  SOM 7 CTI-CFF Implementation Roadmap 2012  SOM 7 CTI-CFF Logo Presentation  SOM 7 Draft Joint Ministerial Statement  SOM 7 Establishment of CTI-CFF Regional Secretariat  SOM 7 Establishment of Technical Working Groups  SOM 7 Financial Regulations  SOM 7 Financial Resources Working Group  SOM 7 First CTI-CFF Annual Report  SOM 7 Indonesia Country Report  SOM 7 Legal Agreements  SOM 7 Logo  SOM 7 Malaysia Country Report  SOM 7 MEWG FINAL  SOM 7 Partner-Report  SOM 7 Partners Update  SOM 7 Philippine Country Report  SOM 7 PNG Country Report  SOM 7 Report of CTI Summit State Department  SOM 7 Roadmap to permanent Regional Secretariat  SOM 7 Rules of Procedure  SOM 7 SCTR Progress  SOM 7 SCTR Progress  SOM 7 Solomon Islands Country Report  SOM 7 Status Report  SOM 7 Timor-Leste Country Report  SOM 7 Staff Regulations  SOM 8 Chairman Summary SOM8  SOM 8 Decision document MEWG Report and M&E System  SOM 8 Decision document on CCA WG Report 80 US CTI Evaluation Report  SOM 8 Decision document on Coordination Mechanisms  SOM 8 Decision document on Coral Triangle Day  SOM 8 Decision document on CTI-CFF Roadmap 2013  SOM 8 Decision document on EAFM WG Report - Framework  SOM 8 Decision document on Financial Resources WG Report  SOM 8 Decision document on Interim Regional Secretariat Report  SOM 8 Decision document on MPA WG Report - CTMPAS  SOM 8 Decision document on Other Matters  SOM 8 Decision document on Seascape Working Group Report  SOM 8 Decision document on Threatened Species Working Group  SOM 8 Putrajaya Joint Ministerial Statement  Toward Ecosystem-based Coastal Areas and Fisheries Management in the Coral Triangle: Integrated Strategies and Guidance  US CTI Midterm Program Evaluation 2010  US CTI Support Program Coordination Meeting: Executive Summary of Meeting Outcomes 2009  US CTI Support Program Coordination Meeting: Summary Report 2009  US CTI Support Program Coordination: Meeting Report 2009  USAID Pacific Islands. 2013. Energy, Environment and Climate Change. Mangrove Rehabilitation for Sustainably-Managed Healthy Forests (MARSH).  Work Plan Y1 US CTI Support Program Integrator  Work Plan Y2 US CTI Support Program Integrator  Work Plan Y3 US CTI Support Program Integrator  Work Plan Y4 US CTI Program Integrator  Work Plan Y5 US CTI Program Integrator  Year 1 Annual Report US CTI Support Program Integrator  Year 1 Quarterly Report US CTI PI April-June 2009  Year 1 Quarterly Report US CTI PI Oct-Dec 2008  Year 1 Semiannual Report US CTI PI Sept 08-Mar 09  Year 2 Annual Report US CTI Program Integrator  Year 2 Quarterly Report US CTI Support Program Integrator April-June 2010  Year 2 Quarterly Report US CTI Support Program Integrator Oct-Dec 2009  Year 2 Semi Annual Report US CTI Program Integrator April 2010  Year 3 Annual Report 2011 US CTI Program Integrator  Year 3 Quarterly Report US CTI PI April-June 2011  Year 3 Quarterly Report US CTI PI Oct-Dec 2010  Year 3 Semi Annual Report 2011 US CTI Program Integrator  Year 3 US CTI Consolidated Work Plan  Year 4 Annual Report US CTI PI Oct 2011-Sept 2012  Year 4 Quarterly Report US CTI PI April- June 2012  Year 4 Quarterly Report Oct Dec 2011 Program Integrator  Year 4 Semi-Annual Report US CTI Support Program Integrator Oct 2011-Mar 2012  Year 4 US CTI Consolidated Work Plan  Year 5 Quarterly Report US CTI PI Oct-Dec 2012  Year 5 Semi-Annual US CTI PI Oct 2012-March 2013  Year 5 US CTI Consolidated Work Plan US CTI Evaluation Report 81 INFORMANTS INTERVIEWED US CTI Informants Interviewed Name Title Affiliation Gender THAILAND Dr. Chumnarn Pongsri Secretary-General and Chief Southeast Asian Fisheries Development Center (SEAFDEC) Male Mr. Magnus Torell Senior Adviser Southeast Asian Fisheries Development Center (SEAFDEC) Secretariat Male Mr. Peter Collier Chief of Party, US CTI Program Integrator Tetra Tech ARD Male Ms. Nives Mattich Deputy Chief of Party Tetra Tech ARD Female Mr. Michael Yates Mission Director USAID RDMA Male Mr. Alfred Nakatsuma Director, Regional Environment Office (REO) USAID RDMA Male Ms. Juniper Neill Deputy Office Director USAID RDMA Female Ms. Supattira Rodboontham Strategic Information Specialist USAID RDMA Female Mr. Renerio Acosta Regional Environment Programme specialist USAID RDMA Male Ms. Danielle Tedesco Natural Resources Officer, Regional Environment Office (REO) USAID RDMA Female Ms. Sylvie Doutriaux Senior Regional Food Security Advisor USAID RDMA Female Ms. Tanya Tam Regional Food Security Monitoring and Evaluation Specialist USAID RDMA Female PHILIPPINES Mr. Bruce Dunn Senior Environment Specialist, Environment and Safeguards Division Asian Development Bank (ADB) Male Ms. Marilou Drilon Project Leader, CT Pacific Project Asian Development Bank (ADB) Female Ms. Lea Tamayo Project Planning and Monitoring Specialist Asian Development Bank (ADB) Female Dr. Raoul Cola Team Leader/Regional Program Manager for the Coral Triangle (CT) Pacific Project Asian Development Bank (ADB) Male Ms. Annabelle Cruz￾Trinidad Team Leader Asian Development Bank (ADB) Female Ms. Marissa C. Garcia Coordinator and M&E Specialist Asian Development Bank (ADB) Female Mr. Pavit Ramachandran Senior Environment Specialist Asian Development Bank (ADB) Male Ms. Annadel Cabanban Ecosystem Approach to Fisheries Management Specialist Asian Development Bank (ADB) Female Mr. Archie Caballero Hook-and-Line Fisher Biton Community, Taytay Male Mr. Elfren Dandal Barangay (Village) Captain Biton Community, Taytay Male 82 US CTI Evaluation Report Mr. Eliseo Dandal Fisher/Fish Cage Owner (LRFFT) Biton Community, Taytay Male Ms. Jessica Muñoz Supervising Aquaculturist Bureau of Fisheries and Aquatic Resources Female Ms. Evangeline Miclat Project Manager Conservation International Female Mr. Raul B. Maximo Project Development Officer III Council for Sustainable Development Staff Male Mr. Jacob Meimban Executive Director, Coastal and Marine Management Office Department of Environment and Natural Resources Male Ms. Lynette Laroya Senior Ecosystems Management Specialist Department of Environment and Natural Resources (DENR) – Protected Areas and Wildlife Bureau (PAWB) Female Ms. Theresa Mundita Lim Director Department of Environment and Natural Resources (DENR) – Protected Areas and Wildlife Bureau (PAWB) Female Ms. Grizelda Mayo￾Anda Executive Director Environmental Legal Assistance Center Female Mr. Rudolfo Baat Secretary LRFFT Buying Station, Paly Community Male Ms. Edith Pantino Secretary LRFFT Wholesaler, Biton Community Female Mr. Robinson Morales Municipal Administrator Municipality of Taytay Male Mr. Romeo Cabungcal Assistant Provincial Agriculturist Office of the Provincial Agriculturist, Puerto Princesa, Palawan Male Mr. Salmero Ecot Hook-and-Line Fisher Paly Community Male Mr. Stephen Adrian Ross Acting Executive Director Partnerships in Environmental Management for the Seas of East Asia (PEMSEA) Male Mr. Guillermo Morales Chief of Party Primex Male Mr. Danilo Fabunan Guard, Tecas Reef Taytay, Palawan Male Dr. Edgardo Gomez Professor University of the Philippines Marine Science Institute Male Dr. Porfirio Aliño Professor University of the Philippines Marine Science Institute Male Ms. Miledel Christine Quibilan Project Coordinator University of the Philippines Marine Sciences Institute (UP-MSI) Female Mr. Roger Carlson Deputy Mission Director USAID/Philippines Male Ms. Rebecca Guieb Governance and Coastal Marine Management Specialist USAID/Philippines Female Ms. Cristina E. Velez Srinivasan Global Climate Change Advisor USAID/Philippines Female Dr. Lota Creencia Professor Western Philippines University (WPU) Female Ms. Chrisma Salao Project Manager World Wildlife Fund (WWF) Philippines Female Mr. Rene J. de la Calzada Provincial Coordinator, Palawan World Wildlife Fund (WWF) Philippines Male Mr. Geoffrey Aludia Municipal Coordinator, Taytay World Wildlife Fund (WWF) Philippines Male Mr. Elmer Montoya Research Assistant World Wildlife Fund (WWF) Philippines Male US CTI Evaluation Report 83 PAPUA NEW GUINEA Ms. Kay Kumaras Kalim Deputy Secretary Department of Environment and Conservation (DEC) Female Ms. Rachel Rabi Fisheries Management Officer Inshore Fishery, National Fisheries Authority (NFA) Female Ms. Luanah Yaman Manager, Sedentary Fisheries Inshore Fishery, National Fisheries Authority (NFA) Female Ms. Piwen Langarap Programs Coordinator Manus Environment Conservation Communities Network Female Mr. Pomat Cyrus Database Manager Manus Environment Conservation Communities Network Male Ms. Yvonne Tio Executive Manager Marine Division, Department of Environment and Conservation (DEC) Female Mr. Leka Pitoi Provincial Support Coordinator National Fisheries Authority (NFA) Male Ms. Luanne Losi Adaptation Policy Analyst Office of Climate Change and Development (OCCD) Female Ms. Emmajil Bogari￾Ahai Senior Policy Analyst Office of Climate Change and Development (OCCD) Female Ms. Helen Rei Program Coordinator Papua New Guinea Center for Locally Managed Areas (CLMA) Female Ms. Mildred Dira Trainer Papua New Guinea Center for Locally Managed Areas (CLMA) Female Ms. Maxine Aujiga Arua Executive Director Papua New Guinea Center for Locally Managed Areas (CLMA) Female Ms. Theresa K. Kas Program Director The Nature Conservancy (TNC), Papua New Guinea Female Ms. Barbara Masike Partnership Coordinator The Nature Conservancy (TNC), Papua New Guinea Female Mr. Walter North United States Ambassador to Papua New Guinea U.S. Embassy, Papua New Guinea Male Ms. Leah Aisi Conservation Officer World Wildlife Fund (WWF) Female Mr. Steven Saleli Finance Manager World Wildlife Fund (WWF) Male SOLOMON ISLANDS Mr. Tastre Ataria Chairman Gizo Environmental Livelihood Conservation Association (GELCA) Male Ms. Nuataly Venni Chairwoman Gizo Environmental Livelihood Conservation Association (GELCA) Female Mr. Willie Kazi Member Gizo Environmental Livelihood Conservation Association (GELCA) Male Mr. Jeffrey Kalamana Member Gizo Environmental Livelihood Conservation Association (GELCA) Male Mr. Eric Kikolo Clerk for the Provincial Government Gizo, Western Province Male Ms. Salome Topo Project Officer, WWF SI Programme Gizo, Western Province Female Ms. Rosalie Masu Deputy Director Inshore Fisheries Management Division, Ministry of Fisheries and Marine Resources Female Ms. Lisa Orodo Wini CTI and CTSP Focal Point for SI Ministry of Environment, Conservation Climate Change, Disaster and Meteorology Female Ms. Agnetha Vave￾Karamui Senior Conservation Officer Ministry of Environment, Conservation, Climate Change, Disaster and Meteorology Female Mr. Kellington Simeon Mobile Platform Lead Surveyor Ministry of Fisheries Male 84 US CTI Evaluation Report Mr. Ben Buga Chief Fisheries Officer Ministry of Fisheries and Marine Resources Male Mr. Kolo Hivu Provincial Fisheries Officer Ministry of Fisheries and Marine Resources Male Mr. Simon Diffey Team Leader/Institutional Development Advisor MISSIF Programme, Ministry of Fisheries and Marine Resources Male Mr. Philip Lydis Community Member and Executive Committee Member Paelonge Community, Gizo, Western Province Male Ms. Joyce Zoti Community Member Paelonge Community, Gizo, Western Province Female Mr. Henis Ghoni Pastor Paelonge Community, Gizo, Western Province Male Mr. Jesse Pitu Community Member Paelonge Community, Gizo, Western Province Male Mr. Michael Vavu Builder, Part-Time Fisher Saeraghi Community, Gizo, Western Province Male Mr. Frazer Nuapito Community Member Saeraghi Community, Gizo, Western Province Male Ms. Judy Alop Community Member Saeraghi Community, Gizo, Western Province Female Ms. Beryl Kolemu Community Member Saeraghi Community, Gizo, Western Province Female Ms. Samantha Mai Community Member Saeraghi Community, Gizo, Western Province Female Mr. Andrian Soni Community Member Saeraghi Community, Gizo, Western Province Male Mr. Raymond Elopala Small Store Operator Saeraghi Community, Gizo, Western Province Male Ms. Loyley Ngira Chief Executive Solomon Telekom Company Limited Female Mr. Collin Generiu Marine Conservation Practitioner The Nature Conservancy (TNC) Male Mr. Shannon Seeto Marine Programme Manager World Wildlife Fund (WWF) Western Melanesia Male Ms. Delvene Boso Country Manager, Natural Resources Management WorldFish Center, Solomon Islands Female Ms. Anne-Maree Schwarz Research Program Leader WorldFish Center, Solomon Islands Female MALAYSIA Mr. Yusuf Bural, Chairman Berungus Youth Club (BYC) Male Ms. Sarmalin Sakirun Secretary Berungus Youth Club (BYC) Female Mr. Rayner Datuk Stuel Galid Director of Fisheries, Sabah, Malaysia Department of Fisheries Male Mr. Johny Wong Chairman Kudat Fishing Boat Owner’s Association Male Mr. Chris Kong Feng Hin Secretary Kudat Fishing Boat Owner’s Association Male Ms. Francesca Ngo Winfield Chair Kudat Turtle Conservation Society Female Mr. Aziz Amin Bangsah Chairman Maliangin Island Conservation Association (MICA) Male Ms. Cheryl Rita Kauer Senior Researcher and Centre Head, Centre for Coastal and Marine Environment Maritime Institute of Malaysia (MIMA) Female US CTI Evaluation Report 85 Ms. Norasma Dacho Senior Fisheries Officer Ministry of Fisheries Female Dr. Nor Aeini Binti Haji Mokhtar Under Secretary Ministry of Science, Technology and Innovation Female Mr. Mohammed Zaini Bin Abdul Rahman Deputy Under Secretary Ministry of Science, Technology and Innovation Male Ms. Uwarahi Krishnan Assistant Secretary Ministry of Science, Technology and Innovation Female Ms. Noor Aznnimm Binti Zahariman Knowledge Management and Communications Officer Ministry of Science, Technology and Innovation Female Ms. Aida Maizuria Masmal Principal Assistant Secretary Ministry of Science, Technology and Innovation Female Mr. Hj Shamruddin Yusof Principal Assistant Secretary Ministry of Science, Technology and Innovation Male Mr. Augustin Binson Park Officer Sabah Parks Male Ms. Ros Illyahni Assistant Officer Sabah Parks Female Ms. Ivy Edward Assistant Officer Sabah Parks Female Mr. Brock Fox Economic Officer, ESTH U.S. Embassy Malaysia Male Dr. Saleem Mustafa Director University Malaysia, Sabah, Marine Research Institute Male Dr. Ejria Saleh Sr. Lecturer University Malaysia, Sabah, Marine Research Institute Female Mr. Muhammad Ali Bin Syed Hussein Lecturer, Head of Marine Science Program University Malaysia, Sabah, Marine Research Institute Male Dr. Rossita Hj. Shapawi Associate Professor Deputy Director - Research & Innovation University Malaysia, Sabah, Marine Research Institute Female Mr. Julian Ransangan Community Works, Deputy Director University Malaysia, Sabah, Marine Research Institute Male Ms. Sofia Johari Community Liaison Officer/Community Engagement Officer - specifically livelihood programs World Wildlife Fund (WWF) Malaysia Female Ms. Bobita Ahad Marine Biologist/Community Engagement and IEC World Wildlife Fund (WWF) Malaysia Female Ms. Robecca Jumin Sulu Sulawesi Marine Ecoregion Programme Manager World Wildlife Fund (WWF), Malaysia Female INDONESIA Mr. Suseno Sukoyono Ministry of Marine Affairs and Fisheries, Head Agency for Marine and Fisheries Human Resources Development Male Dr. Tiene Gunawan Marine Program Director Conservation International (CI) Female Mr. Ketut Putra CI Executive Director Conservation International (CI) Male Mr. Iwan Dewantawa Program Manager Conservation International (CI) Male Mr. Asril Djunaidi Marine Tourism and Capability Building Conservation International (CI) Male Mr. Marthen Welly MPA Learning Sites Manager Coral Triangle Center (CTC) Male Mr. Wira Sanjaya, Outreach Coordinator Coral Triangle Center (CTC) Male Dr. Rili Djohani Executive Director Coral Triangle Center (CTC) Male 86 US CTI Evaluation Report Mr. Johannes Subijanto Deputy Director Coral Triangle Center (CTC) Male Mr. Hendra Siry Vice Coordinator Coral Triangle Initiative (CTI) Interim Regional Secretariat Male Mr. Arwan Rukma Regional Coordinator Coral Triangle Initiative (CTI) Interim Regional Secretariat Male Mr. Mattheus Eko Rudyanto Director of Maritime Planning and Coastal Areas (Maritime Affairs and Fisheries Ministry) Coral Triangle Initiative (CTI) Interim Regional Secretariat Male Mr. Maurice Knight Chief of Party Coral Triangle Support Partnership (CTSP) Male Ms. Lilly Pregiwati Deputy Director of Planning Division Ministry of Marine Affairs and Fisheries Female Mr. Made Gunaja Head of Bali Fishery and Marine Affairs Office Ministry of Marine and Fisheries Male Mr. Abdul Halim Marine Program Director The Nature Conservancy (TNC) Male Mr. Mirza Pedju Area Based Conservation Manager The Nature Conservancy (TNC) Male Mr. Ben Woehlauer ESTH Officer U.S. Embassy Indonesia Male Mr. Ketut Sudiarta Lecturer of Fisheries Faculty University of Warmadewa Male Mr. Agung Surjawan Head of Tourism Research Center University of Warmadewa Male Mr. John Hansen Director of Environment Office USAID/Indonesia Male Ms. Celly Catharina Marine Program Specialist USAID/Indonesia Female Mr. Milen Vollen Senior Marine Fisheries Advisor USAID/Indonesia Male Ms. Heather D’Agnes Environment Officer USAID/Indonesia Female Mr. Wawan Ridwan Marine and Marine Species Program Director World Wildlife Fund (WWF) Indonesia Male Dr. Lida Pet Soede Director World Wildlife Fund (WWF) Coral Triangle Program/ World Wildlife Fund (WWF) Indonesia Female TIMOR-LESTE Mr. Calisto Vilela Suco Chief Com (Village) local/village government Male Ms. Candice Mohan Country Director Conservation International, Timor-Leste Female Mr. Rui Pinto Policy Manager Conservation International, Timor-Leste Male Mr. Tome Da Cruz Harbor Master Direcção Nacional de Pescas, Com (Village) Male Mr. Castro Acosta Officer District Government Environment Male Mr. Eligito Ximenes District Government Fisheries Officer Fisheries Department Male Mr. Antonio Caetano Fisheries officer, Environment Office Fisheries Department Male Mr. Leonardo da Costa Beneficiary fisherman, Com (Village) Fisherman Association, Com (Village) Male Mr. Peter Pechacek Consultant GIZ (Deutsche Gesellschaft für Internationale Zusammenarbeit, GmbH Male US CTI Evaluation Report 87 (German: German Society for International Cooperation, Ltd.)) Mr. Demétrio do Amaral do Carvalho Former Director Haburas Foundation Male Mr. Virgílio Gutterez da Silva Current Director Haburas Foundation Male Mr. Aleixo Leonito Amaral Former CTI focal point currently for ADB Ministry of Agriculture and Fisheries Male Mr. Lourenço Borges Fontes Director General Ministry of Agriculture and Fisheries Male Ms. Ervina Soares Pinto Ministry of Agriculture and Fisheries, Chief of Finance Ministry of Agriculture and Fisheries Female Mr. Augusto Fernandes National Director of Fisheries and Agriculture Ministry of Agriculture and Fisheries Male Mr. Fidelino Sousa Marquez Focal Point for CTI Ministry of Agriculture and Fisheries Male Mr. Nelio Viegas IT and Communication Ministry of Agriculture and Fisheries Male Mr. Pedro Pinto Park Manager National Park, Department of Agriculture and Fisheries Male Mr. Cristôvão Martins Environment Secretary of State Male Mr. Mário Ximenes Environment Secretary of State Male Mr. Antonio Fonseca Suco Chief Tutuala Male Mr. Ryder Rogers Director USAID Economic Growth USAID/Timor-Leste Male Ms. Flavia da Silva Environment and Infrastructure Specialist USAID/Timor-Leste Female Ms. Cheryl A. Williams Program Officer USAID/Timor-Leste Female UNITED STATES Ms. Niquole Esters Program Manager, Coral Triangle Initiative Conservation International (CI) Female Dr. Frazer McGilvray Sr. Manager, Regional Marine Strategy Conservation International (CI) Male Ms. Christy Osoling Sr. Director, Finance and Operations Conservation International (CI) Female Ms. Monique Derfuss Director Conservation International (CI) Female Dr. Robert Pomeroy Technical Expert Coral Triangle Support Program (CTSP), University of Connecticut Male Dr. Alan White Technical Expert Marine Protected Areas (MPA), Coral Triangle Support Partnership (CTSP), The Nature Conservancy (TNC) Male Mr. Scot Frew National Oceanic and Atmospheric Administration (NOAA) Coral Reef Conservation Program (CRCP) National Oceanic and Atmospheric Administration (NOAA) Male Ms. Jennifer Koss National Oceanic and Atmospheric Administration (NOAA) Coral Reef Conservation Program (CRCP) National Oceanic and Atmospheric Administration (NOAA) Female Dr. Catherine Courtney Technical Expert Tetra Tech Female 88 US CTI Evaluation Report Mr. Lucas Wolf Project Manager TetraTech Male Ms. Jess Britt Assistant Project Manager TetraTech Female Ms. Emily Tibbott Director, US Relations, Asia Pacific Region The Nature Conservancy (TNC) Female Dr. Alan White Senior Scientist--Asia Pacific Program The Nature Conservancy (TNC) Male Mr. Bill Raynor Director, Indo-Pacific Division The Nature Conservancy (TNC) Male Dr. Barbara Best Senior Coastal Resources and Policy Specialist, Office of Forestry and Biodiversity USAID Female Mr. Richard Volk USAID/E3/W USAID Male Ms. Catherine Plume Managing Director, Coral Triangle Program World Wildlife Fund (WWF) Female Ms. Kate Newman Senior Director, Public Sector Initiatives World Wildlife Fund (WWF) Female Ms. Lee Zahnow Sr. Director, Strategic Agreement Services World Wildlife Fund (WWF) Female Mr. Felipe Chirinos Sr. Director of Field Program Operations World Wildlife Fund (WWF) Male US CTI Evaluation Report 89 ANNEX V: DISCLOSURE OF ANY CONFLICTS OF INTEREST 90 US CTI Evaluation Report US CTI Evaluation Report 91 92 US CTI Evaluation Report U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523