EVALUATION OF THE GLOBAL CONSERVATION PROGRAM (GCP) FINAL EVALUATION REPORT MAY 2008 This report was prepared for the United States Agency for International Development by ARD, Inc. ACKNOWLEDGEMENTS Many people contributed their time, and provided information and insights throughout the participatory process used in this evaluation. We would like to thank the GCP partner representatives, and their field staff who participated in the survey and hosted us on site visits. During those site visits we also met many other individuals from local partner organizations who helped us understand the site-level benefits of GCP. We also want to thank the USAID staff who manage GCP for the time they spent helping us to understand the program, and USAID staff in the missions we visited during our site visits. Prepared for the United States Agency for International Development, USAID Contract Number EPP-I-00-06­ 0008, Task Order No. 1. Submitted to: USAID/EGAT/NRM/B Submitted by: Bruce A. Byers (ARD), Evaluation Team Leader Jared J. Hardner (Hardner & Gullison Associates, LLC), Evaluation Specialist ARD Contact: Ed Harvey, Project Manager 159 Bank Street, Suite 300 P.O. Box 1397 Burlington, VT 05402 Tel: (802) 658-3890 ext. 2407 Email: eharvey@ardinc.com Cover Photo: Landscape-scale conservation is a hallmark of the Global Conservation Program. In this photo from AWF's Maasai Steppe Heartland landscape in northern Tanzania, Aaron Musiga, the ranch manager of the Manyara Ranch, describes on a map the complex mosaic of land tenure and uses in the area, with the acacia savanna ecosystem of the Rift Valley behind him. Photographer: Bruce Byers. EVALUATION OF THE GLOBAL CONSERVATION PROGRAM (GCP) FINAL EVALUATION REPORT MAY 2008 DISCLAIMER The views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. CONTENTS ACRONYMS AND ABBREVIATIONS..........................................................................ii EXECUTIVE SUMMARY ...............................................................................................iii 1.0 BACKGROUND .......................................................................................................1 1.1 HISTORY AND OBJECTIVES OF THE GLOBAL CONSERVATION PROGRAM ......................1 1.2 PURPOSE OF THE EVALUATION..............................................................................3 2.0 EVALUATION METHODOLOGY.........................................................................4 3.0 RESULTS...................................................................................................................6 3.1 SITE-LEVEL RESULTS .............................................................................................6 3.1.1 What have been the challenges faced at the site level and on which have site managers made the most progress? .................................................................6 3.1.2 What are the landscape-scale conservation planning approaches used by GCP partners? What influence has GCP had in their development? ...........................8 3.1.3 What are the adaptive management approaches used by GCP partners? What influence has GCP had in their development?.................................................13 3.1.4 What best practices were developed? ...........................................................17 3.1.5 What suite of partners was involved at sites? .................................................17 3.1.6 How has GCP funding affected site-level conservation, considering consistency, flexibility and relative contribution to overall site funding?...............................18 3.2 DEVELOPMENT AND ADOPTION OF BEST PRACTICES ............................................... 20 3.2.1 How did grantee formulate best practices to address limiting factors at site? ...20 3.2.2 How did grantee share learning on best practices at site with others (cross GCP partner, cross site)?......................................................................................21 3.3 ADDED VALUE OF GCP..................................................................................... 23 3.3.1 Did GCP successfully promote cross-institutional learning? If so, what were the most effective mechanisms? ..........................................................................23 3.3.2 Did sites benefit from multi-institutional collaboration at headquarters level brought about by GCP? ................................................................................24 3.3.3 From a site perspective, what has central funding helped them achieve that mission funding has not? ...............................................................................25 3.3.4 How has USAID GCP performed administratively as a donor? ........................25 4.0 SUMMARY OF KEY FINDINGS...........................................................................27 5.0 IMPLICATIONS FOR FUTURE PROGRAMS ....................................................29 ANNEX A. SCOPE OF WORK....................................................................................33 ANNEX B. EVALUATION FRAMEWORK ................................................................38 ANNEX C. AGGREGATED SITE-LEVEL SURVEY RESULTS................................42 ANNEX D. AGGREGATED PARTNER REPRESENTATIVE SURVEY RESULTS.................................................................................................................63 ANNEX E. PERSONS CONTACTED.........................................................................75 ANNEX F. SITE VISIT ITINERARIES.........................................................................77 ANNEX G. CATALOGUE OF GCP DOCUMENTATION ......................................78 ANNEX H. GCP PRINCIPLES .....................................................................................81 ANNEX I. EVALUATION TEAM ................................................................................83 GCP EVALUATION FINAL REPORT i ii GCP EVALUATION FINAL REPORT ACRONYMS AND ABBREVIATIONS AWF African Wildlife Foundation BIOFOR Biodiversity and Forestry (IQC) BSP Biodiversity Support Program CI Conservation International CMP Conservation Measures Partnership CTO Cognizant Technical Officer EAME Eastern African Marine Ecoregion EWV Enterprise Works/VITA FY Fiscal Year GCP Global Conservation Program ICAA Initiative for Conservation in the Andean Amazon IQC Indefinite Quantity Contract LLP Living Landscapes Program LWA Leader With Associates M&E Monitoring and Evaluation MAR Meso-American Reef MPA Marine Protected Area NGO Nongovernmental Organization NTFP Non-Timber Forest Product PMP Performance Management Plan PLACE Prosperity, Livelihoods, and Conserving Ecosystems (IQC) RFA Request for Applications RFTOP Request for Task Order Proposals SOW Scope of Work TNC The Nature Conservancy USAID United States Agency for International Development WCS Wildlife Conservation Society WWF World Wildlife Fund EXECUTIVE SUMMARY BACKGROUND Global Conservation Program Background The evaluation of USAID’s Global Conservation Program (GCP) described in this report was conducted between June 2007 and May 2008, through a Task Order under the Prosperity, Livelihoods, and Conserving Ecosystems (PLACE) Indefinite Quantity Contract (IQC). GCP is a Leader with Associates (LWA) assistance mechanism that began in 1999, and funded six US-based nongovernmental organizations (NGOs) involved in biodiversity conservation to conduct site-level work in more than 25 specific landscapes and seascapes worldwide. GCP partner NGOs are the African Wildlife Foundation (AWF), Conservation International (CI), Enterprise Works/VITA (EWV), The Nature Conservancy (TNC), Wildlife Conservation Society (WCS), and World Wildlife Fund (WWF). GCP’s central mandate focuses on achieving landscape-level conservation results in a representative selection of the world’s most biodiverse areas. The program also focuses on the sharing of lessons learned and conservation approaches between sites and among partners. GCP has a life-of￾program funding level of approximately $72 million. Objectives of the Evaluation According to our Scope of Work (SOW), the objectives of this evaluation were to: 1) “…document the added value of the centrally managed, multi-institutional GCP program; 2) …assess development and adoption of best practices within biodiversity conservation promoted by the GCP, including cross-partner, cross-site aspects of such learning; and 3) …document selected site-level conservation results.” This evaluation was a programmatic evaluation, not an evaluation of the performance of particular partners or sites. The SOW reflects an interest in evaluating the cross-site, cross-institutional learning aspect of GCP. Our SOW also asked us to evaluate “The adoption of key concepts, particularly landscape planning and conservation and adaptive management within the GCP family of projects and as a function of GCP partnerships.” USAID also was interested in understanding the roles and value of centrally funded programs such as GCP. Evaluation Methodology The general strategy we used in this evaluation was to gather information at three levels that reflect the three evaluation objectives given in the SOW. We worked upward from information from the sites (e.g., what are best practices employed at the site and how effective are they), to the network of organizations that manage them (e.g., how are best practices developed), to the GCP program that supports them (e.g., how is development and transfer of best practices facilitated). This strategy gave us a comprehensive view of how the program comes together as a whole. Our methodology was highly participatory, collaborative and open, and we engaged USAID staff and GCP partners many times throughout the process to update them on our progress and ask them to give input or review findings. We were gratified by the quality and quantity of engagement of both USAID and their NGO implementing partners throughout the process. GCP EVALUATION FINAL REPORT iii iv GCP EVALUATION FINAL REPORT The evaluation process can be summarized as follows: • Kickoff meeting with USAID and GCP partners at GCP Annual Meeting, 26 June 2007; • Evaluation questions developed with USAID and GCP partners; • Evaluation framework developed, 23 August 2007 (Annex B); • Web-based survey of site-level managers and GCP partner representatives (Annexes C and D); • Individual interviews with Cognizant Technical Officers (CTOs), August 2007 and January 2008 (Annex E); • Individual interviews with GCP partner representatives, August 2007 (Annex E); • Site Visits (Annex F): − Central America (TNC, WCS) (October 2007), − East Africa (WWF, AWF) (November-December 2007), and − Brazil (CI) (February 2008); • Presentation of preliminary results to USAID and GCP partners (February 2008); • Draft report to USAID and GCP partners for review (March 2008); • Comments from USAID and GCP partners (March 2008); and • Final report to USAID (May 2008). RESULTS The evaluation produced a wealth of information about the performance and effectiveness of GCP. We found clear evidence that GCP had achieved some of its objectives, and had positive impacts on some aspects of biodiversity conservation at GCP sites worldwide. USAID and its implementing GCP partners can be justly proud of these achievements. Key positive results and findings are: 1. GCP has been effective in addressing several factors limiting conservation at GCP sites worldwide. In particular, we found clear evidence that GCP influenced the design of conservation programs at GCP sites, in terms of both approach and scale. GCP has facilitated and supported the development of threats￾based conservation design and planning at the landscape and seascape scales. It has also played a catalytic role in starting the development of adaptive management within GCP partner NGOs, although much work remains. In addition, GCP enabled its partners to make significant progress in addressing two other important factors that can limit conservation at sites: the lack of stakeholder engagement and institutional capacity. Our results show that before GCP, these factors prevented, or were significant barriers to, conservation at most GCP sites, but were addressed sufficiently through GCP so that they shifted to become, in general, a manageable problem or not a problem. These are clearly positive results, which show that the program achieved a number of its objectives. 2. GCP enabled major internal learning and program development within some GCP partner institutions. We found evidence that it facilitated the adoption of new approaches for threats-based conservation planning at large spatial scales, and progress toward a more standardized approach to adaptive management of projects and programs. 3. GCP enabled, facilitated, or catalyzed some significant, informal direct communication, collaboration, and learning between NGO partners. Informal, natural collaborations among GCP partner organizations that were mutually beneficial developed in the enabling environment supported by GCP, creating some of the positive results we observed. 4. Centrally funded programs like GCP can contribute to USAID’s global objectives, complement mission programs, and add value by: • Supporting global priorities, long-term strategies, and transboundary projects; • Supplementing limited mission support; • Funding activities in non-presence countries; • Facilitating inter-institutional, global learning and sharing of state-of-the-art approaches, models, and practices; and • Supporting NGO partners in their institutional implementation of new approaches (e.g., landscape￾level conservation) through dedicated programs and staff based at the headquarters level of these international organizations. As would be expected in any large-scale, long-term, multi-partner program such as GCP, not all of the program’s original intentions and objectives were achieved, and much work remains. This evaluation identified a number of remaining challenges: 1. Certain factors limiting conservation at GCP sites remain problems. Government policy and legislation, economic context, illegal activities, and financial sustainability still “prevent conservation” or are a “serious barrier” at many sites. Although they remain problems, the data show that for each of these four factors progress has been made during the period of GCP support. Financial sustainability remains the biggest barrier to conservation. 2. Adaptive management as used in GCP is a vague concept that roughly describes a process of modifying practices over time to improve performance. Most projects do not use counterfactuals (such as control sites or projections of change in the absence of a specific intervention), making it difficult to discern what changes can be attributed to project interventions. In addition, monitoring, analysis, and feedback mechanisms to systematically adapt project management according to empirical evidence of the performance of specific interventions are not common. 3. An effective formal structure to promote cross-institutional learning did not develop, despite significant efforts. Formal learning activities that began in GCP II were not seen as a major influence on the learning or sharing of success stories and best practices among partners and sites. IMPLICATIONS FOR FUTURE PROGRAMS The RFTOP for this evaluation (Annex A) stated that “the evaluation should analyze and provide a discussion of... key recommendations to USAID and partners on comparative advantages and disadvantages of the GCP model.” It also stated, however, that “USAID does not anticipate a direct follow-on ‘GCP III’ activity.” There are a number of specific positive and negative elements of the GCP model, described throughout the report and summarized here. Future programs may wish to maintain positive elements of the GCP model, including: • Logical, threats-based conservation design at the landscape/seascape scales; • An enabling environment that provides support intended to facilitate and catalyze global (cross-site and cross-ecoregional) learning and development of best practices within conservation NGOs; • An enabling environment that has the intention and expectation of cross-institutional sharing of best practices and learning; • An enabling environment with the flexibility to allow for—and support—natural, informal, voluntary collaboration between NGO implementing partners; • Long-term and flexible funding; • A centrally funded mechanism that can complement USAID mission funding and add value through global activities; and GCP EVALUATION FINAL REPORT v vi GCP EVALUATION FINAL REPORT • Continued attention to stakeholder engagement and building the capacity of local institutions in order to maintain site-level successes in addressing these limiting factors to conservation. Future programs may wish to address weaknesses and remaining challenges, including: • Incomplete development and adoption of adaptive management systems; • Lack of a program-wide communications strategy and system that would allow site-level managers to understand and better contribute to the global objectives of the program; • Lack of indicators that track progress in abating threats to biodiversity and addressing limiting factors to conservation; • Poor documentation of successes and failures at the site level as part of an adaptive management system, and to enable cross-site learning; • The need to emphasize learning and development of best practices related to the limiting factors that remain the most serious barriers to site-level conservation: government policy and legislation, economic context, illegal activities, and financial sustainability, with financial sustainability at the top of the list; • The need to develop an effective structure to promote formal learning, including clear leadership/ responsibility, and realistic incentives for participation, if that is an objective; and • The need to simplify and streamline reporting requirements. We believe there are three general options, or models, for establishing processes for interinstitutional knowledge-sharing and learning: 1. Voluntary, informal collaboration and learning that develops naturally in an “enabling environment” that provides the conditions and support for inter-institutional communication; 2. A “Learning Panel” model, as tried in GCP II, in which representatives from the NGO conservation partners engage in a collaborative decision-making process to select learning topics of mutual interest; and 3. A central learning secretariat in one responsible institution, empowered through either a contract or cooperative agreement mechanism, with performance targets and an adaptive management system for generating inter-institutional learning results. In our view, all models are dependent on several independent factors that will determine their success: a) a willingness and genuine interest among parties to share knowledge; b) realistic financial incentives to cover the costs (staff time, travel, communications) of sharing knowledge; c) cooperative and constructive individuals representing the organizations involved; d) leadership of the knowledge-sharing process; and e) actual knowledge to share, generated through rigorous processes that have technical validity. In GCP II, we saw some evidence of weakness in each of these factors. These factors are important considerations in implementing any of the above models. Although USAID does not anticipate a direct follow-on to GCP, we assume that donors will continue to support conservation NGOs such as GCP partners, in their work to conserve the Earth’s biological diversity. We hope that the findings of this evaluation will contribute in a small way to foster the evolution of effective conservation programs. 1.0 BACKGROUND 1.1 HISTORY AND OBJECTIVES OF THE GLOBAL CONSERVATION PROGRAM The Global Conservation Program (GCP) is a partnership between USAID and six leading US-based nongovernmental organizations (NGOs) that aims to conserve globally significant areas of biodiversity. Partner organizations implement site-based programs around the world. These programs work at varying scales, from the community level to large landscape and seascape scales. GCP was funded at a level of $4.2 million dollars in FY2005 with a life-of-program funding level of $72 million. GCP is USAID’s only global conservation initiative, complementing a wide array of Agency-funded biodiversity activities around the world. Management is based in USAID/Washington with a central manager and Cognizant Technical Officers (CTOs) for different partners. GCP employs competitively awarded cooperative agreements with the six NGOs under USAID’s Leader with Associates (LWA) award mechanism. To date over 50 Associate Awards for over US $130 million have been awarded using USAID mission funds. In addition to site-specific conservation, GCP has supported learning activities for many years; these have evolved and now involve all partners. Learning themes include socioeconomic tools and methods for conservation, marine protected area learning network and landscape planning, among others. GCP was designed to achieve conservation results in partnership with NGOs, and to promote best practices, partnerships, and build communities of practice. It is a long-term centrally funded and managed program that complements investments of bilateral missions and partner organizations. GCP’s central mandate focuses on achieving landscape-level conservation results in a representative selection of the world’s most biodiverse areas. The program also focuses on the sharing of lessons learned and conservation approaches among sites and partners. GCP has gone through two phases. Some sites have carried over from the first phase (1999-2004) while others have been closed or newly initiated in the second phase (2003-2008), as shown in Table 1.1. The Request for Applications (RFA) for the first phase of GCP (GCP I), issued in January 1999, stated that the program would support “two types of inter-related and broadly defined approaches: (a) site￾based (in situ) activities in areas of globally significant biodiversity; and (b) interventions which improve the policy environment and thereby substantially contribute to biodiversity conservation.” The RFA for GCP I included 14 principles intended to guide the program. These principles included a threats-based approach to conservation, adaptive management, attention to financial sustainability, in￾country institutional capacity building, orientation to results and performance monitoring, and the integration of analysis and dissemination of lessons learned. In the RFA for GCP II, some principles were combined into 10 guiding principles for the GCP program (see Annex H for full description). Box 1.1. Ten Guiding Principles of GCP II 1. Programs should use a threats-based approach. 2. Programs should focus on globally important sites for biodiversity conservation. 3. Programs should be adaptive. 4. Programs should foster sustainability. 5. Programs should be participatory. 6. Programs should help NGOs expand their initiatives. 7. Programs should strengthen in-country capacity and foster collaboration. 8. Programs must be results oriented. 9. Programs should integrate learning into program design. 10. Programs should complement other conservation and development activities. GCP EVALUATION FINAL REPORT 1 TABLE 1.1. SITES FUNDED BY GCP I AND II Organization Site Funding History (FY '99-'07) GCP I GCP II Conservation International Pantanal & Cerrado -- Brazil FY '99-03 FY '04-'07 Menabe Corridor -- Madagascar Not Funded FY '03-'07 Guyana FY '99-'03 Not Funded Luzon Sierra Madre -- Philippines FY '99-'03 Not Funded Wildlife Conservation Society Maya Biosphere Reserve -- Guatemala Not Funded FY '03-'07 Glover's Reef -- Belize Not Funded FY '03-'07 NW Andes / Madidi -- Bolivia FY '99-'02 FY '03-'07 Eastern Steppe -- Mongolia Not Funded FY '03-'07 Ndoki-Likouala -- Congo FY '99-'03 Not Funded Yasuni-Napo Forest -- Ecuador FY '99-'03 Not Funded African Wildlife Foundation Kilimanjaro -- Kenya/Tanzania FY '99-'02 FY '03-'07 Maasai Steppe -- Tanzania FY '01-'02 FY '03-'07 Laikipia-Samburu -- Kenya FY '99-'02 FY '03-'07 Mana-Zambezi -- Zimbabwe/Zambia/Mozambique FY '99-'02 Not Funded World Wildlife Fund Eastern African Marine Ecoregion -- Mozambique/Tanzania/Kenya Not Funded FY '03-'07 Forests of the Lower Mekong -- Cambodia/Vietnam/Thailand/Laos FY '99-'02 FY '03-'07 Terai Arc Landscape -- Nepal FY '01-'03 FY '04-'07 SW Amazon -- Bolivia/Peru FY '99-'03 Not Funded Bering Sea -- Russia FY '99-'03 Not Funded Atlantic Forest -- Brazil FY '99-'01 Not Funded Sulu-Sulawesi -- Indonesia FY '99-'03 Not Funded The Nature Conservancy Komodo NP Marine -- Indonesia FY '99-'03 Not Funded Raja Ampat Island -- Indonesia Not Funded FY '03-'07 Meso-American Reef -- Belize/Honduras/Guatemala/Mexico Not Funded FY '03-'07 Wakatobi NP -- Indonesia Not Funded FY '03-'07 Kimbe Bay -- Papua New Guinea FY '99-'03 FY '04-'07 Enterprise Works/VITA Himalayas -- Nepal FY '99-'03 Not Funded Community Based Forest Management -- Philippines FY '01-'03 FY '04-'07 2 GCP EVALUATION FINAL REPORT The word “learning” or phrase “learning activities” was not used in the GCP I RFA. However, something broadly related to learning, described as “analysis” and “lessons learned,” was described under “Program Principles” in that document. Even though the GCP I RFA did not use the word “learning” or list learning as a broad objective of the program, four of the six GCP I partners proposed activities designed to lead to learning better practices for landscape-scale conservation within their own organizations, and two used the word “learning” in their descriptions of these activities. Formal “Learning Activities” were added to GCP II. The GCP II RFA has guidelines for three categories of activities: 1) site-based activities, 2) policy activities, and 3) learning activities. 1.2 PURPOSE OF THE EVALUATION The Scope of Work (SOW) for the evaluation (Annex A) asked us to: 1) “…document the added value of the centrally managed, multi-institutional GCP program; 2) …assess development and adoption of best practices within biodiversity conservation promoted by the GCP, including cross-partner, cross-site aspects of such learning; and 3) …document selected site-level conservation results.” The SOW reflects an interest in evaluating the cross-site, cross-institutional learning aspect of GCP. The program was designed, in part, to respond to the challenge stated in the RFTOP: “To achieve results, conservation best practices should be disseminated widely for broader adoption and practice. However, implementers often do not have the time or mandate to share knowledge across institutions. Funding typically goes to one institution with the hope that knowledge generated will be shared.” The evaluation SOW and our initial meetings with USAID made it clear that USAID was interested in understanding the roles and value of centrally funded programs such as GCP. One such role would be to facilitate global, inter-institutional learning and knowledge-sharing. This evaluation was clearly meant to be a programmatic evaluation, not an evaluation of the performance of particular partners or sites: “While the evaluation will include site visits, their purpose is not to attempt to capture the full impact and results at the sites but to document the impact of being part of a larger program in terms of cross-site and cross-institutional learning.” Our SOW for this evaluation also asked us to evaluate “the adoption of key concepts, particularly landscape planning and conservation and adaptive management within the GCP family of projects and as a function of GCP partnerships.” The evaluation methodology we employed—summarized in the following section—was designed to achieve the purpose and objectives laid out in our SOW. GCP EVALUATION FINAL REPORT 3 2.0 EVALUATION METHODOLOGY The RFTOP from USAID provided three general lines of inquiry for the evaluation: 1. Document site-level conservation results; 2. Assess development and adoption of best practices within biodiversity conservation promoted by GCP, including cross-partner, cross-site aspects of such learning; and 3. Document the added value of the centrally managed, multi-institutional GCP program. The evaluation relies on three levels of information that reflect the lines of inquiry described above. We worked up from information from the sites (e.g., what are best practices employed at the site and how are they performing), to the network of organizations that manage them (e.g., how are best practices developed), to the GCP program that supports them (e.g., how is development and transfer of best practices facilitated). This provided a comprehensive view of how the program comes together as a whole. This was fundamentally an evaluation of the program—not of partner or site-level performance. As part of a consultative process, we asked GCP NGO partners to suggest specific questions that they wanted the evaluation to address, within the general lines of inquiry stated in the RFTOP. We viewed this step as important in ensuring that the results of the evaluation would be as relevant as possible to all involved in GCP. We synthesized those questions into a revised list, which was reviewed by GCP CTOs and partner NGO representatives. In addition, we discussed possible methods and data sources for answering those questions. Based on the feedback we received in this process, we developed a final list of evaluation questions (Box 2.1), and a proposed strategy for answering each. We provided a summary document for final approval to GCP CTOs and partner NGO representatives, which we refer to as our evaluation framework (Annex B). Box 2.1. Evaluation Questions Document Site-Level Conservation Results 1. What have been the challenges faced at the site level and on which have site managers made the most progress? 2. What are the landscape-scale conservation planning approaches used by GCP partners? What influence has GCP had in their development? 3. What are the adaptive management approaches used by GCP partners? What influence has GCP had in their development? 4. What best practices developed? 5. What suite of partners was involved at sites? How does this correlate with ability to overcome challenges? How does this makeup compare across GCP sites? 6. How has GCP funding affected site-level conservation, considering consistency, flexibility, and relative contribution to overall site funding? Assess development and adoption of best practices within biodiversity conservation promoted by the GCP, including cross-partner, cross-site aspects of such learning 7. How did grantee formulate best practices to address limiting factors at site? 8. How did grantee share learning on best practices at site with others (cross GCP partner, cross site)? Document the added value of the centrally managed, multi-institutional GCP program 9. Did GCP successfully promote cross-institutional learning? If so, what were the most effective mechanisms? 10. From a site perspective, what has central funding helped them achieve that mission funding has not? 11. Did sites benefit from multi-institutional collaboration at headquarters level brought about by the GCP? 12. How has USAID GCP performed administratively as a donor? 4 GCP EVALUATION FINAL REPORT We gathered information from a variety of sources, including: • GCP documents provided by USAID and GCP partners (Annex G); • Personal interviews with CTOs (Annex E); • Personal interviews with GCP partner representatives (Annex E); • A Web-based survey of managers of sites supported by GCP (Annex C); • A Web-based survey of GCP partner representatives (Annex D); and • Site visits to a subset of projects in the GCP portfolio in Central America, East Africa, and Brazil (Annex F). The quantitative information that we present in this report is from the Web-based surveys. We used interviews and site visits to validate the information from the surveys, to supplement it, and to develop a richer understanding of its meaning. For the sake of concise presentation, we do not always provide a site-level anecdote or perspective on the quantitative findings, but our statements and conclusions are supported by the interviews and our personal observations from the site visits. Several points merit mention or explanation on the data collection. First, GCP CTOs and NGO partner representatives were very cooperative and constructive, providing candid and informative interviews. Second, NGO partner representatives and site managers were highly responsive to our request to participate in Web￾based surveys. All six NGO partner representatives participated, and 26 sites participated (only the Komodo National Park and Atlantic Forest sites did not participate in the survey). Third, partner NGOs provided excellent support for site visits. Each visit provided us an opportunity to understand the projects’ scale, context, and challenges through meeting with site-based staff and local partners. We also used the site visits to validate the Web-survey results by crosschecking responses with our observations and further questioning on the ground.1 Note: Following professional standards for independent evaluations, it has been our practice to keep individual responses in surveys and interviews confidential, and not attribute particular responses to specific individuals. We then analyzed and presented the data gathered in the evaluation to GCP CTOs and NGO partners to check its validity and to test our preliminary conclusions. We documented feedback, made relevant corrections to the information, and formulated our final conclusions. GCP CTOs and NGO partners were also provided an opportunity to review and comment on the draft report. GCP EVALUATION FINAL REPORT 5 1 BOX 2.2. Summary of Evaluation Process • Kickoff meeting with USAID and GCP partners at GCP Annual Meeting, 26 June 2007; • Evaluation questions developed with USAID and GCP partners; • Evaluation framework developed, 23 August 2007 (Annex B); • Web-based survey to site-level managers and GCP partner representatives (Annexes C and D); • Individual interviews with CTOs, August 2007 and January 2008 (Annex E); • Individual interviews with GCP partner representatives, August 2007 (Annex E); • Site Visits (Annex F): − Central America (TNC, WCS) (October 2007), − East Africa (WWF, AWF) (November-December 2007), and − Brazil (CI) (February 2008); • Presentation of preliminary results to USAID and GCP partners; • Draft report to USAID & GCP partners for review; • Comments from USAID & GCP partners; and • Final report to USAID. 3.0 RESULTS 3.1 SITE-LEVEL RESULTS 3.1.1 What have been the challenges faced at the site level and on which have site managers made the most progress? Information about site-level challenges and progress came from two questions on the site-level survey (Annex C), which asked site managers to rate the degree to which eight different factors were limiting conservation at their site before GCP, and today or when GCP funding concluded (see Table 1.1 for funding periods). These eight factors are: • Conservation design (including a threats-based approach and landscape scale), • Management system (including monitoring and evaluation [M&E] and adaptive management), • Stakeholder engagement, • Government policy and legislation, • Institutional capacity, • Economic context, • Illegal activities, and • Financial sustainability. Several of these factors correspond to the principles listed as essential elements or ingredients of the GCP approach in the RFAs for GCP I and GCP II (Box 1.1 and Annex H). Respondents were asked to rate whether each of these factors was: • Not a problem, • A manageable problem, • A serious barrier to conservation, or • Prevented conservation. The results are presented graphically in Figure 3.1 and Figure 3.2 below. For each of the eight factors, there was a clear reduction in the rating of the factor as an impediment to conservation; that is, a clear improvement in the conservation situation during the period of GCP funding. Before GCP, the average rating for each of the eight factors was in the “serious barrier” category, or worse (i.e., “prevents conservation”). As shown in Figure 3.1, the average ratings for four of the eight factors changed from “serious barriers” to “manageable problems,” crossing a critical threshold. This clearly suggests success in addressing the following barriers to conservation: project design, management system, stakeholder engagement, and institutional capacity. These factors changed from “serious barriers” to “manageable problems” at many sites. The threats-based, landscape-scale approach supported by GCP seems to have successfully influenced these limiting factors. However, another four of the eight factors—government policy and legislation, economic context, illegal activities, and financial sustainability—still “prevent conservation” or are a “serious barrier” at many sites. Although these four factors remain problems, the data show that for each improvement has occurred during the period of GCP support. Financial sustainability was at the beginning, and remains, the limiting factor rated as the biggest remaining problem at these GCP sites. Most sites reported a diversity of sources of funding beyond GCP support, including private donors at all sites and other foreign governments at 39 percent of sites. However, respondents at 79 percent of GCP sites felt that the amount of funding they received was insufficient to perform necessary 6 GCP EVALUATION FINAL REPORT conservation work. Most sites reported a dependency on GCP funding, with 83 percent relying on GCP funding to maintain their core activities. Sixty-five percent of sites reported having partial funding, and 35 percent reported facing uncertain futures at the conclusion of GCP support. One USAID CTO said that pushing conservation organizations to “shift from the crisis mentality of conservation” and move in the direction of financially self-sustaining conservation was intended to be a big part of GCP from its inception. Building a mix of funding sources at the site level, such as endowments, government funding, user fees, and enterprises was supposed to be happening at GCP sites. But, said this CTO, “nothing has come of that,” frustrated that there has been so little progress on financial planning by partners. FIGURE 3.1. SITE-LEVEL RATINGS OF LIMITING FACTORS AS A PROBLEM FOR CONSERVATION BEFORE AND AFTER GCP SUPPORT Limiting Factors to Conservation -- Before and After GCP Project Design Management System Stakeholder Engagement Gov't Policy and Legislation Institutional Capacity Economic Context Illegal Activities Financial Sustainability Limiting Factor Degree of Limiting Facto B e fo r e G C P F u n d i n g A f t e r G C P F und i n g P r even t s Co n s e r v a t i o n No t a Pr ob l e m Ma na g e a b le Pr ob l e m Se r i o u s B a rri e r Another serious limiting factor, both at the beginning and end of GCP, is illegal activities and lack of enforcement of laws supporting conservation. During our site visits, project managers commonly identified illegal activities and lack of enforcement as a common challenge (fishery regulations for conch, lobster, and fishing spawning aggregations at Glover’s Reef in Belize; illegal incursions by foreign fishing fleets in the Eastern African Marine Ecoregion; local fishing regulations at Kiunga, Kenya; enforcement of community agreements regarding woodcutting and grazing in the Kittenden wildlife corridor on the north side of Mt. Kilimanjaro, on the Kenya-Tanzania border; and enforcement of natural forest set-asides on cattle ranches and farms in the Cerrado of Brazil). The graph in Figure 3.2, showing the percentage of sites at which each factor was a serious barrier or prevented conservation before and after GCP, reflects these results in a slightly different way. It can be seen that project design, management system, stakeholder engagement, and institutional capacity remain big problems at fewer than 10 percent of the sites, whereas the other four factors remain problems at between 30 and nearly 70 percent of the sites. Again, financial sustainability remains the biggest unaddressed barrier to conservation, remaining a problem at approximately 67 percent of the sites. GCP EVALUATION FINAL REPORT 7 FIGURE 3.2. PERCENTAGE OF SITES RATING EACH LIMITING FACTOR AS A MAJOR PROBLEM FOR CONSERVATION BEFORE AND AFTER GCP SUPPORT Percentage of Sites Scoring "Serious Barrier" or Higher for Each Lim 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Design Management System Stakeholder Engagement Gov't Policy & Legislation Institutional Capacity Economic Context Illegal Activities Financial Sustainability Pre-project Post-projec Some CTOs and NGO partners challenged the importance of reporting on limiting factors in the evaluation that were “outside the control” of conservation projects, and consequently showed little improvement. Such a position suggests a view of conservation design that is more limited than that of a landscape-scale, threats-based approach, an approach in which conservation planners are challenged to move all factors that affect the conservation target, both directly and indirectly, into their sphere of influence. An important finding, related to our measure of limiting factors, is that the Performance Management Plan (PMP) indicators that GCP partners report to USAID, “hectares under improved management” and “hectares under effective management,” measure results at such a high level that they are not especially useful for program managers tracking quarterly or yearly progress. Also, these high-level indicators do not directly measure progress toward abating threats to biodiversity or toward addressing the limiting factors to conservation. Although our data suggest that progress has been made in addressing limiting factors at GCP sites, it is difficult to attribute with certainty the progress made at these sites to GCP. In order to do so, the following information would be required: a) baseline measurements of key indicators; b) counterfactuals, either control sites or projections of changes at the sites in the absence of interventions; c) regular measurements of key indicators; and d) information on complementary funding for the sites, and complementary interventions that affected the sites. 3.1.2 What are the landscape-scale conservation planning approaches used by GCP partners? What influence has GCP had in their development? We examined two components of conservation planning by GCP partners: design based on reducing threats to conservation targets, and design at relatively large spatial scales. We found that GCP partners generally utilized a threats-based approach to conservation, and planned at a landscape or seascape scale. We observed that most NGO partners, on an institutional level, were influenced and assisted by GCP in making a shift to threats￾based, landscape-level planning. Among the most notable examples were the African Wildlife Foundation’s (AWF’s) Heartlands and Wildlife Conservation Society’s (WCS’s) Living Landscapes, as well as the support to the World Wildlife Fund’s (WWF’s) ecoregion-based conservation approach. 8 GCP EVALUATION FINAL REPORT 3.1.2.1 Threats-Based Planning Approach All but one of the 24 sites responding to the Web-based survey on this issue stated that they use a threats-based approach in conservation design, and five of six partner representatives said that their organization uses a threats-based approach. More than half of sites responding to the survey reported that they had formally analyzed and documented the specific elements or steps of the process of a threats-based approach to conservation design (identification of direct/proximate threats, prioritization of direct threats, identification of causes of threats, development of activities to address the causes), as seen in Figure 3.3. This is a strong, positive result, although there is room for improvement in formalizing this approach across all sites. 79% 17% 54% 42% 54% 38% 4% 58% 38% FIGURE 3.3. USE OF SPECIFIC ELEMENTS OF THREATS-BASED APPROACH AT GCP SITES Use of Specific Elem ents of the Threats-Based Approach (n=24) 4% 4% 4% Identification of Threats Prioritization of Threats Indentification of Causes of Threats Development of Strategies to Address Threats Threats-Based App roa ch E lem en ts 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Analyzed but not d o cum en ted In form ally analyzed and d o cum en ted Form ally analyzed and d o cum en ted Both the site-level and GCP partner representative’s surveys show that use of a threats-based approach in conservation planning is attributed in part to GCP’s influence. Results from the site-level survey are graphed in Figure 3.4. The GCP partner representative’s survey gave a more mixed picture of GCP’s influence: two of the six partners said that GCP did not affect the way the organization designs conservation programs, one of the six said that GCP had some influence, and three of the six said that GCP was the major influence (see Annex D, Question # 7). GCP EVALUATION FINAL REPORT 9 FIGURE 3.4. GCP’S INFLUENCE ON PROGRAM DESIGN (THREATS-BASED APPROACH) ACCORDING TO SITE-LEVEL SURVEY GCP's Influence on Conservation Approach (n=24) 0% 10% 20% 30% 40% 50% 60% 70% None Some Influence Major Influence Degree of GCP Influence Open-ended survey responses and personal interviews generated examples of the perceived value of a threats￾based approach (Box 3.1). One finding from our interviews is that some GCP partners view USAID’s definition of “threats-based” approach as excluding conservation “opportunities,” rather than a conceptual approach to understanding the drivers of conservation, and ensuring that interventions are based on a thorough understanding of context. GCP’s written guidance,2 however, is clear on this point, stating: A threats-based approach does not and should not exclude taking advantage of opportunities to conserve biodiversity. Indeed, an opportunity must in some way mitigate a threat to biodiversity or it would not be an opportunity. However, a threats approach helps managers not to fall into the trap of only taking advantage of opportunities. A threats approach requires prioritizing threats not on the basis of the opportunities to mitigate them but on the necessity of mitigating them due to their impact on conservation targets. A threats approach can help a manager decide if it is worthwhile or not to pursue an opportunity. In some cases, it may make sense not to pursue an opportunity if it is not sufficient to mitigate the critical threats and conserve the conservation targets. USAID, undated. USAID's Global Conservation Program and a threats approach. GCP program documentation. 10 GCP EVALUATION FINAL REPORT 2 BOX 3.1. Perceived Benefits of Threats-Based Approach Interviews and site visits provided examples of the perceived benefits of a threats-based approach. At one site (Brazil Cerrado-Pantanal), we were told that using a threats-based approach “was very helpful to us. It helped us organize in order to select the four main lines of activities we are carrying out... In my opinion it was a very useful change, to the threats-based approach, because before we were working without a guide. Now, it suggests, for example, that we need to work with municipalities [in order to address the causes of the threats].” A threats-based approach to strategic planning was supported in WWF’s Meso-American Reef (MAR) Ecoregion by GCP I funding to WWF for “global learning activities.” In this case the threats-based conceptual model and analysis led to a major shift in activities. According to Sylvia Marín, WWF Central America Director, who participated in this conservation planning process, the threats-based approach had a major influence on their strategy, which until then had been focused on marine threats, and was supporting activities related to Marine Protected Areas (MPAs) and basic science, and was working to influence the practices of fishing communities. The threats-based approach caused a reorientation toward a land-based focus, because they realized that the most urgent threats were coming from land—from agriculture, sedimentation, pollution, tourism and coastal development, and loss of mangroves. They shifted toward a “ridge to reef” approach to conservation in the MAR Ecoregion. The WWF Meso-American Reef Strategic Plan 2005-2009 can be found at: http://fosonline.org/Site_Documents/Grouped/WWFMARStrategicNov2004.pdf. 3.1.2.2 Landscape- or Seascape-Scale Planning Approach The geographic scale of conservation efforts varied widely, from 35 thousand hectares at WCS’s program at Glover’s Reef in Belize to 33 million hectares at WWF’s Eastern African Marine Ecoregion. The median spatial scale of GCP projects was approximately 1.2 million hectares. Descriptions of work at a landscape level varied across sites, ranging from a focused effort at a single site within a broader landscape to working on landscape￾level issues in tandem with focused efforts at core sites. Nevertheless, most efforts can be characterized as an attempt to address threats to conservation targets by working at the appropriate scale. Both the site-level and GCP partner representatives’ surveys show that GCP influenced the scale at which they carry out conservation work. Results from the site-level survey are graphed in Figure 3.5. The GCP partner representatives’ survey gave similar results (see Annex D, Question # 11). FIGURE 3.5. INFLUENCE OF GCP ON SPATIAL SCALE OF PROJECTS ACCORDING TO SITE-LEVEL SURVEY GCP's Influence on Spatial Scale of Projects (n=23) 60% 50% 40% 30% 20% 10% 0% None Some Influence Major Influence Degree of GCP Influence GCP EVALUATION FINAL REPORT 11 Open-ended survey responses and personal interviews generated a wealth of testimonials and examples (Box 3.2, 3.3, and 3.4). • “... participation in GCP allowed us to better understand the landscape-level rationale and effectively use it to engage more partners and interventions to impact conservation.” • “Prior to GCP [we] were already working at a spatial scale that extended beyond national parks and reserves. However ... GCP allowed [us] to develop spatial planning tools that help us to explicitly define the spatial extent and configuration of landscapes or seascapes sufficient to conserve viable populations of conservation targets.” • “[We] had already decide[ed] to operate at a broad scale. However, GCP funding helped the organization work out how to do this...” • “GCP allowed us to begin implementation of conservation work explicitly designed for the seascape scale. Until we received this funding, we had done little actual work guided by this scale of thinking.” BOX 3.2. WWF Ecoregions WWF’s original proposal for GCP I, in 1999, had an explicit “global learning” component. WWF was just developing its ecoregion-based conservation planning process at that time, and it saw in the GCP funding a way to develop an internal learning process to share lessons among WWF ecoregions. GCP learning money was spent to develop a system of learning and adaptive management at the ecoregion scale. Although this was not seen primarily as an interinstitutional vision of sharing lessons and best practices with other GCP partners, WWF did reach out to and interact with some other GCP partners working in some of the same ecoregions. They did so, in part, because of the spirit of cross-institutional collaboration that had developed under the Biodiversity Support Program (BSP), which was housed at WWF. The WWF East Africa Marine Ecoregion (EAME), for example, used funding from the GCP to help develop their ecoregion strategy. Even though it was not a GCP site, WWF’s MAR ecoregion also used GCP funding to develop its ecoregion strategy (see Box 3.1). The Bering Sea ecoregion benefited from GCP learning funds by allowing collaboration with The Nature Conservancy (TNC) to use some of their planning tools. BOX 3.3. Evolution of AWF’s Heartlands Planning Process AWF designed its Heartlands program in 1999 using a landscape planning and monitoring approach and system, which the GCP provided core funding to develop. According to AWF, the GCP “catalyzed” the organization’s approach to doing landscape-scale conservation, and provided the funding for implementing it. GCP funding also allowed AWF to learn from other conservation organizations. In 2000 and 2001, TNC provided technical assistance to AWF in landscape planning, and AWF learned and adapted TNCs “site conservation planning” approach. A site conservation planning expert from TNC worked with AWF in summer of 1999 in the Samburu landscape, and all summer of 2000 to help facilitate planning workshops in four AWF “Heartlands” landscapes. The initial engagement between the two “would have happened without GCP, but GCP supported the work and was a huge part of making it happen.” BOX 3.4. Development of the WCS Living Landscapes Program The GCP provided the core support to WCS that allowed it to establish the Living Landscapes Program (LLP). The program was started to develop strategic planning tools for WCS field sites, and now those tools are being disseminated and adopted widely across the organization. Living Landscapes “Technical Manuals” and “Bulletins” are used to share best practices related to many GCP core themes, such as conservation planning and adaptive management. The primary learning and sharing of knowledge in this case was intra-institutional. Although the GCP is currently funding work at four sites, the LLP has seen an explosion of interest within the organization: 12 sites now participate in the program, eight of which are not receiving GCP funding. From USAID’s point of view, GCP catalyzed a major shift in the way WCS approached conservation, moving it—through the LLP—from basic scientific research toward applied conservation action. At Glover’s Reef in Belize, WCS’s first tropical marine site, two decades of basic ecological research and monitoring of “landscape species” including the hawksbill turtle, Nassau grouper, and queen conch is still a foundation of conservation at the site, but has led to work at the seascape level with fishermen’s associations from communities on the mainland to monitor catch levels, and with the Belize Fisheries Department to enforce zoning regulations and harvest limits within the MPA. 12 GCP EVALUATION FINAL REPORT W 3.1.3 What are the adaptive management approaches used by GCP partners? What influence has GCP had in their development? 3.1.3.1 Adaptive Management Approaches Adaptive management, as used in GCP, refers to a process of modifying practices over time to improve performance. Interviews and surveys revealed that a precise definition of adaptive management was not agreed upon by USAID CTOs, GCP partner representatives, or site-level managers. We developed a list of 10 basic elements of adaptive management, based on technical principles, and asked site￾level managers to rate the degree to which the management of their conservation program employed each. The aggregated results from the site-level, Web-based survey are shown in Figure 3.6. The level of implementation of the various elements of adaptive management ranges widely and is reflected in the GCP partner representative’s survey (see Annex D, Question #13). Written management objectives Performance metrics/indicators Experimental and control sites Baseline data for metrics/indicators ri tten description of activities necessary to reach objectives Monitoring System for metrics/indicators Regular analysis of monitoring data Feedback mechanism to adapt mgm't based on result Documentation of successes and failures FIGURE 3.6. PERCENTAGE OF SITES EMPLOYING SPECIFIC ELEMENTS OF ADAPTIVE MANAGEMENT TO VARYING DEGREE Percentage of Sites Em p loy ing Specific Elements of Adaptive Management (n=23) 100% Not used s 80% Early stage s o f developmen t, only par tially or not y et used D evelop ed and u sed but 60% needs im provem en t W ell develop ed and fully used 40% 20% 0% Adaptive Management Element From a technical standpoint, a number of elements of adaptive management are underdeveloped, including regular analysis of monitoring data, and documentation of successes and failures. Also often lacking are feedback mechanisms to systematically adapt management accordingly to empirical evidence of the performance of specific interventions. GCP EVALUATION FINAL REPORT 13 When we posed the question of why there are weaknesses in some of these elements, we got a variety of responses. One was that the priority for GCP was formalizing and implementing a logical, threats-based planning process throughout the program, and such a logical planning process was seen as a prerequisite to adaptive management. Another kind of response was that the site-level programs are too young to have implemented more sophisticated adaptive management systems. Some respondents thought that monitoring and analysis is too expensive and time consuming given limited resources, and that available resources were better used for implementing activities. On the latter point, we observed some tendencies to confuse scientific research with simpler monitoring that would be sufficient for the purposes of basic adaptive management. This could be the result of a perception we heard expressed that in the past scientific research and analysis had received too much attention and resources, compared to implementation of on-the-ground conservation actions. We also heard strong statements of support for better monitoring, and support from USAID to implement it: “Monitoring results are really critical to know if what we are doing is working.” One result that stood out on both the site-level and GCP partner surveys relates to low levels of “documentation of successes and failures.” On the site-level survey, approximately 45 percent of the sites (10/22 sites) reported that documentation of results either was not being done, or was in the early stages of development, or only partially used. On the GCP partners survey, 67 percent (4/6 partner NGOs) reported that documentation was only in the early stages of development, or partially, or not yet used. Further to this point, GCP site managers implied in their responses that annual reporting to USAID did not constitute meaningful reporting of conservation performance, potentially because of a complaint we heard from NGO partners and CTOs alike that the performance metrics required in GCP were not useful (we observed ourselves that the metrics are too coarse to measure annual progress). One of the GCP II guiding principles was “Programs should integrate learning into program design,” and the statement “We support the learning and dissemination from both successes and failures that improve the design and management of programs,” as part of the further explanation of that learning principle. However, without documentation of successes and failures, the ability to learn and share lessons is obviously constrained. Of all the factors, the lowest levels of implementation were for what we called “experimental and control sites.” In fact, proving or attributing an outcome or result to an activity or intervention requires “counterfactual” evidence of some sort, but not necessarily always experimental or control sites. Perhaps the respondents were interpreting the question narrowly, and were imagining elaborate experimental designs with areas of intervention and non-intervention which they might have imagined would either be expensive or unethical in conservation settings. However, during our site visits we came across a number of cases in which a simple expansion of monitoring that was already happening in the area of intervention to a site outside that area would likely have produced robust evidence about whether the intervention was or was not working (e.g., reef monitoring inside and outside of a no-take zone). In any case, we often observed a lack of recognition of the need for counterfactual evidence to assess the performance of a conservation intervention. Such information is essential feedback for “adaptive” managers. As one site manager told us: “Control sites are an important issue for M&E—[our organization] gets tough questions from stakeholders about the benefits of MPAs and where the science is behind it. There are no control sites where data is being collected. It is very doable, but nobody is funding it.” 3.1.3.2 Influence of GCP on Adaptive Management According to our surveys of both site-level managers and GCP partner representatives, GCP is seen as a significant influence on the development and implementation of adaptive management approaches and systems. The site-level survey results provide one indication of this influence. As seen in Figure 3.7, almost 70 percent of the sites responding (16/23 sites) answered “We were already developing adaptive management, and GCP helped us to develop it further,” while another 17 percent of the sites (4/23 sites) said “GCP is responsible for our development of adaptive management.” 14 GCP EVALUATION FINAL REPORT BOX 3.5. Site Visit to East Africa S REY BECURO: BOTPH Acacia savanna landscape in AWF's Kilimanjaro Heartland, Tanzania, with Mt. Meru in the distance. In late November and early December, 2007, we visited two AWF Heartlands landscapes in Tanzania, the Kilimanjaro and Maasai Steppe Heartlands. These two landscapes have many ecological and social aspects in common. Visiting both provided an opportunity to see different aspects of AWF’s work, and sites at which they have been working for different amounts of time. The Kilimanjaro Heartland is a transnational landscape surrounding Mt. Kilimanjaro in northern Tanzania and southern Kenya. This transboundary conservation work is a prime example of one of the benefits of a centrally funded program like the GCP. Although USAID Mission staff in both Kenya and Tanzania told us that while, in principle, there was no reason USAID country missions could not design transboundary programs, on a practical level they did not think it would happen without central funding from GCP. The movement and migration of wildlife (such as elephants, wildebeest, and zebra) and hydrological boundaries are the two main criteria for defining the boundaries of these AWF “Heartlands” landscapes, and animal movement and water are interrelated. We traveled with Alfred Kikoti, an AWF elephant researcher, in the area between Mt. Kilimanjaro and Mt. Meru, where he maps the movements of radio-collared elephants that show the linkages in these landscapes and the influence of human activities on animal movement. On the north slopes of Mt. Kilimanjaro, in the Kittenden area, we saw where the last remaining forest corridor for the movement of elephants and other wildlife between the mountain forests and the swamps below was being protected through agreements with farming communities. In Box 3.3 we discuss the interinstitutional collaboration between AWF and TNC that occurred in these landscapes during in 2000 and 2001, when the threats-based, landscape-scale conservation planning and design process used by TNC was being adapted and applied here by AWF. We found that on the ground this collaboration has continued, and grown. TNC has recently provided assistance to AWF on land acquisition methodology – legal options such as trusts, and the pros and cons of the various options. TNC has developed this thoroughly in the US context. We found AWF staff wearing TNC hats after a recent exchange of technical experts. On this trip we also visited the secretariat of WWF’s Eastern African Marine Ecoregion (EAME) in Dar es Salaam, Tanzania. According to Amani Ngusaru, EAME leader, “The problem here is the scale. Eastern African Marine Protected Areas were not talking to each other, and money from GCP helped to ‘link these dots.’ You can’t conserve marine turtles at Mafia Island only. You have to look at the whole region, which is defined by oceanographic factors. Those factors define its boundaries from southern Somalia to northern South Africa.” Activities of the EAME Secretariat seek to reduce threats to marine biodiversity through work at the national and regional policy level. For example, the EAME Secretariat contributed to Tanzania’s national Coastal Management Strategy. In this huge marine ecoregion, WWF is now focusing its work with communities at two specific sites, Kiunga on the northern Kenya coast, and Quirimbas in northern Mozambique. According to Mr. Ngusaru, fundraising for site-based work is much easier than for large￾scale policy work. “Only WWF decided to shoulder the big-picture policy stuff, and put money into it through the GCP.” We had hoped to visit WWF’s Kiunga site, but USAID/Kenya would not allow us to travel because of fears of pre-election violence. We were able to speak with Sam Weru, the Kiunga Program Director, in Nairobi. One key message that he said he would like to have us reflect in the GCP evaluation was “Let’s not just do biology. We really have to integrate biology with socioeconomic development.” PHOTO: BRUCE BYERS Talking with Maasai men, AWF Maasai Steppe Heartland, Tanzania. Elephants, Tarangire National Park, Tanzania. PHOTO: BRUCE BYERS GCP EVALUATION FINAL REPORT 15 One-half of the GCP partner representatives (3/6 organizations) said that “We were developing an adaptive management system and GCP helped us to develop it further,” and one of the GCP partners stated that “GCP is responsible for our development of an adaptive management system.” FIGURE 3.7. GCP’S INFLUENCE ON SITE-LEVEL USE OF ADAPTIVE MANAGEMENT GCP's Influence on Site-level Use of Adaptive Management 100% 80% 60% 40% 20% 0% Not familiar W e already We were GCP is with adaptive em ployed developing responsible for management adaptive adap tive our m gm't and mgm't and developm ent GCP had little GCP helped us of adaptive influence to develop it mgm't further GCP Influence One example of GCP’s influence we heard about was that of the development of adaptive management within WWF. WWF was already developing adaptive management approaches through the Biodiversity Support Program, a predecessor of GCP, and early work by their Conservation Strategies Unit. GCP provided timely support for WWF to help found and participate in the Conservation Measures Partnership (CMP). That led eventually to the development of the “Open Standards for the Practice of Conservation” by the CMP (June 2004), and WWF’s adaptation of those standards for its own needs, “WWF Standards of Conservation Project and Programme Management” (February 2007). BOX 3.6. Conservation Measures Partnership In 2002, representatives of three GCP partner organizations presented a joint poster on “measuring effectiveness” at the Society for Conservation Biology annual meeting. Then in a side meeting they brought together the GCP partner organizations and USAID, plus some other international conservation NGOs, to discuss how to improve methods of measuring and reporting conservation progress. This led to a follow-up meeting in November, where the GCP NGO partner organizations and Foundations of Success agreed to establish the CMP. The first formal “learning grant” funded by GCP II was made to the CMP in 2003, supporting completion of the “Rosetta Stone” of conservation terms, and development of a set of workable open standards for the practice of conservation. The latter were translated into French and Spanish. Since CMP was established, its members have collectively developed: a) standard typologies for characterizing direct threats and conservation actions, b) process standards that define adaptive management best-practices for conservation actions, c) a standard approach for auditing the effectiveness of conservation projects, and d) a desktop software application for projects to implement the CMP adaptive management standards. In an interview one respondent said: “The CMP is an example of learning involving GCP partners, catalyzed through GCP funding. It is a success story for GCP, even though GCP can’t take credit for all of it.” 16 GCP EVALUATION FINAL REPORT 3.1.4 What best practices were developed? Our strategy for this evaluation assumed that successes at the site level would be documented, shared, and replicated at other sites and by other partners, and developed into shared “best practices”—that is, standard, state-of-the-art ways of addressing certain problems, limiting factors, or causes of threats. One source of information about site-level successes came from content analysis of the individual responses to Questions #24 and #31 on the site-level survey (Annex C). Site visits and document review were other sources of success stories. Successful activities and interventions identified include: • Incorporation of a sustainable livelihood approach in the Terai Arc (WWF); • Application of large-scale planning within the Eastern African Marine Ecoregion (WWF); • MPA design and establishment at Raja Ampat, Kimbe Bay, and Wakatobi (TNC); • Work with indigenous communities and organizations at Madidi (WCS); • Community agreements at Kilimanjaro Heartland (AWF); • Land trust in Maasai Steppe Heartland (AWF); • Whale shark observation standards for tourism in the MAR (TNC); • Spawning aggregation and fisheries monitoring techniques at Glover’s Reef (WCS); • Capacity building of local NGOs and working with business, private sector, and NGO partners in Cerrado￾Pantanal (CI); and • Development of community-based non-timber forest products (NTFP) enterprise development in Nepal (EWV). Given the generally low level of monitoring, evaluating, and documenting performance discussed in Section 3.1.3, a necessary condition for sharing and replicating success stories, we are not fully convinced that any of these successes have developed to the level of “best practices.” However, they certainly indicate some positive outcomes of GCP investments. 3.1.5 What suite of partners was involved at sites? One of the guiding principles of GCP was “Programs should strengthen in-country capacity and foster collaboration.” Conservation of natural systems depends critically on the engagement and commitment of key stakeholders—local people, government, corporations, NGOs, and donor institutions. Establishing strategic partnerships to help achieve conservation goals is key.” USAID and GCP partners themselves were interested in the array and diversity of partners that had been engaged at the site-level, and the results from the survey are shown in Figure 3.7. In summary: • Diverse partners are usually involved: the majority of sites had at least six types of partners. • Local partnerships are generally strong: 82 percent of sites had local NGO partners, 55 percent had local government partners, and 64 percent had local community partners. • Businesses and private individuals are the least-common partners: only 27 percent of sites had private individuals as partners, and only 14 percent had business partners. GCP EVALUATION FINAL REPORT 17 FIGURE 3.8. TYPES OF INSTITUTIONS ENGAGED AS PARTNERS AT THE SITE LEVEL Partnership Engagement (n=22) 82% 59% 55% 73% 64% 27% 14% 55% 5% Local NGOs Int'l NGOs Local Gov't National Gov't Local Communities Private Individuals Businesses Universities Other Partnering Institutions Percentage of Sites Although this survey question provided information about the diversity of partners at GCP sites, we did not evaluate the quality of engagement of the different partners, something that may be crucial in effectively addressing certain barriers to conservation. For example, we learned that illegal activities and lack of enforcement of laws were problems at several of the sites we visited. Although national or local governments responsible for enforcement of laws affecting conservation were usually nominal partners of GCP at those sites, we found that they often were not fully engaged, effective partners. Although partnerships with private individuals and businesses were not common, we identified some promising examples of fruitful collaborations with these types of partners. In Brazil, CI has partnered with the soybean buyer, Bunge; and in Honduras, TNC is working with the private conservation area of Cayos Cochinos. 3.1.6 How has GCP funding affected site-level conservation, considering consistency, flexibility and relative contribution to overall site funding? The evaluation provided us with a clear picture of the importance of consistent and flexible funding. GCP sites have had relatively consistent GCP funding, which has in part allowed them to engage in meaningful long-term planning. Some NGO partners have described GCP as flexible in accommodating modifications in those plans as site managers deal with dynamic contexts and as site-level work evolves, although few partners found USAID more flexible than other donors. We performed a number of quantitative analyses to look for a relationship between various characteristics of funding (years of GCP support, relative proportion of GCP funding to total funding, sufficiency of total funding relative to perceived needs, total overall funding per unit area) and progress in addressing the factors limiting conservation. Given the relatively small number of sites in our analysis and the large number of potential variables, we were not able to generate strong conclusions. 18 GCP EVALUATION FINAL REPORT PHOTO: BRUCE BYERS PHOTO: BRUCE BYERS BOX 3.7. Site Visit to Brazil Clearing native woodland for cattle pasture, soybean and sugarcane farming, and charcoal production is a threat to biodiversity and ecosystem services in the Cerrado-Pantanal landscape of central Brazil. In February 2008, we visited Conservation International’s Brazil program to learn more about GCP’s support of conservation here. CI is working to address these threats with private ranchers and farmers, local governments, and private corporations. They support protected areas and encourage compliance among private landowners with the legal requirement that 20 percent of each private landholding must be left in native vegetation. One of the innovations of the program is a partnership with Bunge, a large international soybean-exporting corporation and a major purchaser of soybeans in this area, to promote compliance with the legal requirement for conserving native woodland. Another innovation of this program is to perform on-the￾ground work through local NGOs, thereby building local capacity. We visited the municipality of Rio Negro, where CI is working with ranchers through the local NGO APREMARINE. We learned on our site visit that CI-Brazil was notably influenced by GCP’s emphasis on the threats-based approach, resulting in a significant reorientation of their conservation design to include, for example, work with municipalities. One site manager told us, “It forced us to focus on different priorities that we would not have thought of.” Another positive observation was the complementarity of GCP and mission funding in Brazil. USAID/Brazil saw the GCP as a critical source of funding for regions, such as the Cerrado-Pantanal, where the mission cannot invest because its efforts are fully focused on the Amazon and Atlantic forest ecosystems. The site visit also illustrated two weaknesses of GCP. First, despite CI-Brazil’s emphasis on legal compliance by private landowners with forest set-aside requirements in a region where most landowners violate the law with impunity, there was no awareness of GCP’s work on enforcement (even though it was conducted by researchers at CI itself). This appears to be the result of the failure to disseminate GCP learning products to sites. Second, although there are no other GCP-funded sites in Brazil, GCP partner organizations are working in close proximity to CI on similar issues in the Cerrado, especially TNC, no formal collaboration or knowledge sharing is occurring—something that might have been facilitated by the GCP. Left: Waterfall marking the geographic transition between Brazil’s Cerrado and Pantanal biomes. A Brazilian charcoal maker takes shelter from the rain in his kiln. Clearing native woodland for cattle pasture is one threat to biodiversity and ecosystem services in the Cerrado-Pantanal landscape of Brazil. Right: CI and APREMARINE staff in the field (from left to right): Elisângela Arruda, Environmental Education Specialist, APREMARINE; Ricardo Bini, Director of APREMARINE; Sandro Menezes, CI Pantanal Program Manager. PHOTO: BRUCE BYERS PHOTO: BRUCE BYERS GCP EVALUATION FINAL REPORT 19 3.2 DEVELOPMENT AND ADOPTION OF BEST PRACTICES One objective of this evaluation was to “…assess development and adoption of best practices within biodiversity conservation promoted by GCP, including cross-partner, cross-site aspects of such learning.” In our evaluation framework (Annex B), we broke this question into two parts, a “learning from” aspect and a “teaching to” aspect, as discussed in the following sections. 3.2.1 How did grantee formulate best practices to address limiting factors at site? To assess the sources of “best practices” at the site and organizational levels, we asked “Who or what has significantly influenced your current practices in addressing the themes below?” on both versions of the Web￾based survey (Annexes C and D). A variety of sources of learning were listed, and the themes were each of the limiting factors to conservation already discussed. The results from the site-level survey are shown in Table 3.1, which can be summarized as follows: • The highest numbers throughout the table are in column one, indicating that the overwhelming influence on current practices at the site level is experience from within each partner organization. • The second most important source of influence is the conservation community in general, through publications or presentations. • The influence of other GCP partners, through formal GCP learning activities or annual meetings, is uniformly quite low. • Only for current practices related to conservation design did the influence of GCP partner NGOs—in this case through direct communication, not formal learning activities—rise to a relatively high level (52%). TABLE 3.1. SOURCES OF CURRENT PRACTICES BY CONSERVATION LIMITING FACTOR Significantly Influencing Site’s Current Practice (n=23) Our own experience GCP partner NGOs via direct communication NGOs that are not GCP partners via direct communication GCP partners via formal GCP "learning activities" GCP partners via GCP meetings Guidance from USAID's GCP staff Conservation community via publications or presentations Conservation design 87% 52% 30% 22% 30% 35% 70% Management system 87% 44% 35% 17% 17% 35% 52% Stakeholder engagement 100% 32% 55% 18% 23% 23% 50% Government policy and legislation 91% 19% 48% 14% 19% 29% 33% Institutional capacity 91% 41% 55% 23% 23% 23% 46% Economic context 83% 35% 44% 22% 22% 30% 52% Compliance and enforcement 86% 29% 52% 24% 24% 33% 52% Financial sustainability 76% 38% 48% 14% 29% 33% 52% A similar picture emerges from the results of the survey of GCP partner representatives (see Annex D, Question #16 results). For all GCP partners, experience internal to the organization was always seen as the main source of current practices. Influences from other GCP partner organizations through direct communication were moderately important overall, and quite strong as a source of best practices for conservation design and institutional capacity building, where five of the six partners reported other GCP 20 GCP EVALUATION FINAL REPORT 3 partners as an important influence. As for formal GCP learning activities, only for conservation design did these raise to a significant level, with three of six GCP partners reporting an influence of formal learning activities on this conservation element. A striking example that illustrates the problems of formal learning in GCP is that of a study on enforcement carried out by Conservation International. 3 Akella, A.S., J.B. Cannon. 2004. Strengthening the weakest links: strategies for improving the enforcement of environmental laws globally. Washington, DC: Conservation International. The study is, in our opinion, excellent. It uses an analytic framework for diagnosing weak links in enforcement systems and provides a series of case studies. Despite the pervasive problems with enforcement experienced by GCP sites, none of the sites we visited were aware of this study—including the CI site we visited in Brazil. Based on interviews with both the authors of the study and the CTOs responsible, it appears there was no system in place for disseminating this study to the field or to USAID missions. 3.2.2 How did grantee share learning on best practices at site with others (cross GCP partner, cross site)? To assess the sharing of best practices at the site and organizational levels, we asked “If you have documented your own ‘best practices’, on which themes and with whom have you shared them?” on both versions of the Web-based survey (Annexes C and D). Results from the site-based survey are shown in Table 3.2, and can be summarized as follows: • The main target audience for sharing is the conservation community in general, not GCP partners in particular, with the highest numbers in the table in the right column. • Informal sharing, directly with GCP partner NGOs, occurs in the enabling environment created by GCP. • Formal (GCP II) learning activities are not listed as a major method for sharing best practices. TABLE 3.2. SHARING OF DOCUMENTED BEST PRACTICES BY CONSERVATION LIMITING FACTOR Percentage of Sites that have Documented Best Practices and Shared Them with Other Organizations (n=21) Directly Directly With GCP Conservation w/GCP w/other NGOs partners via With GCP Directly w/ community via partner that are not formal "learning partners via USAID's GCP publications or NGOs GCP partners activities” GCP meetings staff presentations Conservation design Special scale of 67% 67% 29% 24% 57% 86% conservation Adaptive management Stakeholder 70% 60% 35% 25% 55% 85% 44% 56% 22% 22% 39% 67% engagement Government 50% 69% 25% 19% 50% 69% policy & legislation Institutional 50% 75% 19% 31% 38% 69% capacity Economic 60% 67% 27% 33% 53% 47% pressures Compliance and 47% 40% 33% 20% 33% 67% enforcement Financial 33% 56% 17% 17% 33% 72% sustainability 39% 62% 23% 31% 46% 54% GCP EVALUATION FINAL REPORT 21 PHOTO: JARED HARDNER PHOTO: JARED HARDNER BOX 3.8. Site Visit to Central America In October of 2007 we visited two marine sites in the GCP portfolio, Glover’s Reef Marine Reserve in Belize, supported by the Wildlife Conservation Society’s Living Landscapes Program, and Cayos Cochinos Marine Monument and Scientific Station in Honduras, supported by The Nature Conservancy’s Meso-American Reef (MAR) Program. The two sites protect coral reef systems and have scientific stations where marine research and education is conducted by a variety of organizations. The primary threats to these areas are overfishing and coastal pollution. Glover’s Reef is an interesting example of WCS’s landscape-scale approach to conservation, in which the conservation landscape, or in this case seascape, is determined by the ecological requirements of selected target species. At Glover’s Reef, these target species include the Nassau grouper, hawksbill turtle, and queen conch. On our visit to Glover’s Reef, WCS-Belize staff described the scientific basis for their seascape definition, and explained how they used a threats-based approach to design a conservation plan. A primary focus of this plan is engagement with coastal fishermen to build awareness of the need for regulating the harvest of fish and shellfish, and gain their support for conservation of the marine protected area at Glover’s Reef. One innovative aspect of this plan is cooperation of local fishermen in fisheries monitoring; another is using funds from private dive lodge owners at Glover’s Reef to assist the Belize Ministry of Fisheries in enforcing catch limits for selected species. Research on spawning aggregations of Nassau grouper at Glover’s Reef led to the identification of 11 other spawning sites in the region, in collaboration with TNC, WWF, and other NGOs, and to formation of the Spawning Aggregation Working Group. WCS and TNC have worked together on “visual census” monitoring protocols for spawning aggregations. In Belize, spawning aggregations were closed to fishing five years ago, but enforcement of this regulation is not sufficient, and spawning aggregations continue to decline. WCS staff emphasized the importance of monitoring to know whether conservation interventions are really working. Cayos Cochinos is a privately owned conservation area that receives technical assistance from TNC in areas such as management planning. Most striking about Cayos Cochinos is the business-like management culture. This is the only area we visited in the evaluation that is financially self-sufficient. Building on the private business experience and values of its founders and board, the Cayos Cochinos Foundation has capitalized on opportunities to generate substantial business revenues on the islands that cover the majority of the costs of conservation management. Among their profit-making business innovations is hosting international scientific and educational programs and the filming of “reality” television shows. Another positive feature of work at Cayos Cochinos is the sharing of biological monitoring information across sites in the TNC MAR Program, and the use of control sites—which is otherwise rare among GCP sites. An undeveloped bay on principal island of Cayos Cochinos Marine Monument and Scientific Station. Cayos Cochinos is privately owned, and supported in part by TNC. PHOTO: JARED HARDNER WCS staff at the scientific station in Glover’s Reef in Belize (from left to right): Archie Carr III, Regional Coordinator of Mesoamerica Caribbean Program; Janet Gibson, Glover’s Reef Program Director; and David Wilke, Living Landscapes Program. These site visits also illustrated some weaknesses of GCP. First, despite having very similar characteristics and relatively close proximity, Glover’s Reef and Cayos Cochinos have had no direct, formal collaboration or knowledge-sharing. That struck us as a lost opportunity for GCP to add value. Second, in the case of Glover’s Reef, enforcement of laws is a significant problem, but there was no awareness of the work done on enforcement through GCP’s formal learning activities. This appeared to be the result of the failure to disseminate the products of the formal learning activities to sites. 22 GCP EVALUATION FINAL REPORT As for targets of sharing of best practices a similar picture emerges from the results of the survey of GCP partner representatives (see Annex D, Question #18 results). The main target audience for sharing was the conservation community in general, although respectable numbers of GCP partners reported sharing information about conservation design and spatial scale of conservation directly with GCP partner NGOs. Fewer reported sharing through formal learning activities than informal direct communication, but some sharing was reported through the formal mechanism. BOX 3.9. Selecting Conservation Targets for Landscape-Scale Priority Setting One example of sharing of information among GCP partners is the comparative assessment of the processes for selecting conservation targets used by five of the six GCP partners. At a workshop held in Washington, DC in 2005, representatives from AWF, CI, TNC, WCS, and WWF compared landscape-scale target setting approaches, using AWF’s Samburu Heartland in Kenya as a focal example. Although approaches to target selection varied considerable among the five GCP partner organizations, in the end the sets of conservation targets selected in this landscape were quite similar in four of the five organizations. http://www.worldwildlife.org/science/pubs/landscapeplanningreport.pdf. 3.3 ADDED VALUE OF GCP The third main objective of this evaluation was to “…document the added value of the centrally managed, multi-institutional GCP program.” Our evaluation framework laid out a number of questions related to this “added value,” and these are discussed below. 3.3.1 Did GCP successfully promote cross-institutional learning? If so, what were the most effective mechanisms? GCP I RFA (1999) did not use the word “learning” or list learning as a broad objective of the program, but comparing experience, analysis, and sharing of lessons was intended by program designers. Four of the six GCP I partners proposed activities designed for learning better practices for landscape-scale conservation within their own organizations, and two used the word “learning” in their descriptions of these activities. For example, WWF’s “global learning” component in GCP I was mainly aimed internally, but involved other GCP partners and facilitated some important cross-institutional learning. The RFA for GCP II (2002) did list learning explicitly as a broad objective of the program. Both GCP I and GCP II enabled, facilitated, and catalyzed cross-institutional learning through both informal and formal mechanisms. Themes on which institutions shared knowledge included: • Landscape-scale conservation design, including threats-based approach; • Adaptive management standards (Conservation Measures Partnership); • Comparative terminology for conservation (“Rosetta Stone” activity); • Hydrological processes and landscape-scale planning (Brazil workshop); • Conservation target setting (Samburu Heartland workshop); and • Marine protected areas planning (Tropical MPA Network). According to the site-level survey, one of the most effective mechanisms for learning involved on-the-ground collaboration with other NGOs. In those cases where such interactions have occurred (e.g., EWV-CI collaboration in the Philippines, EWV-AWF collaboration in Kenya, and AWF-TNC collaboration in Tanzania) participating organizations speak quite highly of the experience. Cross-site exchanges and visits were also rated highly as a source of learning. As one GCP partner put it: “We’re all field biologists and we want to stand in the dirt and see what others are doing.” There was an attempt early in GCP II to create a formal structure to promote cross-institutional learning, including the formation of a Learning Panel of NGO representatives, who were charged with developing a GCP EVALUATION FINAL REPORT 23 4 collaborative agenda for documenting new field-based knowledge and sharing it across GCP partners. In general, USAID CTOs and NGO partners did not feel that this formal approach to learning had worked very effectively. A number of reasons were offered, including weaknesses in the following factors: • Willingness and genuine interest among parties to share knowledge, • Realistic financial incentives to cover the costs (staff time, travel, communications) of sharing knowledge, • Cooperative and constructive individuals representing the organizations involved, and • Leadership of the knowledge-sharing process. BOX 3.10. Developing a “Rosetta Stone” to Compare Conservation Terminology across Organizations WWF organized a meeting with other GCP partners in the Adirondacks in 2000 as part of its global learning activity during GCP I, and according to one participant they “felt like they needed a translator; they realized they needed to be using the same language.” Participants from the six GCP partners’ organizations realized that in the process of developing their own approaches and systems for conservation planning, they had also developed their own terminology for describing key concepts, and that made communication and comparison a challenge. The “Rosetta Stone” eventually grew out of that experience. In tabular form, the Rosetta Stone compares the various project management systems used by the conservation organizations in the CMP, most of which are GCP partners. It also provides a dictionary and thesaurus of conservation terms. http://conservationmeasures.org/Rosetta2/ 3.3.2 Did sites benefit from multi-institutional collaboration at headquarters level brought about by GCP? A confounding factor in the evaluation was an apparent lack of awareness among site-level managers of the GCP program’s emphasis on cross-institutional learning. Indeed, most site managers with whom we spoke did not even know about the formal learning activities. Our survey attempted to gather information about best practices adopted at sites as a result of interactions between NGO partners’ US-based headquarters, but the results are somewhat confusing.4 In the site-level survey, 48% of sites said they adopted best practices based on interactions at the HQ level; 52% said no, they did not (Annex C, Question #30). The site-level surveys do not show any clear pattern by partner – that is, five of the six partners had sites with both “Yes” and “No” answers. Even sites where there has been clear collaboration between GCP partners sometimes answered “No” – a puzzling result. Content analysis of individual site-level survey responses provide few clear examples of adoption of best practices at sites based on headquarter-level GCP interactions. The fact that site-level managers often did not know much about GCP or its overall objectives, including its objective of cross-institutional learning, may reflect the absence of a GCP communications system that could have led to such an understanding. USAID CTOs and GCP partner representatives were not surprised by this, saying that they would not necessarily have expected staff at the site to recognize GCP’s influence on approaches and practices brought down from the headquarters level—but that they may have been influenced nevertheless. Even so, this may be evidence of a missed opportunity to involve site managers more directly in a global program of learning and sharing. However, NGO partner representatives describe collaborations with other NGOs as a product of participation in GCP. The GCP partner representative survey (Annex D, Question #20) gave the following key results: • All partner representatives reported developing partnerships or collaborations with other GCP partners. • The CMP was mentioned by three partners as a good example of headquarters-level collaboration. • The Hydrological Processes and Tropical MPA Networks learning activities were listed as examples by one partner each. • Some partners collaborated mainly on a bilateral basis with one other GCP partner. 24 GCP EVALUATION FINAL REPORT One respondent wrote: The most powerful collaboration borne out of GCP was the Conservation Measures Partnership, which brought a much needed collective conversation on the common questions and best practices for how we measure success in conservation. GCP learning funds helped support analysis, exchanges, and partnerships on a variety of issues. GCP was particularly important during the early days of the program when there wasn’t as strong a culture of NGO collaboration as there is now today. Not all collaborations grew into larger things and much was dependent on relevance, timing and staff who had the time/ability to tap into the broader community. 3.3.3 From a site perspective, what has central funding helped them achieve that mission funding has not? Surveys, site visits and interviews identified a number of roles for central funding: • To fund global priorities and longer-term strategies; • To supplement to mission funds for the same activities; • To complement mission funds by providing money for related activities, or to provide funds for activities that are not mission priorities but may be global priorities (for example, the Cerrado-Pantanal ecosystem in Brazil); • To fund transboundary projects (for example, the AWF Kilimanjaro Heartland in Tanzania and Kenya); • To provide funding for global priorities in USAID non-presence countries; • To facilitate inter-institutional, global learning and sharing of state-of-the-art approaches, models, and practices; and • To support NGO partners in their institutional implementation of new approaches (e.g., landscape-level conservation) through dedicated programs and staff based at the headquarters-level of these international organizations. We found on our site visits that mission views about the role and value of centrally funded programs are diverse. That is, some missions saw a clear value for centrally funded programs for one or more of the reasons just listed, while other missions felt that they could have programmed the money now going to central programs more effectively themselves. It was clear from site visits and interviews that communication about, and coordination of, centrally funded activities with missions could be improved in some cases. In the course of the evaluation we had the opportunity to visit only four country missions (and in one case appropriate staff were not available for an interview), so we are not able to state the degree to which our observations are reflective of all countries where GCP funding is allocated. From a perspective broader than that of the site alone, many sites benefited from central funding support to their institutions as a whole. GCP supported NGO partners in their development of global methodologies (e.g., landscape-scale conservation approaches) through dedicated programs and staff based at the headquarters-level of these international organizations. One clear example of this is the WCS Living Landscapes Program (LLP), which was created through GCP funding, and supports four GCP sites as well as eight other WCS sites that utilize the same landscape approach to conservation. 3.3.4 How has USAID GCP performed administratively as a donor? We asked the question “Compared to other donors, how has USAID performed in its administration of GCP?” on both the site-level and partner representatives’ surveys. Because most of the administrative interaction with USAID occurred at the headquarters level of GCP partners, those results are probably most relevant (Figure 3.9). It appears that USAID added value as administrators most directly through technical assistance by knowledgeable and effective staff. While the reporting requirements for USAID were deemed more cumbersome than most, we actually heard some positive comments about them: “The process of work planning and reporting had a positive side for [our organization]—we don’t have thorough internal processes GCP EVALUATION FINAL REPORT 25 for overseeing projects. GCP annual reporting helped to enforce that discipline.” GCP partners also highly appreciated the length of time that USAID was willing to support their projects, viewing this as critical to establishing meaningful programs on the ground. Proposal proce ss Reporting requirements Prompt delivery of funding Technical assistance Knowledgeable and e ffective sta ff Term (# yrs) of funding Flexibility in use of funding Facilitated NGO networking Facilitated donor networking FIGURE 3.9. USAID’S ADMINISTRATIVE PERFORMANCE RELATIVE TO OTHER DONORS ge Donor Donor onor Percentage of GCP Partner Representatives USAID Administrative Performance (n=6) N/A Better than Avera Same As Average Worse than Average D 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Administrative Element From the site-level perspective (not shown in Figure 3.9—see Annex C, Question #37 for aggregated results), USAID was on average viewed as “better than the average donor” in terms of the proposal process, reporting requirements, prompt delivery of funding, and knowledgeable and effective staff. It is notable that at the field level, the two aspects of administration for which GCP partners rated USAID as “worse than the average donor” were rated as “better than the average donor.” This is probably due to the fact that GCP partner staff at the headquarters level took on the burden of proposal preparation and reporting, insulating the site-level staff from these burdens. 26 GCP EVALUATION FINAL REPORT 4.0 SUMMARY OF KEY FINDINGS The evaluation produced a wealth of information about the performance and effectiveness of the Global Conservation Program. Many of these have been highlighted in the previous section of this report. However, certain key results and findings merit special emphasis. 1. GCP has been effective in addressing several factors limiting conservation at GCP sites worldwide. In particular, we found clear evidence that GCP influenced the design of conservation programs at GCP sites, in terms of both scale and approach. GCP has facilitated and supported the development of threats-based conservation design and planning at the landscape and seascape scales. It has also played a catalytic role in the development of adaptive management standards within GCP partner NGOs, although much work remains. In addition, GCP enabled its partners to make significant progress in addressing two other important factors that can limit conservation at sites: the lack of stakeholder engagement, and institutional capacity. Our results show that before GCP, these factors prevented, or were significant barriers to, conservation at most GCP sites, but were addressed sufficiently through GCP so that they shifted to become, in general, a manageable problem or not a problem. These are clearly positive results, which show that the program achieved a number of its objectives. 2. Other factors limiting conservation at GCP sites remain problems. Government policy and legislation, economic context, illegal activities, and financial sustainability still “prevent conservation” or are a “serious barrier” at many sites. Although they remain problems, the data show that for each of these four factors, progress has been made during the period of GCP support. Financial sustainability remains the biggest unaddressed barrier to conservation, remaining a problem at approximately 67 percent of the sites, even though GCP’s intention was to support partners in developing this. In no cases are GCP sites fully secured for conservation. Some CTOs and NGO partners challenged the importance of reporting on limiting factors in the evaluation that were “outside the control” of conservation projects, and consequently showed little improvement. Such a position suggests a view of conservation design that is more limited than that of a landscape-scale, threats-based approach, an approach in which conservation planners are challenged to move all factors that affect the conservation target, both directly and indirectly, into their sphere of influence. 3. Adaptive management systems need further definition and development. The use of counterfactuals is not common (such as control sites or projections of change in the absence of a specific intervention), making it difficult to discern what changes can be attributed to the projects funded by GCP. Monitoring, analysis, and feedback mechanisms for management need improvement. Lack of documentation of successes at the site level limits ability to learn and share lessons in some cases. 4. GCP enabled major internal learning and program development within some GCP partner institutions. We found evidence that it facilitated the adoption of new approaches for threats-based conservation planning at large spatial scales, and progress toward a more standardized approach to adaptive management of projects and programs. 5. GCP enabled, facilitated, or catalyzed significant, direct communication, collaboration, and learning between NGO partners. Informal, natural collaborations among GCP partner organizations that were mutually beneficial developed in the enabling environment supported by GCP, facilitating and catalyzing some of the positive results we observed. 6. An effective formal structure to promote cross-institutional learning did not develop, despite significant efforts. Formal learning activities that began in GCP II were not seen as a major influence on the learning or sharing of success stories and best practices among partners and sites. Nevertheless, GCP partners did GCP EVALUATION FINAL REPORT 27 effectively use mechanisms for sharing information, such as publications for the broader conservation community. 7. Centrally funded programs like GCP can contribute to USAID’s global objectives, complement mission programs, and add value by: • Supporting global priorities, longer-term strategies, and transboundary projects; • Supplementing limited mission support; • Funding activities in non-presence countries; • Facilitating interinstitutional, global learning, and developing and sharing of state-of-the-art approaches, models, and practices; and • Supporting NGO partners in their institutional implementation of new approaches (e.g., landscape￾level conservation) through dedicated programs and staff based at the headquarters level of these international organizations. 28 GCP EVALUATION FINAL REPORT 5.0 IMPLICATIONS FOR FUTURE PROGRAMS The RFTOP for this evaluation (Annex A) stated that “the evaluation should analyze and provide a discussion of... key recommendations to USAID and partners on comparative advantages and disadvantages of the GCP model.” It also stated, however, that “USAID does not anticipate a direct follow-on ‘GCP III’ activity.” Rather than making recommendations, we will discuss the implications of our key findings for the design of programs with objectives related to those of GCP, including landscape-scale, threats-based conservation; generation of best practices, shared knowledge, and learning; and financial and institutional sustainability of conservation programs. These implications should be of interest to conservation NGOs and donors supporting them. Our evaluation results imply that: • Future programs could expect to achieve similar positive results by maintaining certain elements of GCP structure; and • Future programs could be designed with elements that address certain weaknesses identified in GCP model. Future programs could maintain positive elements of GCP, including: • Logical, threats-based conservation design at the landscape/seascape scale; • An enabling environment that provides support intended to facilitate and catalyze global (cross-site and cross-ecoregional) learning and development of best practices within conservation NGOs; • An enabling environment that has the intention and expectation of cross-institutional sharing of best practices and learning; • An enabling environment that allows for—and supports—natural, informal, voluntary collaboration between NGO implementing partners; • Long-term and flexible funding; • A centrally funded mechanism that can contribute to USAID’s global objectives, complement USAID mission funding, and add value through global activities; and • Continued attention to stakeholder engagement and building the capacity of local institutions in order to maintain site-level successes in addressing these limiting factors to conservation. Future programs could address weaknesses of GCP, including: • Incomplete development and adoption of adaptive management systems; • Lack of a program-wide communications strategy and system that would allow site-level managers to understand and better contribute to the global objectives of the program; • Lack of indicators that track progress in abating threats to biodiversity and addressing limiting factors to conservation; • Poor documentation of successes and failures at the site level as part of an adaptive management system and to enable cross-site learning; • The need to emphasize learning and development of best practices related to the limiting factors that remain the most serious barriers to site-level conservation: government policy and legislation, economic context, illegal activities, and financial sustainability, with financial sustainability at the top of the list; • The need to develop an effective structure to promote formal learning, including clear leadership/responsibility, and realistic incentives for participation, if that is an objective; and GCP EVALUATION FINAL REPORT 29 • The need to simplify and streamline reporting requirements. We believe there are three general options, or models, for establishing processes for interinstitutional knowledge-sharing and learning: 1. Expect voluntary, informal, beneficial collaboration and learning to develop organically in an “enabling environment” that provides the conditions and support for interinstitutional communication. (This happened in GCP, and is very likely to happen again in a similar enabling environment as GCP.) 2. Support a “Learning Panel” model, as tried in GCP II, in which representatives from the NGO conservation partners engage in a collaborative decision-making process to select learning topics of mutual interest. Stronger direction/leadership from USAID or designated leader, and larger financial incentives for participation than present in GCP II, will be needed to develop this into an effective model, in our view. (This was tried in GCP, but did not work well for a number of reasons; however, learning from that experience, a stronger version of this model probably could be developed.) 3. Create a central learning secretariat function in one responsible institution, empowered through either a contract or cooperative agreement mechanism, with performance targets and an adaptive management system for generating interinstitutional learning results. (This model was not really tried in GCP. Some possible models can be found with USAID, such as the contracted coordinating “secretariat” of the Initiative for Conservation in the Andean Amazon [ICAA], and the learning component of the work funded by the Microenterprise Development Office, including MicroLinks.) In our view, all models are dependent on the several key factors that will determine their success: a) a willingness and genuine interest among parties to share knowledge; b) realistic financial incentives to cover the costs (staff time, travel, communications) of sharing knowledge; c) cooperative and constructive individuals representing the organizations involved; d) leadership of the knowledge-sharing process; and e) actual knowledge to share, generated through rigorous processes that have technical validity. In GCP II, we saw some evidence of weakness in each of these factors. These factors are important considerations in implementing any of the above models. Although USAID does not anticipate a direct follow-on to GCP, we assume that donors will continue to support conservation NGOs such as GCP partners in their work to conserve the Earth’s biological diversity so that it can continue to provide humans with multiple benefits to sustain our development. We hope that the findings of this evaluation will contribute in a small way to fostering the evolution of effective conservation programs. 30 GCP EVALUATION FINAL REPORT ANNEXES GCP EVALUATION FINAL REPORT 31 ANNEX A. SCOPE OF WORK . Statement of Work for Evaluation of the Global Conservation Program USAID/EGAT/NRM/Biodiversity & Forestry Team Introduction USAID/EGAT is soliciting proposals for the “Global Conservation Program (GCP) Evaluation” Task Order. The evaluation will analyze USAID/EGAT investments in GCP from 1999 to the present to determine return on investment in terms of the program impact and the development and adoption of best practices in biodiversity conservation. USAID does not anticipate a direct follow-on ‘GCP III’ activity. Development challenge For many years USAID, other donors and partner organizations have been interested in optimizing approaches for effective biodiversity conservation. Conservation is a global benefit that requires long-term investment to achieve sustained and measurable results. Yet donors including USAID require accountability and measurable results in the shorter term. To achieve results, conservation best practices should be disseminated widely for broader adoption and practice. However, implementers often do not have the time or mandate to share knowledge across institutions. Funding typically goes to one institution with the hope that knowledge generated will be shared. The Global Conservation Program (GCP) was designed to achieve conservation results in partnership with non-governmental organizations (NGOs), and to promote best practices, partnerships and build communities of practice. It is a long-term centrally funded and managed program that complements investments of bilateral missions and partner organizations. GCP’s central mandate focuses on achieving landscape level conservation results in a representative selection of the world’s most biodiverse areas. The program also focuses on the sharing of lessons learned and conservation approaches between sites and amongst partners. Specific learning activities across institutions were instituted in the second phase of GCP. USAID is interested in understanding the impact of centrally funded programs such as GCP, particularly as there is an emphasis to increasingly shift towards bilateral programming. In particular, this evaluation will explore questions regarding how national level results can be measured and how programs such as GCP fit within an integrated country strategy. From the partners’ point of view, there is a need to understand the opportunity costs of cross-institutional learning with respect to investment in site level work that typically seems more pressing. In short, did GCP achieve an “added value” that justifies longer term and centrally managed funding as well as the effort that went into the learning component? If so, how and where did this value manifest? Where there were weaknesses, did these arise from the principles of GCP or externalities of implementation? GCP Background GCP is a partnership between USAID and six leading US-based non-governmental organizations (NGOs) that aims to conserve globally significant areas of biodiversity. Partner organizations implement site-based programs around the world. These programs work at varying scales, from the community level to large landscape and GCP EVALUATION FINAL REPORT 33 seascape scales. GCP was funded at a level of $4.2 million dollars in FY2005 with a life of program funding level of $72 Million. GCP is USAID’s only global conservation initiative, complementing a wide array of Agency-funded biodiversity activities around the world. Management is based in USAID/Washington with a central manager and Cognizant Technical Officers (CTOs) for different partners. GCP employs competitively awarded cooperative agreements with the six NGOs under USAID’s Leader with Associates (LWA) award mechanism. To date over 50 Associate Awards for over 130 million USD have been awarded using USAID Mission funds. In addition to site specific conservation, GCP has supported learning activities for many years; these have evolved and now involve all partners. Learning themes include socioeconomic tools and methods for conservation, marine protected area learning network and landscape planning, among others. GCP has gone through two phases. Some sites have carried over from the first phase (1999-2004) while others have been closed or newly initiated in the second phase (2003-2008). A mid-term evaluation took place in 2002 during GCP’s first phase (by ARD, ICS through BIOFOR IQC). Findings relevant to the current evaluation include: • Partners have applied “threats-based” approaches in various ways, and these approaches have evolved over the course of the Program. USAID and Partners should continue discussions on the “threats-based approach” to improve their understanding of these approaches, lessons learned, and to clarify USAID’s implicit assumptions. • USAID/GCP and partners should further explore support of various mechanisms for learning opportunities for internal learning for all partners and for shared learning among partners and other members of the conservation community. • USAID/GCP and partners should all explore opportunities for improved coordination, cooperation, and collaboration as these arise. Some partners find reporting requirements arduous. USAID/GCP should work with partners to improve the work plan process and streamline semi-annual reporting. During the second phase, USAID and partners decided that it was particularly important to evaluate the effectiveness of GCP as a mechanism to fund conservation, to generate and disseminate learning among partners and sites, and to foster best practices. An evaluation of the whole program in all its elements is not possible, however, given the current complexity of the program, its scope, and the expense of such an endeavor. This evaluation will draw on evaluations and audits of specific activities and programs within GCP. Secondary purposes of the evaluation include gaining a better understanding of the use of key conservation concepts such as adaptive management and landscape; information on site-level impacts at sites visited and studied; and insight into opportunities for and challenges to effective collaboration among large international NGOs. Scope of Work This evaluation will provide an assessment of return on USAID investment in a centrally managed program that convenes partners and generates knowledge outputs. It will assess the development, dissemination and adoption of best practices in biodiversity conservation, knowledge products produced and disseminated, and other key results. It will focus on core funding rather than the entire scope of GCP’s Associate Awards. While the evaluation will include site visits, their purpose is not attempt to capture the full impact and results at the sites but to document the impact of being part of a larger program in terms of cross-site and cross-institutional learning. USAID anticipates a participatory methodology will be employed for this evaluation. Evaluation objectives 1. To document the added value of the centrally-managed, multi-institutional GCP program. 2. To assess development and adoption of best practices within biodiversity conservation promoted by GCP, including cross-partner, cross-site aspects of such learning. 34 GCP EVALUATION FINAL REPORT 3. To document selected site level conservation results. With these overarching objectives in mind, the evaluation should analyze and provide a discussion of: • Selected site level impacts due to GCP investment focusing especially on GCP additionality and spillover effects (at sites, in country, and within organizations) • Selected and prioritized national, regional and global impacts catalyzed by GCP (within partner organizations, USAID, global fora and national fora) • The adoption of key concepts, particularly landscape planning and conservation and adaptive management within GCP family of projects and as a function of GCP partnerships • Key lessons learned and how they have been made available to the broader conservation community • Key recommendations to USAID and partners on comparative advantages and disadvantages of GCP model Illustrative approaches to achieve these objectives might include, but are not limited to: • Desktop review and analysis of expected impacts against results, including indicators and benchmarks • Review of selected GCP products and learning materials • Survey instruments and interviews of key GCP staff, partners and beneficiaries • Site visits that involve partners using a participatory methodology The Contractor will supply short-term technical assistance for evaluation of USAID’s centrally managed Global Conservation Program. Two consultants will perform these services with backstopping from the successful offeror. The USAID GCP CTO will provide overarching guidance and support, with review and acceptance of intermediate and final deliverables. Deliverable Schedule and Payment Schedule 1. Within one week of the TO being awarded the Consortium leader and Key Personnel with meet with USAID/EGAT/NRM CTO, Activity Manager and other Biodiversity staff to discuss the TO and agree on expectations and site visit criteria and deliverable formats. 2. Within 15 working days of Award, the TO Key Personnel will deliver a detailed proposed methodology and timeline for the evaluation. Preliminary discussions with GCP partners regarding potential site visits will be required. Key personnel, CTO and Activity Manager will meet to discuss draft and finalize. The final methodology should be submitted within 15 working days of receipt of comments on the draft. • Deliverable #1: Draft methodology. • Deliverable #2: Final methodology and timeline, site visit itineraries. 3. Upon acceptance of methodology, contractor and USAID will hold a meeting with GCP partners to explain the evaluation process, answer questions, and develop detailed plans for site visits. Note: The final choice of site visits will be made in consultation with USAID/Washington, Missions and partners. Illustrative sites for the Task Order proposal should reflect an appropriate balance of activities, partners and the overall technical approach, as well as cost-effectiveness. 4. Implement agreed upon methodology for evaluation, including appropriate document reviews, interviews, two to three site visits, and associated data analysis. Site visits will include some USAID participation. 5. Prepare draft report for review by CTO, USAID staff, and GCP partners. It is expected that USAID will review the draft focusing on sensitivities and overall focus and not comment on specific technical findings. It is expected that 15 working days will be allotted for review. GCP EVALUATION FINAL REPORT 35 • Deliverable #3: Draft report of not more than 25 pages, with additional associated appendices and supporting materials. 6. Incorporate necessary edits and prepare final report and associated presentation materials. 7. Deliver two presentations of findings, one for an internal USAID audience and one to a broader body of GCP and conservation partners. • Deliverable #5: Presentation of findings in PowerPoint format to be delivered in two meetings. • Deliverable #4: Final report of not more than 25 pages, with additional associated appendices and supporting materials; including Executive Summary of not more than 5 pages; Microsoft Word; 11 or 12 point font submitted in 2 paper and electronic copies not more than one week after comments are due. Report must meet all legal USAID formatting requirements. Note: presentation of findings must take place before May 31, 2008; final report must be submitted before May 31, 2008. Payment Schedule 1. Upon submission and acceptance of Draft #2: Final methodology and timeline, site visit itineraries - 15% of total contract price. 2. Upon submission and acceptance by USAID of Deliverable #3 – Draft Report – 65% of total contract price. 3. Upon submission and acceptance by USAID of Deliverable #4 – Final Report – 20% of total contract price. Key Personnel The Key Personnel for this Task Order consist of a Lead Consultant/conservation biologist and Evaluation Specialist as described below: Lead consultant/conservation biologist The lead consultant will have an advanced degree in conservation biology, wildlife biology, conservation management or related specialization with no less than seven years experience in international conservation. The person should have solid team leadership and evaluation experience. Other key characteristics will include excellent communication skills (oral and written), analytic skills, and people skills to gain trust of all participants in the evaluation process. The person should be conversant with landscape scale conservation approaches, adaptive management processes and conservation knowledge management. Familiarity with USAID programs is strongly preferred. Evaluation specialist The evaluation specialist will have not less than seven years of evaluation experience, with an emphasis on the natural resource management sector, and an advanced degree in a social/economic science. This experience should include evaluation of USAID programs (not just activities). Other key characteristics will include excellent communication skills (oral and written), analytic skills, and people skills to gain trust of all participants in the evaluation process. Technical Evaluation Criteria 1. Quality of technical approach: 50 Points 2. Quality and relevance of Key Personnel: 40 points 36 GCP EVALUATION FINAL REPORT 3. Past Performance: 10 points Proposal Format The Response to this RFTOP is limited to: 1. Technical Proposal (limit 8 pages maximum) 2. Cost Proposal (limit 2 pages maximum) 3. Resumes and Biographical Data Sheets for Key Personnel with 3 references (maximum 4 pages each) 4. Consortia Past Performance in evaluation of Biodiversity Programs (Maximum 2 pages) Cost Proposal Instructions: 1. Use the following geographic areas for the site visits: 1 Central American, 1 African and 1 Asian trip to cover 5 GCP sites. 2. Include biographical data sheets for the key personnel. Technical Instructions: 1. Quality of Technical Approach: The offeror must demonstrate an overall understanding of landscape-scale conservation efforts and evaluation methodologies for large-scale and complex conservation initiatives. Other quality criteria include innovation of the evaluation approach, ability to analyze complex variables within a limited budget and relevance of the evaluation approach to USAID and partners’ programs. The Offeror’s must also include their approach for analysis and evaluation of GCP and the illustrative methodology and timeline including site visits to 3 GCP sites. 2. Quality and Relevance of Key Personnel: The strength and relevance of the key personnel’s professional qualifications, expertise and experience relative to this SOW. Quality of proposed technical staff that maximizes use of partner expertise and participation is another criterion. The offeror must submit resume, biographical data sheets, and list of 3 references for each individual. 3. Past Performance: This criterion will be measured by demonstrated knowledge of landscape-scale conservation efforts and methods to evaluate them. USAID will also evaluate the past performance in terms of: 1) the relevance of past performance projects submitted to the RFP SOW; and 2) reference checks that assess the offeror’s demonstrated timeliness of performance, cost control, product/service quality, customer satisfaction, and effectiveness of key personnel. The offeror must include a minimum of three (3) past performance examples with accompanying references for the past five (5) years for current public or private sector type awards for efforts similar to this requirement (i.e., examples must be where the contractor provided a significant contribution to the overall objective). Identify the program activities as it relates to scope of work. The reference information shall include the location, current telephone number, e-mail addresses, point of contact, award number, dollar value, and brief description of work performed. GCP EVALUATION FINAL REPORT 37 ANNEX B.EVALUATION FRAMEWORK Final Evaluation Framework for the Evaluation of the Global Conservation Program (GCP) Contract No. EPP-I-00-06-00008-00, Task Order No. 01 23 August, 2007 GCP Evaluation Framework Question Data Collection & Analysis Document Site-Level Conservation Results What have been the challenges faced at the site level and on which have site managers made the most progress? Data Collection: • Site-level survey • GCP-Rep survey & interviews • Site visits • GCP PMP indicators and other partner reports to USAID Analysis: • Compare rating of challenges at outset of GCP funding with today (or end of GCP funding). • Themes that should be addressed include: − Project design − Spatial scale − Management system − Government policy & legislation − Institutional capacity − Economic factors − Financial sustainability − Other: open for site managers to offer other themes • Document, possibly as short case studies, noteworthy successes and challenges at site level. • Relate results of above analysis to GCP PMP-level indicators as reported by GCP partners What are the landscape-scale conservation planning approaches used by GCP partners? What influence has GCP had in their development? Data Collection: • Site-level survey • GCP-Rep survey & interviews • Site visits • CTO interviews • GCP partner documentation of approach Analysis: • Catalogue definition and key parameters of spatial scale planning used by each GCP partner • Compare spatial scale planning of GCP partners with USAID’s “threat￾based” approach • Document development of GCP partner spatial scale planning, and degree of GCP influence 38 GCP EVALUATION FINAL REPORT GCP Evaluation Framework Question Data Collection & Analysis What are the adaptive Data Collection: management approaches used by • Site-level survey GCP partners? • GCP-Rep survey & interviews • Site visits What influence has GCP had in • CTO interviews their development? • GCP partner documentation of approach Analysis: • Catalogue definition and technical elements of adaptive management used by each GCP partner • Compare adaptive management of GCP partners with USAID’s definition of adaptive management • Document development of GCP partner adaptive management, and degree of GCP influence What best practices developed? Data Collection: • Site-level survey • GCP-Rep survey & interviews • Site visits • CTO interviews • GCP partner documentation of approach Analysis: • Identify best practices that have helped overcome challenges at site level (see above analysis) • Document, possibly as short case studies, noteworthy examples at site level What suite of partners was Data Collection: involved at sites? How does this • Site-level survey correlate with ability to overcome • Site visits challenges? How does this makeup compare across GCP sites? Analysis: • Document partnerships in addressing each of the categories of challenges (see above analysis) • Categories of partnerships include: − Local NGOs − International NGOs − Local Government − National Government − Local Communities − Private Individuals − Businesses − Universities − Others • Compare partnership makeup across sites • Compare success in addressing limiting factors across sites with patterns of partnerships • Document, possibly as short case studies, noteworthy examples at site level How has GCP funding affected Data Collection: site-level conservation, • Site-level survey considering consistency, flexibility • GCP-Rep survey & interviews and relative contribution to • Site visits overall site funding? GCP EVALUATION FINAL REPORT 39 GCP Evaluation Framework Question Data Collection & Analysis Analysis: • Compare progress of sites in overcoming challenges as a function of: − Overall funding consistency for core activities − Magnitude of overall funding − Duration of GCP funding − Percentage of funding from GCP • Document noteworthy open-ended survey responses re: importance of GCP funding Assess development and adoption of best practices within biodiversity conservation promoted by GCP, including cross-partner, cross-site aspects of such learning How did grantee formulate best practices to address limiting factors at site Data Collection: • Site-level survey • GCP-Rep survey & interviews • Site visits Analysis: • Document source of best practices, organized by categories of conservation challenges (see above analysis) • Sources may include: − Grantee’s own experience − GCP partner NGOs via direct communication − NGOs that are not GCP partners via direct communication − GCP partners via formal GCP “learning activities” − GCP partners via GCP meetings − Guidance from USAID’s GCP staff − Conservation community via publications or presentations − Other (open ended) • Document media by which knowledge was received in each instance • Compare sources of best practices with catalogue of “learning activities” • Compare site level challenges (see above analysis) with degree of GCP knowledge transfer for each challenge How did grantee share learning on best practices at site with others (cross GCP partner, cross site)? Data Collection: • Site-level survey • GCP-Rep survey & interviews • Site visits Analysis: • Document sharing of best practices, organized by categories of conservation challenges (see above analysis) • Sources may include − Grantee’s own experience − GCP partner NGOs via direct communication − NGOs that are not GCP partners via direct communication − GCP partners via formal GCP “learning activities” − GCP partners via GCP meetings − Guidance from USAID’s GCP staff − Conservation community via publications or presentations − Other (open ended) • Document media by which knowledge was shared in each instance • Compare sharing of knowledge with catalogue of formal “learning activities” 40 GCP EVALUATION FINAL REPORT GCP Evaluation Framework Question Data Collection & Analysis • Compare site level challenges (see above analysis) with degree of GCP knowledge transfer for each challenge Document the added value of the centrally managed, multi-institutional GCP program Did GCP successfully promote cross-institutional learning? If so, what were the most effective mechanisms? Data Collection: • Site-level survey • GCP-Rep survey & interviews • Site visits • CTO interviews Analysis: • See GCP partner knowledge transfer addressed above • Compare GCP-facilitated knowledge transfer to site-level challenges • Rank effectiveness ratings of site-level managers for learning media • Document, possibly as short case studies, noteworthy examples of knowledge transfer From a site perspective, what has central funding helped them achieve that mission funding has not? Data Collection: • Site-level survey • GCP-Rep survey & interviews • Site visits • CTO interviews Analysis: • Document distinguishing characteristics of GCP funding model from open￾ended interview questions Did sites benefit from multi- Data Collection: institutional collaboration at headquarters level brought about by GCP? • Site-level survey • GCP-Rep survey & interviews • Site visits • CTO interviews Analysis: • Document distinguishing characteristics of GCP partner collaboration from open-ended interview questions How has USAID GCP performed administratively as a donor? Data Collection: • Site-level survey • GCP-Rep survey & interviews Analysis: • Compile GCP partner ratings of USAID GCP relative to other donors in specific factors, including: − Proposal process − Reporting requirements − Prompt delivery of funding − Technical assistance − Knowledgeable and effective staff • Document GCP-partners’ examples of GCP’s administrative strengths • Document GCP-partners’ suggestions for improvement GCP EVALUATION FINAL REPORT 41 ANNEX C.AGGREGATED SITE￾LEVEL SURVEY RESULTS 42 GCP EVALUATION FINAL REPORT 1. Which area are you working to conserve? Please select only one as the focus for the survey -- if you work in more than one, please make sure your organization completes a separate survey for each. Response Response Percent Count Amboseli Heartland (AWF) Kilimanjaro Heartland (AWF) Laikipia-Samburu Heartland (AWF) Maasai Steppe Heartland (AWF) Manazambezi Heartland (AWF) Cerrado-Pantanal Biodiversity Corridor (CI) Menabe Biodiversity Corridor (CI) Guyana (CI) Kimbe Bay (TNC) Raja Ampat Islands (TNC) Meso-American Reef (TNC) Wakatobi National Park (TNC) Komodo National Park (TNC) 0.0% 3.8% 3.8% 3.8% 3.8% 3.8% 3.8% 3.8% 3.8% 3.8% 3.8% 3.8% 0.0% 0 1 1 1 1 1 1 1 1 1 1 1 0 Chaco-Pantanal Cross Border 0.0% 0 Initiatives (TNC) Glover's Reef Living Landscape (WCS) Madidi Living Landscape (WCS) Eastern Steppe Living Landscape (WCS) 3.8% 3.8% 3.8% 1 1 1 Maya Biosphere Reserve Living Landscape (WCS) 3.8% 1 Eastern Himalayas/Terai Arc Landscape (WWF) Forests of Lower Mekong (WWF) 3.8% 3.8% 1 1 GCP Site-Level Survey Page 1 East African Marine Ecoregion (WWF) Nepal-Himalaya (EW) Phillippines (EW) Other (please specify) 3.8% 3.8% 3.8% 23.1% answered question 1 1 1 6 26 skipped question 0 2. How long has your organization been working to conserve this area? Please state the number of years. Response Count 26 answered question 26 skipped question 0 3. What is your name (for our internal use only, in case we need to contact you)? Response Count 26 answered question 26 skipped question 0 4. What is your position in your organization? Response Count 26 answered question 26 skipped question 0 Page 2 5. How long have you worked in this area with the organization? Response Count 26 answered question 26 skipped question 0 6. How many years did(has) GCP support(ed) your conservation work in this area? Response Count 20 answered question 20 skipped question 6 7. For the period of GCP support, please tell us about the staffing, budgetary, and funding situation of your program. Staffing less than 10 11-30 31-50 more than 50 Respon Coun Year 1 90.9% (20) 4.5% (1) 4.5% (1) 0.0% (0) Year 2 81.8% (18) 18.2% (4) 0.0% (0) 0.0% (0) Year 3 68.2% (15) 31.8% (7) 0.0% (0) 0.0% (0) Year 4 57.1% (12) 33.3% (7) 9.5% (2) 0.0% (0) Year 5 56.3% (9) 37.5% (6) 6.3% (1) 0.0% (0) Year 6 60.0% (6) 30.0% (3) 0.0% (0) 10.0% (1) Year 7 57.1% (4) 28.6% (2) 0.0% (0) 14.3% (1) Year 8 25.0% (1) 50.0% (2) 0.0% (0) 25.0% (1) Annual Budget (all sources of funding) less than USD 250,000 USD 251,000- 500,000 USD 501,000­ 1,000,000 USD 1,000,001- 2,000,000 more than USD 2,000,000 Respon Coun Year 1 68.2% (15) 27.3% (6) 4.5% (1) 0.0% (0) 0.0% (0) Page 3 Year 2 50.0% (11) 45.5% (10) 4.5% (1) 0.0% (0) 0.0% (0) Year 3 31.8% (7) 59.1% (13) 9.1% (2) 0.0% (0) 0.0% (0) Year 4 28.6% (6) 52.4% (11) 14.3% (3) 4.8% (1) 0.0% (0) Year 5 43.8% (7) 37.5% (6) 12.5% (2) 6.3% (1) 0.0% (0) Year 6 40.0% (4) 20.0% (2) 20.0% (2) 20.0% (2) 0.0% (0) Year 7 28.6% (2) 14.3% (1) 42.9% (3) 14.3% (1) 0.0% (0) Year 8 50.0% (2) 0.0% (0) 25.0% (1) 25.0% (1) 0.0% (0) GCP support / total funding less than 25% 26-50% 51-75% more than 75% Respon Coun Year 1 31.8% (7) 31.8% (7) 18.2% (4) 18.2% (4) Year 2 18.2% (4) 40.9% (9) 27.3% (6) 13.6% (3) Year 3 31.8% (7) 27.3% (6) 22.7% (5) 18.2% (4) Year 4 38.1% (8) 19.0% (4) 23.8% (5) 19.0% (4) Year 5 37.5% (6) 31.3% (5) 0.0% (0) 31.3% (5) Year 6 60.0% (6) 20.0% (2) 0.0% (0) 20.0% (2) Year 7 83.3% (5) 0.0% (0) 16.7% (1) 0.0% (0) Year 8 50.0% (2) 25.0% (1) 0.0% (0) 25.0% (1) answered question skipped question Page 4 8. Which other funders also supported your program? Response Percent Response Count USAID mission in your country 34.8% 8 Other US Government agency 34.8% 8 Government of country where you work 8.7% 2 Other foreign government 39.1% 9 Multi-lateral development bank(s) 21.7% 5 Private foundations or individuals 100.0% 23 Other (please specify) 34.8% 8 answered question 23 skipped question 3 9. What best describes your funding situation, considering all sources of funding (during period of GCP funding)? Response Percent Response Count Consistent and sufficient funding to perform necessary conservation work 16.7% 4 Consistent but insufficient funding to perform necessary conservation work 79.2% 19 Inconsistent but sufficient funding to perform necessary conservation work 0.0% 0 Inconsistent and insufficient funding to perform necessary conservation work 4.2% 1 answered question 24 skipped question 2 Page 5 10. How important was GCP funding for maintaining consistent core activities (i.e. the essential activities of your conservation program) in the landscape/seascape (during period of GCP funding)? Response Percent Response Count Not important 0.0% 0 17.4% 4 82.6% 19 answered question 23 skipped question 3 Important Very Important 11. What is the long-term financial sustainability of the conservation program? Response Percent Response Count None, without GCP the program would end (or ended, if GCP I only) 0.0% 0 Uncertain, may or may not find new sources of funding once GCP ends (or since it has ended, GCP I only) 34.8% 8 Partial, some activities have secure funding (e.g. endowment or other source of long-term funding) 65.2% 15 Complete, all activities have secure funding (e.g. endowment or other source of long-term funding) 0.0% 0 answered question 23 skipped question 3 Page 6 12. USAID's GCP emphasizes a "threats-based approach" in conservation design. Please tell us if you use this concept within your organization. Response Response Yes No Percent 95.8% 4.2% answered question Count 23 1 24 skipped question 2 13. If so, how do you define it? If not, is there another way you describe the design for your conservation program? Response Count 24 answered question 24 skipped question 2 14. In the design of your conservation program did you use any of the elements below? Please check one box per row. Yes, formally analyzed and documented in written form Yes, informally analyzed and documented in written form Yes, but not No, we take documented in another written form approach Response Count Identification of direct/proximate threats to biodiversity (i.e. habitat loss, overexploitation/overharvesting of species, invasive species, etc). 79.2% (19) 16.7% (4) 4.2% (1) 0.0% (0) 24 Prioritization of direct threats to biodiversity to be addressed by conservation program 54.2% (13) 41.7% (10) 4.2% (1) 0.0% (0) 24 Identification of causes of threats (e.g. economic, policy, etc.) 56.5% (13) 39.1% (9) 4.3% (1) 0.0% (0) 23 Development of strategies and activities to address the causes of the threats 58.3% (14) 37.5% (9) 4.2% (1) 0.0% (0) 24 answered question 24 skipped question 2 Page 7 15. Did your participation in GCP affect the design of your conservation program? Response Response Percent Count No 4.2% 1 Yes, GCP had some influence 62.5% 15 Yes, GCP was the major influence 33.3% 8 answered question 24 skipped question 2 16. What is the actual area your program will conserve (in hectares)? Response Count 20 answered question 20 skipped question 6 17. How would you describe the focus of your organization's work in this area? Response Response Percent Count Focused effort in a single site within the larger area 8.7% 2 Focused efforts in multiple sites within the larger area 39.1% 9 Broad effort that encompasses the entire area 4.3% 1 A combination of broad effort encompassing the entire area with focused initiatives at one or more 43.5% 10 specific sites within it Other (please specify) 4.3% answered question 1 23 skipped question 3 Page 8 18. What are the criteria you used to determine the spatial scale of your conservation program? (In some cases, this may be closely related to your responses on the previous page.) Response Count 22 answered question 22 skipped question 4 19. Did your participation in GCP affect the spatial scale at which you worked? Response Response Percent Count No 26.1% 6 Yes, GCP had some influence 47.8% 11 Yes, GCP was the major influence 26.1% 6 answered question 23 skipped question 3 Page 9 20. Please rate the degree to which the management of your conservation program employs the following elements. We understand that many programs are only recently thinking about developing many of these elements. Please do not feel pressure to select ratings higher than those you believe most accurately describe the current situation. Well developed and fully used Developed and used, but needs improvement Early stages of development, only partially or not yet used Not used Rating Average Response Count Written Management Objectives 47.8% (11) 47.8% (11) 4.3% (1) 0.0% (0) 1.57 23 Performance Metrics/Indicators 30.4% (7) 47.8% (11) 17.4% (4) 4.3% (1) 1.96 23 Experimental and Control Sites 4.5% (1) 22.7% (5) 13.6% (3) 59.1% (13) 3.27 22 Baseline Data for Metrics/Indicators 17.4% (4) 47.8% (11) 30.4% (7) 4.3% (1) 2.22 23 Written Description of Management Activities Necessary to Achieve Objectives 47.8% (11) 26.1% (6) 21.7% (5) 4.3% (1) 1.83 23 Monitoring System for Metrics/Indicators 26.1% (6) 47.8% (11) 21.7% (5) 4.3% (1) 2.04 23 Regular Analysis of Monitoring Data 13.0% (3) 47.8% (11) 34.8% (8) 4.3% (1) 2.30 23 Feedback Mechanism to Adapt Management According to Performance Results 13.0% (3) 47.8% (11) 30.4% (7) 8.7% (2) 2.35 23 Documentation of Successes and Failures -- Based on Use of Above System Elements 18.2% (4) 36.4% (8) 36.4% (8) 9.1% (2) 2.36 22 answered question 23 skipped question 3 Page 10 21. To what degree did GCP stimulate the development and implementation of the "adaptive management" concept in your conservation program. Response Response Percent Count I am not familiar with the concept of "adaptive management" 4.3% 1 We do not use the adaptive 0.0% 0 management concept We already employed adaptive management and GCP had little influence on it 8.7% 2 We already were developing adapative management, and GCP helped us to develop it further 69.6% 16 GCP is responsible for our development of adaptive management 17.4% 4 answered question 23 skipped question 3 Page 11 22. BEFORE you received GCP funding, please rate the degree to which each of these factors were limiting the conservation of your landscape/seascape. We did not have this problem Manageable problem Serious barrier to conservation Prevented conservation Rating Average Response Count Design: insufficient understanding of the ecological needs of the conservation target(s), and/or inadequate understanding of threats 13.6% (3) 36.4% (8) 50.0% (11) 0.0% (0) 1.36 22 to the conservation target(s) and how to overcome them, including necessary spatial scale Management System: conservation objectives not clearly identified, indicators and/or monitoring of indicators was inadequate, and 13.6% (3) 54.5% (12) 27.3% (6) 4.5% (1) 1.23 22 processes to respond systematically to needs was lacking Stakeholder Engagement: key stakeholders such as local communities, government, or even other NGOs, were not engaged and 9.1% (2) 31.8% (7) 45.5% (10) 13.6% (3) 1.64 22 opposed or prevented conservation activities Gov't Policy & Legislation: Government did not support conservation and acted in ways that were destructive to conservation target such as promoting extractive 13.6% (3) 27.3% (6) 50.0% (11) 9.1% (2) 1.55 22 industries in landscape/seascape, and/or there was no legal basis to protect the conservation targets Institutional Capacity: no management plan(s), insufficient trained conservation managers, or inadequate infrastructure and 0.0% (0) 27.3% (6) 54.5% (12) 18.2% (4) 1.91 22 equipment on part of government, NGOs, or other entities, to conserve the landscape/seascape Economic Context: economic activities in and around the landscape/seascape were not 0.0% (0) 27.3% (6) 45.5% (10) 27.3% (6) 2.00 22 compatible with conservation Illegal Activities: compliance with laws that protect the conservation targets in the landscape/seascape 0.0% (0) 22.7% (5) 59.1% (13) 18.2% (4) 1.95 22 Page 12 were not monitored, and/or violators were not prosecuted. Financial Sustainability: inadequate funding to support long-term conservation of the landscape/seascape 0.0% (0) 9.1% (2) 68.2% (15) 22.7% (5) 2.14 22 Other (please describe any other factors and tell us how much of a problem they presented for conservation) 3 answered question 22 skipped question 4 Page 13 23. TODAY (or when your GCP funding concluded), please rate the degree to which each of these factors are still limiting the conservation of your landscape/seascape. Not a problem Manageable problem Serious barrier to conservation Prevents conservation Rating Average Response Count Design: insufficient understanding of the ecological needs of conservation target(s), and/or inadequate understanding of the 47.8% (11) 47.8% (11) 4.3% (1) 0.0% (0) 0.57 23 threats to conservation target(s) and how to overcome them, including necessary spatial scale Management System: conservation objectives not clearly identified, indicators and/or monitoring of indicators is inadequate, and 47.8% (11) 47.8% (11) 4.3% (1) 0.0% (0) 0.57 23 processes to respond systematically to needs is lacking Stakeholder Engagement: key stakeholders such as local communities, government, or even other NGOs, are not engaged and 30.4% (7) 65.2% (15) 4.3% (1) 0.0% (0) 0.74 23 oppose or prevent conservation activities Gov't Policy & Legislation: Government does not support conservation and acts in ways that are destructive to the conservation target(s) such as promoting 21.7% (5) 39.1% (9) 34.8% (8) 4.3% (1) 1.22 23 extractive industries in the landscape/seascape, and/or there is no legal basis to protect conservation target(s) Institutional Capacity: no management plan(s), insufficient trained conservation managers, or inadequate infrastructure and equipment on the part of government, NGOs, or other entities, to conserve the landscape/seascape Economic Context: economic activities in and around the landscape/seascape are not compatible with conservation Illegal Activities: compliance with laws that protect the conservation targets in the landscape/seascape are not monitored, and/or violators are not prosecuted Financial Sustainability: inadequate funding to support long-term conservation of the landscape/seascape 13.0% (3) 78.3% (18) 8.7% (2) 0.0% (0) 0.96 4.3% (1) 60.9% (14) 34.8% (8) 0.0% (0) 1.30 0.0% (0) 60.9% (14) 30.4% (7) 8.7% (2) 1.48 0.0% (0) 39.1% (9) 56.5% (13) 4.3% (1) 1.65 Other (how did the "other" factors you listed in the previous question change?) answered question skipped question 23 23 23 23 4 23 3 24. Please describe your greatest success(es) in the conservation of this area. answered question skipped question Response Count 23 23 3 25. Any failures? Can you tell us about them? answered question skipped question Response Count 23 23 3 Page 14 26. Who or what has significantly influenced your current practices in addressing the themes below. Please check all options that a per row). Our own experience GCP partner NGOs via direct communication NGOs that are not GCP partners via direct communication GCP partners via formal GCP "learning activities" GCP partners via GCP meetings Guidance from USAID's GCP staff Co co pu pre Conservation Design: assessing ecological needs of conservation target(s), understanding of threats and how to manage them, including necessary spatial scale 87.0% (20) 56.5% (13) 30.4% (7) 21.7% (5) 30.4% (7) 34.8% (8) 6 Management System: developing objectives, indicators, monitoring, and process for responding to needs 87.0% (20) 39.1% (9) 34.8% (8) 17.4% (4) 17.4% (4) 34.8% (8) 5 Stakeholder Engagement: engaging key stakeholders and securing their support 100.0% (22) 27.3% (6) 54.5% (12) 18.2% (4) 22.7% (5) 22.7% (5) 5 Government Policy & Legislation: promoting formalized gov't support for conservation 90.5% (19) 19.0% (4) 47.6% (10) 14.3% (3) 19.0% (4) 28.6% (6) 3 Institutional Capacity: developing management plans, staffing, trained conservation managers, building and maintaining infrastructure, acquiring and maintaining equipment 90.9% (20) 36.4% (8) 54.5% (12) 22.7% (5) 22.7% (5) 22.7% (5) 4 Economic Context: finding a balance between economic needs of local communities and conservation 82.6% (19) 30.4% (7) 43.5% (10) 21.7% (5) 21.7% (5) 30.4% (7) 5 Compliance & Enforcement: ensuring compliance with laws that protect conservation targets 85.7% (18) 23.8% (5) 52.4% (11) 23.8% (5) 23.8% (5) 33.3% (7) 5 Financial Sustainability: funding both short- and long-term site-level conservation needs 76.2% (16) 33.3% (7) 47.6% (10) 14.3% (3) 28.6% (6) 33.3% (7) 5 Other (plea answere skippe Page 15 27. In those cases where you have learned "best practices" from sources other than your own experience, what have been the most effective media? Not Effective Moderately Effective Very Effective Most Effective N/A Rating Average Response Count Non-technical written documents 4.5% (1) 40.9% (9) 40.9% (9) 9.1% (2) 4.5% (1) 2.57 22 Technical documents and journal papers 4.5% (1) 36.4% (8) 40.9% (9) 18.2% (4) 0.0% (0) 2.73 22 Presentations at conferences 0.0% (0) 40.9% (9) 36.4% (8) 22.7% (5) 0.0% (0) 2.82 22 Visits to other sites 0.0% (0) 9.1% (2) 40.9% (9) 50.0% (11) 0.0% (0) 3.41 22 Direct on-the-ground collaboration with other NGOs 0.0% (0) 17.4% (4) 39.1% (9) 39.1% (9) 4.3% (1) 3.23 23 Direct communication with USAID 4.8% (1) 42.9% (9) 9.5% (2) 28.6% (6) 14.3% (3) 2.72 21 Other (please specify) 0 answered question 23 skipped question 3 Page 16 28. If you have documented your own "best practices," on which themes and with whom have you shared them? Please check all options that apply (multiple checks per row). Directly Conservation with With GCP Directly Directly With GCP community other partners with with GCP partners via Response NGOS via formal USAID's partner via GCP publications Count that are "learning GCP NGOs meetings or not GCP activities" staff presentations partners 66.7% 66.7% 57.1% Conservation Design 28.6% (6) 23.8% (5) 85.7% (18) 21 (14) (14) (12) 65.0% 60.0% 55.0% Spatial Scale of Conservation 35.0% (7) 25.0% (5) 85.0% (17) 20 (13) (12) (11) 44.4% 55.6% 38.9% Adaptive Management 22.2% (4) 22.2% (4) 66.7% (12) 18 (8) (10) (7) 43.8% 68.8% 50.0% Stakeholder Engagement 25.0% (4) 18.8% (3) 68.8% (11) 16 (7) (11) (8) 50.0% 75.0% 37.5% Government Policy & Legislation 18.8% (3) 31.3% (5) 68.8% (11) 16 (8) (12) (6) 57.1% 71.4% 57.1% Institutional Capacity 28.6% (4) 35.7% (5) 50.0% (7) 14 (8) (10) (8) 40.0% 40.0% 33.3% Economic Pressures 33.3% (5) 20.0% (3) 66.7% (10) 15 (6) (6) (5) 33.3% 55.6% 33.3% Compliance & Enforcement 16.7% (3) 16.7% (3) 72.2% (13) 18 (6) (10) (6) 38.5% 46.2% Financial Sustainability 61.5% (8) 23.1% (3) 30.8% (4) 53.8% (7) 13 (5) (6) Other (please specify) 1 answered question 21 skipped question 5 Page 17 29. In those cases where you have shared your "best practices," what media have you used? Non-technical written documents Technical documents and journal papers Presentations at conferences Visits to other sites Direct on-the-ground collaboration with other NGOs Direct communication with USAID Other (please specify) Response Percent 76.2% 66.7% 90.5% 61.9% 76.2% 57.1% 9.5% answered question Response Count 16 14 19 13 16 12 2 21 30. Did your organization adopt any new "best practices" that you use at the site level, based skipped question on interactions with other GCP 5 partner NGOs at the level of your U.S.-based headquarters? No Yes Response Response Percent Count 52.4% 11 47.6% 10 answered question 21 31. If so, tell us more about it. What practices? How did it happen? skipped question 5 Response Count 10 answered question 10 skipped question 16 Page 18 32. During the period of GCP funding did you engage partners, in a formal agreement, to assist your conservation program? For example, did you enter an agreement with a government agency to assist in the management of a protected area? If so, identify which types of institutions you partnered with. Response Response Percent Count Local NGOs 81.8% 18 Int'l NGOs 59.1% 13 Local Gov't 54.5% 12 National Gov't 72.7% 16 Local Communities 63.6% 14 Private Individuals 27.3% 6 Businesses 13.6% 3 Universities 54.5% 12 N/A 9.1% 2 Other (please specify) 1 answered question 22 skipped question 4 33. Please describe briefly your most important partnerships. Tell us the name of the entity and what they helped you to do. Response Count 20 answered question 20 skipped question 6 Page 19 34. Have you also received funding from a USAID mission? Response Response Percent Count Yes 39.1% 9 No 60.9% 14 answered question 23 skipped question 3 35. If you answered "yes" above, has GCP funding helped you to achieve conservation results that would not have been possible if ALL USAID funding came through the USAID mission in the country where you work? Please describe. Response Count 8 answered question 8 skipped question 18 36. If you answered "yes" above, did USAID mission funding help you to achieve conservation results that would not have been possible if ALL USAID funding came through GCP alone? Please describe. Response Count 9 answered question 9 skipped question 17 Page 20 37. Compared to other donors, how has USAID performed in its administration of GCP? Worse than Same as Better than average average average donor donor donor Proposal process 8.7% (2) 34.8% (8) 47.8% (11) Reporting requirements 8.7% (2) 30.4% (7) 56.5% (13) Prompt delivery of funding 8.7% (2) 34.8% (8) 52.2% (12) Technical assistance 0.0% (0) 47.8% (11) 39.1% (9) Knowledgable and effective staff 0.0% (0) 30.4% (7) 52.2% (12) Rating N/A Average 8.7% (2) 2.43 4.3% (1) 2.50 4.3% (1) 2.45 13.0% (3) 2.45 17.4% (4) 2.63 answered question skipped question Response Count 23 23 23 23 23 23 3 38. Do you have any suggestions for improving USAID's administration of GCP? answered question skipped question Response Count 15 15 11 39. What is USAID doing well? answered question skipped question Response Count 20 20 6 Page 21 40. Anything we forgot to ask that you'd like to tell us? Response Count 10 answered question 10 skipped question 16 Page 22 ANNEX D. AGGREGATED PARTNER REPRESENTATIVE SURVEY RESULTS GCP EVALUATION FINAL REPORT 63 GCP-Rep Survey 1. Which GCP Partner NGO do you represent? Response Response Percent Count AWF 16.7% 1 CI 16.7% 1 TNC 16.7% 1 WWF 16.7% 1 EW/VITA 16.7% 1 WCS 16.7% 1 Other (please specify) 0.0% 0 answered question 6 skipped question 0 2. What is your name (for our internal use only, in case we need to contact you)? answered question skipped question Response Count 6 6 0 3. What is your position in your organization? answered question skipped question Response Count 6 6 0 Page 1 4. How long have you represented your organization in GCP? Response Count 6 answered question skipped question 5. USAID's GCP emphasizes a "threats-based approach" to conservation design. Do you use this concept within your organization? Response Percent Yes 83.3% No 16.7% answered question skipped question 6. If so, how do you define it? If not, is there another way you describe the design for your conservation programs? 6 0 Response Count 5 1 6 0 Response Count 6 answered question skipped question 7. Did your participation in GCP affect the way your organization designs conservation programs? Response Percent No 33.3% Yes, GCP had some influence 16.7% Yes, GCP was the major influence 50.0% answered question skipped question 6 0 Response Count 2 1 3 6 0 Page 2 8. If you would like to elaborate on ways in which GCP's "threats-based approach" influenced your organization, please do so here. Response Count 4 answered question 4 skipped question 2 9. USAID's GCP uses the term "landscape" or "seascape" to describe the area where partner NGOs operate. Please tell us if you use this concept to determine the spatial scale of your organization's programs. Response Response Percent Count Yes 83.3% 5 No 16.7% 1 answered question 6 skipped question 0 10. If so, how you define it? If you don't use it, is there another way you describe the spatial scale at which you work?What are the criteria you use to determine the spatial scale at which you work? (In some cases, this may be closely related to your answers on the previous page.) Response Count 6 answered question skipped question 6 0 Page 3 11. Did your participation in USAID's GCP affect the spatial scale at which your organization operates? Response Response Percent Count No 16.7% 1 Yes, GCP had some influence 66.7% 4 Yes, GCP was the major influnce 16.7% 1 answered question 6 skipped question 0 12. If you would like to elaborate on ways in which GCP influenced the spatial scale of your organization's work, please do so here. Response Count 5 answered question 5 skipped question 1 Page 4 13. Please rate the degree to which your organization uses the following management system elements for site-level conservation. We understand that many programs are only recently thinking about developing many of these elements. Please do not feel pressure to select ratings higher than those you believe most accurately describe the current situation. Well developed and fully used Developed and used, but needs improvement Early stages of development, only partially or not yet used No plan to develop Rating Average Response Count Written Management Objectives 16.7% (1) 83.3% (5) 0.0% (0) 0.0% (0) 1.83 6 Performance Metrics/Indicators 33.3% (2) 50.0% (3) 16.7% (1) 0.0% (0) 1.83 6 Experimental and Control Sites 0.0% (0) 0.0% (0) 66.7% (4) 33.3% (2) 3.33 6 Baseline Data for Metrics/Indicators 16.7% (1) 83.3% (5) 0.0% (0) 0.0% (0) 1.83 6 Written Description of Management Activities Necessary to Achieve Objectives 0.0% (0) 100.0% (6) 0.0% (0) 0.0% (0) 2.00 6 Monitoring System for Metrics/Indicators 16.7% (1) 50.0% (3) 33.3% (2) 0.0% (0) 2.17 6 Regular Analysis of Monitoring Data 0.0% (0) 66.7% (4) 33.3% (2) 0.0% (0) 2.33 6 Feedback Mechanism to Adapt Management According to Performance Results 0.0% (0) 50.0% (3) 50.0% (3) 0.0% (0) 2.50 6 Documentation of Successes and Failures -- Based on Use of Above System Elements 0.0% (0) 33.3% (2) 66.7% (4) 0.0% (0) 2.67 6 answered question 6 skipped question 0 Page 5 14. To what degree did GCP stimulate the development and implementation of the "adaptive management" concept in your organization? Response Percent Response Count I am not familiar with the term "adaptive management" 0.0% 0 We do not employ an adaptive management system 0.0% 0 We already employed an adaptive management system and GCP had little influence on it 33.3% 2 We already were developing an adapative management system, and GCP helped us to develop it further 50.0% 3 GCP is responsible for our development of an adaptive management system 16.7% 1 answered question 6 skipped question 0 15. If GCP was a significant influence in the development of adaptive management within your organization, tell us more about how that occurred and its importance for your organization. Response Count 3 answered question 3 skipped question 3 Page 6 16. Who or what has significantly influenced your current practices in addressing the themes below. Our own experience GCP partner NGOs via direct communication NGOs that are not GCP partners via direct communication GCP partners via formal GCP "learning activities" GCP partners via GCP meetings Guidance from USAID's GCP staff Co co pu pre Conservation Design: assessing ecological needs of conservation target(s), understanding of threats and how to manage them, including necessary spatial scale 100.0% (6) 83.3% (5) 83.3% (5) 50.0% (3) 50.0% (3) 16.7% (1) 8 Management System: developing objectives, indicators, monitoring, and process for responding to needs 100.0% (6) 66.7% (4) 83.3% (5) 33.3% (2) 33.3% (2) 33.3% (2) 6 Stakeholder Engagement: engaging key stakeholders and securing their support 100.0% (6) 33.3% (2) 66.7% (4) 16.7% (1) 16.7% (1) 16.7% (1) 6 Government Policy & Legislation: promoting formalized gov't support for conservation 100.0% (6) 33.3% (2) 33.3% (2) 0.0% (0) 0.0% (0) 50.0% (3) 5 Institutional Capacity: developing management plans, staffing trained conservation managers, building and maintaining infrastructure, acquiring and maintaining equipment 100.0% (6) 83.3% (5) 66.7% (4) 16.7% (1) 16.7% (1) 0.0% (0) 5 Economic Context: finding a balance between economic needs of local communities and conservation 100.0% (6) 50.0% (3) 83.3% (5) 16.7% (1) 16.7% (1) 33.3% (2) 6 Compliance & Enforcement: ensuring compliance with laws that protect conservation targets 100.0% (5) 40.0% (2) 40.0% (2) 0.0% (0) 0.0% (0) 20.0% (1) 6 Financial Sustainability: funding both short- and long-term site-level conservation needs 100.0% (5) 20.0% (1) 40.0% (2) 0.0% (0) 0.0% (0) 60.0% (3) 6 Other (plea answere skippe Page 7 17. In those cases where you have learned "best practices" from sources other than your own experience, what have been the most effective media? Not Moderately Very Most Rating Response N/A Effective Effective Effective Effective Average Count Non-technical written documents 0.0% (0) 33.3% (2) 50.0% (3) 16.7% (1) 0.0% (0) 2.83 6 Technical documents and journal 16.7% (1) 16.7% (1) 50.0% (3) 16.7% (1) 0.0% (0) 2.67 6 papers Presentations at conferences 16.7% (1) 33.3% (2) 50.0% (3) 0.0% (0) 0.0% (0) 2.33 6 Visits to other sites 0.0% (0) 16.7% (1) 50.0% (3) 33.3% (2) 0.0% (0) 3.17 6 Direct on-the-ground collaboration 0.0% (0) 16.7% (1) 33.3% (2) 50.0% (3) 0.0% (0) 3.33 6 with other NGOs Direct communication with USAID 16.7% (1) 50.0% (3) 33.3% (2) 0.0% (0) 0.0% (0) 2.17 6 Other (please specify) 1 answered question 6 skipped question 0 18. If you have documented your own "best practices," on which themes and with whom have you shared them? Directly Conservation with With GCP Directly Directly With GCP community other partners with with GCP partners via NGOS via formal USAID's partner via GCP publications that are "learning GCP NGOs meetings or not GCP activities" staff presentations partners 83.3% 83.3% 66.7% Conservation Design 50.0% (3) 83.3% (5) 100.0% (6) (5) (5) (4) 50.0% 50.0% 50.0% Spatial Scale of Conservation 33.3% (2) 50.0% (3) 83.3% (5) (3) (3) (3) 60.0% 60.0% Adaptive Management 80.0% (4) 40.0% (2) 40.0% (2) 60.0% (3) (3) (3) 66.7% 100.0% 66.7% Stakeholder Engagement 0.0% (0) 66.7% (2) 66.7% (2) (2) (3) (2) 50.0% 75.0% Government Policy & Legislation 75.0% (3) 0.0% (0) 50.0% (2) 25.0% (1) (2) (3) 100.0% 50.0% Institutional Capacity 0.0% (0) 0.0% (0) 50.0% (1) 0.0% (0) (2) (1) Response Count 6 6 5 3 4 2 Page 8 Economic Context 40.0% (2) 80.0% (4) 60.0% (3) 80.0% (4) 60.0% 60.0% (3) (3) 5 Compliance & Enforcement 66.7% (2) 66.7% (2) 0.0% (0) 33.3% (1) 100.0% 33.3% (1) (3) 3 Financial Sustainability 100.0% (2) 100.0% (2) 0.0% (0) 50.0% (1) 50.0% 100.0% (2) (1) 2 Other (please specify) 0 answered question 6 19. In those cases where you have shared your "best practices," what media have you skipped question used? (We may ask you to submit 0 examples at a later date) Response Response Percent Count Non-technical written documents 66.7% 4 Technical documents and journal 83.3% 5 papers Presentations at conferences 100.0% 6 Visits to other sites 50.0% 3 Direct on-the-ground collaboration 50.0% 3 with other NGOs Direct communication with USAID 83.3% 5 Other (please specify) 16.7% 1 answered question 6 skipped question 0 Page 9 20. Through GCP, did you develop any partnerships or collaborations with other GCP partners? If so, with which partner organization(s)? Briefly tell us about the partnership(s). Response Count 6 answered question 6 skipped question 0 21. Compared to other donors, how has USAID performed in its administration of GCP? Worse than Same as Better than average average average donor donor donor Proposal process 50.0% (3) 33.3% (2) 16.7% (1) Reporting requirements 66.7% (4) 16.7% (1) 16.7% (1) Prompt delivery of funding 16.7% (1) 33.3% (2) 50.0% (3) Technical assistance 16.7% (1) 16.7% (1) 50.0% (3) Knowledgable and effective staff 0.0% (0) 0.0% (0) 100.0% (6) Term (# of years) of funding 0.0% (0) 0.0% (0) 100.0% (6) Flexibility in use of funding 33.3% (2) 50.0% (3) 16.7% (1) Facilitated networking with other 0.0% (0) 50.0% (3) 50.0% (3) NGOs Facilitated networking with other 0.0% (0) 83.3% (5) 0.0% (0) donors Rating N/A Average 0.0% (0) 1.67 0.0% (0) 1.50 0.0% (0) 2.33 16.7% (1) 2.40 0.0% (0) 3.00 0.0% (0) 3.00 0.0% (0) 1.83 0.0% (0) 2.50 16.7% (1) 2.00 answered question skipped question Response Count 6 6 6 6 6 6 6 6 6 6 0 Page 10 22. Do you have any suggestions for improving USAID's administration of GCP? Response Count 5 answered question 5 skipped question 1 23. What is USAID doing well? Response Count 6 answered question 6 skipped question 0 24. In an average year during your organization's participation in GCP, what percentage (roughly rounded to nearest 5%) of your organization's overall budget came from GCP? Response Count 6 answered question 6 skipped question 0 25. Did your organization's programs that received GCP support also receive support from USAID country missions? Response Response Percent Count Yes 66.7% 4 No 33.3% 2 answered question 6 skipped question 0 Page 11 26. If you answered "yes" above, has GCP funding allowed you to do things differently than if you received ALL USAID funding only from USAID missions? Please describe. Response Count 4 answered question 4 skipped question 2 27. If you answered "yes" above, has USAID mission funding allowed you to do things differently than if you received ALL USAID funding only from GCP? Please describe. Response Count 4 answered question 4 skipped question 2 28. Anything we forgot to ask that you'd like to tell us? answered question skipped question Response Count 2 2 4 Page 12 ANNEX E. PERSONS CONTACTED USAID • Diane Russell, EGAT/NRM/B Evaluation CTO • Cynthia Gill, EGAT/NRM/B Biodiversity Team Leader • Barbara Best, EGAT/NRM/B GCP CTO for TNC • Mary Rowen, EGAT/NRM/B GCP CTO for AWF and EWV • Doreen Robinson, EGAT/NRM/B GCP Manager; CTO for CI and WCS • Hannah Fairbank, EGAT/NRM/B GCP CTO for WWF • Jerry Bisson, LAC/ CTO at beginning of GCP I US-Based GCP Partner Representatives • Adam Henson, AWF • Monique Derfuss, CI • Ann Koontz, EWV • Scott Smith, TNC • Sarah Davidson, TNC • David Wilkie, WCS • Leticia Orti, WCS • Judy Oglethorpe, WWF • Drew Crandall, WWF • Sarah Christiansen, formerly WWF • Sylvia Marin, WWF Central America Program Director • Phillip Goeltenboth, WWF manager for EAME in Washington, DC • Shubash Lohani, WWF Senior Program Officer - Eastern Himalayas (pilot-tested site-level survey) Central America Site Visits • Janet Gibson, WCS Glover’s Reef Program Director • Archie Carr III, WCS Regional Coordinator, Mesoamerica and Caribbean Program • Roberto Pott, Marine Protected Areas Manager, Belize Audubon Society • Isaias Majil, Marine Protected Areas Coordinator, Belize Fisheries Department • Jim and Kendra Schofield, Owners and Operators, Off the Wall Dive Center & Resort, Long Caye, Glover’s Reef Atoll • Sergio Hoare, WCS Glover’s Reef Monitoring Specialist • Danny Wesby, WCS Glover’s Reef Fisheries Monitor • Hopkins Fishermen’s’ Association, Belize • Adrian Oviedo, Executive Director, Fundación Cayos Cochinos, La Ceiba, Honduras • Nestor Windevoxhel, TNC Meso-American Reef Program Director, Guatemala City, Guatemala • Alejandro Arrivillaga, Marine Conservation Specialist, TNC MAR Program • Glenda de Paiz, Consultant in Development and Rural Enterprises, USAID-Guatemala GCP EVALUATION FINAL REPORT 75 East Africa Site Visits • Philip Muruthi, Director of Conservation Science, AWF, Nairobi, Kenya • Paul Ntiati, Director, AWF Kilimanjaro Heartland, Namanga, Kenya • Seif Hamisi Mutinda, Heartland Ecologist, AWF Kilimanjaro Heartland, Namanga, Kenya • Daudi Sumba, Director, Capacity Building & Leadership Development, AWF, Nairobi, Kenya • Alfred Kikoti, Research Scientist, Kilimanjaro Elephant Research & Conservation Program, AWF Kilimanjaro Heartland, Arusha, Tanzania • James Kahurananga, Program Director, AWF Maasai Steppe Heartland, Arusha, Tanzania • Thadeus Binamungu, Senior Project Officer, AWF Maasai Steppe Heartland • Aaron Musiga, Manyara Ranch Manager, AWF Maasai Steppe Heartland • Pastor Magingi, Ecologist, AWF Maasai Steppe Heartland • Amani Ngusaru, WWF Eastern African Marine Ecoregion (EAME) Leader, Dar es Salaam, Tanzania • Sam Weru, WWF EAME Kiunga Program Director, Kiunga, Kenya • Charles Oluchina, Environment & Natural Resources Project Management Specialist, USAID/Kenya, Nairobi, Kenya • Walter Knausenberger, Senior Regional Environmental Officer, USAID/East Africa • Gilbert Kajuna, Environmental Officer, USAID/Tanzania, Dar es Salaam, Tanzania • Juniper Neill, Team Leader – Natural Resources Management & Economic Growth, USAID/Tanzania Brazil Site Visit • Ricardo Machado, CI Cerrado Program Director, Brasilia, Brazil • Mario Barroso, CI Cerrado Program Manager, Brasilia, Brazil • Sandro Menezes, CI Pantanal Program Manager, Campo Grande, Brazil • Paula Valéria, Oréades, Brasilia, Brazil • Ricardo Bini, Director, Associação para Preservação Meio Ambiente de Rio Negro – APREMARINE, Rio Negro, Brazil • Elisângela Arruda, Environmental Education Specialist, APREMARINE, Rio Negro, Brazil • Eric Stoner, General Development Officer – Environment, USAID/Brazil, Brasilia, Brazil 76 GCP EVALUATION FINAL REPORT ANNEX F. SITE VISIT ITINERARIES BELIZE, HONDURAS, GUATEMALA (OCTOBER 2007) 22 Oct., Mon. Travel to Belize City, Belize 23 Oct., Tues. Meetings with WCS Glover’s Reef staff and partners, Belize Audubon Society and Fisheries Department, in Belize City, fly to Dangriga, car to Hopkins, overnight in Hopkins and meeting with Hopkins Fishermen’s’ Association 24 Oct., Wed. To Glover’s Reef by boat, and return to Hopkins 25 Oct., Thurs. From Hopkins travel to La Ceiba, Honduras 26 Oct. Fri Meeting with TNC MAR Cayos Cochinos protected area program staff in La Ceiba, Honduras 29 Oct. Mon Meetings with TNC MAR Program Secretariat staff in Guatemala City, Guatemala; meeting with USAID staff at Guatemala Mission 30 Oct. Tues Travel from Guatemala City KENYA, TANZANIA (NOVEMBER-DECEMBER 2007) 25 Nov., Sun. Travel to Nairobi via Amsterdam 26 Nov., Mon. Arrive Nairobi; overnight in Nairobi 27 Nov., Tues. Meetings with AWF Staff 28 Nov. Wed. Fly to Kilimanjaro Airport from Nairobi, drive to West Kilimanjaro area of AWF Kilimanjaro Heartland with site-level director 29 Nov., Thurs. Drive to Arusha; meetings with AWF Maasai Steppe Staff 30 Nov., Fri. Visit Manyara Ranch area of Maasai Steppe Heartland, return to Arusha 3 Dec., Mon. Fly to Dar es Salaam; meet with USAID-Tanzania staff 4 Dec., Tue. Meet with WWF EAME coordinator and staff; fly to Nairobi 5 Dec. Wed. Meet WWF EAME Kiunga site director 6 Dec., Thurs. Meeting with USAID staff BRAZIL (FEBRUARY 2008) 11 Feb., Mon. Travel to Brasilia 12 Feb., Tues. Meetings with CI staff 13 Feb., Wed. Meeting with USAID-Brazil; travel to Campo Grande 14 Feb., Thurs. Drive to Rio Negro in Cerrado-Pantanal with site-level director; meetings with local NGO partner and field observations 15 Feb., Fri. Campo Grande; begin travel home via Sao Paolo GCP EVALUATION FINAL REPORT 77 ANNEX G. CATALOGUE OF GCP DOCUMENTATION Catalogue of GCP Documents for the Evaluation (compiled by USAID as of July 13, 2007) 1.0 Annual and Quarterly Meeting Notes 1.1 2000 “Action Recommendations” workshop notes 1.2 2003 Annual Meeting Notes 1.3 2005 Annual Meeting Notes 1.4 2006 Annual Meeting Notes 1.5 2007 Annual Meeting Notes 1.6 Quarterly meeting notes from 17 separate meetings between 1999 and 2007 1.7 GCP Evaluation Brainstorming notes from USAID – 3/7/2006 1.8 GCP Lessons Themes & Related Questions – USAID, June 2007 2.0 Cooperative Agreements 2.1 AWF, Sept. 1999 2.2 CI, Oct. 1999 2.3 EWV, June 2002 2.4 TNC, Oct. 1999 2.5 WCS, Sept. 1999 2.6 WWF, May 2002 2.7 Reporting Requirements, Aug. 2004 2.8 AWF modification, Maasai Steppe Landscape, August 2001 2.9 CI modification, Learning Component, August 2005 2.10 EWV modification, Learning Component, July 2005 2.11 WWF modification, Terai Arc Wildlife Corridors, June 2001 2.12 TNC modification, New Conservation Finance activities, July 2001 3.0 History of GCP 3.1 GCP I & II Site Lists 3.2 GCP I Site Descriptions, Nov. 2002 3.3 History of GCP I, Dec. 2002 3.4 RFA Principles, Oct. 2002 3.5 Description of Threats-based Approach, Nov. 2002 3.6 New GCP II Sites List, Jan. 2004 3.7 GCP II RFA/Proposal Format Guidelines, Oct. 2002 3.8 GCP Justification, July 2006 3.9 Learning Panel SOW, March 2003 3.10 Learning Fund Principles, March 2003 78 GCP EVALUATION FINAL REPORT 3.11 Learning Panel Activities, March 2004 3.12 Learning Panel – suggestions for modification, April 2005 3.13 Learning Activities Framework, June 2005 4.0 Learning Products from July 2007 Annual Meeting 4.1 GCP Learning Activities Brochure, June 2006 4.2 SOW for Communication Product on Lessons Learned about Enterprise Development and Livelihoods 4.3 SOW for Communication Product on Lessons Learned about “Entry Points”/Engaging Stakeholders 4.4 SOW for Communication Product on Lessons Learned about Landscape-Scale Conservation 5.0 PMPs and Guidance on M&E 5.1 PMP Guidelines FY 2000, Sept. 2000 5.2 PMP Guidelines FY 2004, Nov. 2004 5.3 PMP Guidelines FY 2006, Oct. 2006 5.4 PMP “Cheat Sheet,” Oct. 2005 5.5 GCP PMP Table, Oct. 2006 5.6 CI 2003 PMP Master, Sept. 2004 5.7 EWV 2003-2004 Philippines 5.8 TNC Final Indicators, Sept. 2004 5.9 WCS PMPs for 3 sites, Sept. 2004 5.10 WWF, Sept. 2004 6.0 Selected Annual Reports 6.1 AWF – 6 years of annual reports and related docs 6.2 CI – 6 years and final GCP I sites report 6.3 EWV – 7 years and related docs 6.4 TNC – 6 years and related docs 6.5 WCS – 6 years and related docs 6.6 WWF – 7 years and related docs 7.0 Selected Work Plans 7.1 AWF, 7 years 7.2 CI, 7 years 7.3 EWV, 7 years 7.4 TNC, 8 years 7.5 WCS, 7 years 7.6 WWF, 8 years 8.0 Summaries of Key Lessons and Publications Lists 8.1 Template/Table for GCP Lessons, April 2007 8.2 AWF summary of GCP lessons, May 2007 8.3 CI summary of GCP lessons, May 2007 8.4 EWV summary of GCP lessons, June 2007 GCP EVALUATION FINAL REPORT 79 8.5 TNC summary of GCP lessons, May 2007 8.6 WCS 8.7 WWF summary of GCP lessons, May 2007 80 GCP EVALUATION FINAL REPORT ANNEX H. GCP PRINCIPLES Global Conservation Program Principles from GCP II RFA of 30 October, 2002. The following principles will guide the development of Phase II of the Global Conservation Program. These principles represent the approach the Economic Growth Agriculture and Trade Biodiversity Team’s Global Conservation Program will employ: • Programs should use a threats based approach. (see annex xxx) Programs should clearly identify and prioritize the proximate threats to the conservation targets. Although it may not be possible to address all threats at the site, a clear, site-specific link between threats and proposed abatement activities must be demonstrated. We do recognize that there are activities that may not address the threat directly, or fit neatly into the threats framework, but that the activity is necessary to conserve biodiversity. • Programs should focus on globally important sites for biodiversity conservation. Programs must demonstrate the global priority of the site. For example, partners may find it useful to make reference to one of the widely accepted, peer-reviewed priority setting exercises. Additional, site-specific information substantiating the value of a particular area is useful, where available. • Programs should be adaptive. While the initial design of program activities should be sound, conservation needs are complex and constantly evolving. Programs should therefore be structured in such a way that they monitor their progress, generate timely information for management, and adapt the program as needed. • Programs should foster sustainability. Partners should discuss how conservation achievements will be sustainable beyond the end of the Agreement. Partners should also explain how continued financing for ongoing activities will be secured. While it is not necessary to identify specific sources of continued financing, applications should describe the approach for identifying and securing this funding. In addition, any programs that involve extractive use should clearly discuss (a) the likelihood that extractive activities will be ecologically, socially, and economically sustainable; (b) how over-harvesting will be controlled; and (c) how extractive use will contribute directly to biodiversity conservation. • Programs should be participatory. (See annex x) Applicants should discuss how programs incorporate the equitable and active involvement of stakeholders in all stages of program design and implementation. Attention should be given to the differences in the ways men, and women and indigenous groups use, mange, and conserve biological resources. The inclusion of traditionally marginalized stakeholders, such as women and indigenous peoples should occur whenever possible. • Programs should help NGOs expand their initiatives. Proponents are expected to have ownership of proposed programs and to invest their own resources in accomplishing the results defined under the program. Proposed cost-share should be clearly elaborated, along with other indications of institutional commitment to the program. • Programs should strengthen in-country capacity and foster collaboration. Conservation of natural systems depends critically on the engagement and commitment of key stakeholders – local people, government, corporations, NGOs and donor institutions. Establishing strategic partnerships to help achieve conservation goals is key. To become strategic partners institutional strengthening may be needed for both government and nongovernmental organizations. GCP EVALUATION FINAL REPORT 81 • Programs must be results-oriented. Since the goal of the program is biodiversity conservation, largely through improved management of globally significant habitats, the programs should articulate how they plan to assess program impacts. Partners should discuss how they would track performance and report on progress. Efforts to measure habitat quantity and/or quality are encouraged where appropriate. • Programs should integrate learning into program design. Substantive analysis at the site level and efforts to disseminate lessons learned to the broader conservation community should be integrated into programs, particularly at multiple sites or larger scales. We support the learning and dissemination from both successes and failures that improve the design and management of programs. The Biodiversity Team supports approaches that best achieve biodiversity conservation, whether they are tested and proven or new and innovative. Innovation will be supported where programs demonstrate an understanding of risk and the ways in which they intend to manage the risk. • Programs should complement other conservation and development activities. Integrated conservation and development at the landscape or regional scale requires coordinated action by many actors. Where appropriate, partners should indicate how the their conservation efforts contribute to or compliment development activities of USAID, other donors, host-country governments, the private sector, and other institutions. However, proposed development activities must demonstrate a link to the conservation objectives. 82 GCP EVALUATION FINAL REPORT ANNEX I. EVALUATION TEAM Dr. Bruce Byers served as the Team Leader/Conservation Biologist on the evaluation team. He is an ARD Senior Associate, and a biodiversity conservation and natural resources management specialist with more than 20 years of experience in conservation and natural resources management. He is based in ARD’s Washington, DC office. His work combines an academic background in ecology and conservation biology with extensive practical experience in both applied biological and social sciences, and focuses on the development of sustainable solutions to conservation problems. Dr. Byers has had extensive field and teaching experience in Africa, as well as in North and South America. He has significant experience in program and project monitoring and evaluation, and has led many assessment teams and managed analytical processes with multiple stakeholders. Mr. Jared Hardner was the Evaluation Specialist on the evaluation team. He is Managing Partner of Hardner & Gullison Associates, LLC, and has 15 years of professional experience in the natural resources and environmental management sector. He has worked extensively on conservation issues in the US and Latin America, and also has experience in Asia, Africa, and Eastern Europe. Mr. Hardner has designed and managed a series of large-scale multidisciplinary program evaluations for major conservation donors such as the Gordon & Betty Moore Foundation and National Fish and Wildlife Foundation. GCP EVALUATION FINAL REPORT 83 84 GCP EVALUATION FINAL REPORT US Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523 Tel: (202) 712-0000 Fax: (202) 216-3524 www.usaid.gov