PROMOTING TRANSFORMATIONS BY LINKING NATURE, WEALTH AND POWER (TransLinks) PERFORMANCE EVALUATION FINAL REPORT Tonle Sap Biosphere Reserve, Cambodia. Photo: David Wilkie, WCS April 2014 PROMOTING TRANSFORMATIONS BY LINKING NATURE, WEALTH AND POWER (TransLinks) PERFORMANCE EVALUATION FINAL REPORT Evaluation Services IQC Task Order: AID- OAA-TO-13-00040 Order No: 01 April 2014 This publication was produced at the request of the United States Agency for International Development under the Evaluation Services IQC Task Order AID-OAA-TO-13-00040, awarded to International Business and Technical Consultants, Inc. (IBTCI). Dr. James Morton (IBTCI) conducted the TransLinks evaluation and prepared this report. Dr. Arthur Mitchell (IBTCI) served as Team Leader. DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page i CONTENTS Executive Summary .......................................................................................................................................................... iii Acronyms............................................................................................................................................................................. v A. Introduction ................................................................................................................................................................... 1 1. Introduction to TransLinks ................................................................................................................................ 1 2. The Evaluation Questions .................................................................................................................................. 1 3. Evaluation Approach and Methodology .......................................................................................................... 2 4. Stakeholder Review and e-Survey .................................................................................................................... 2 5. Structure of the Report ...................................................................................................................................... 2 B. Findings ............................................................................................................................................................................ 4 1. Project Design and Management Results ............................................................................................................ 4 The Initial Program Description............................................................................................................................ 4 An Early Change of Direction ............................................................................................................................... 6 The Standing Forest Markets Conservation Initiative ..................................................................................... 7 Consortium Structure and Management ............................................................................................................ 8 Performance Management Plan Results ............................................................................................................10 Conclusion ...............................................................................................................................................................11 2. TransLinks Knowledge Generation ....................................................................................................................13 Product Typology ...................................................................................................................................................13 PES - REDD ..............................................................................................................................................................15 PES - Water Services .............................................................................................................................................16 Livelihoods-Biodiversity Interfaces.....................................................................................................................16 PES - Cross-Cutting Issues ...................................................................................................................................18 Outliers .....................................................................................................................................................................18 Dissemination, Communication, and Advocacy ..............................................................................................19 Knowledge Uptake .................................................................................................................................................20 Conclusion ...............................................................................................................................................................21 3. TransLinks Outcomes ...........................................................................................................................................22 Global/International Outcomes ..........................................................................................................................23 PES - REDD ..............................................................................................................................................................24 PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page ii PES - Water Services .............................................................................................................................................25 Livelihoods-Biodiversity Interfaces .....................................................................................................................26 TransLinks Partner Programs and Capacities ..................................................................................................27 Key Informant and e-Survey Responses ............................................................................................................27 Conclusion ...............................................................................................................................................................32 4. Potential Return on the TransLinks Investment .............................................................................................32 C. Conclusions ................................................................................................................................................................. 34 1. Project Design and Management Results ..........................................................................................................34 2. Performance Outcomes ........................................................................................................................................34 3. Program Dissemination Results ..........................................................................................................................35 Appendix 1. Key Informants Contacted by the TransLinks Evaluation ................................................................. i Appendix 2. Outcomes Checklist ................................................................................................................................. iii Appendix 3. Key Informant Checklist ........................................................................................................................ viii Appendix 4. TransLinks e-Survey Summary ...............................................................................................................xi Appendix 5. Discounted Cashflow Analysis of TransLinks-Supported REDD+ Schemes ......................... xxix Appendix 6. Evaluation Methodology ..................................................................................................................... xxxi List of Tables Table 1 TransLinks Budget Breakdown FY 07 .................................................................................................... 9 Table 2 TransLinks Website Analytics ................................................................................................................20 Table 3 Possible TransLinks Outcomes at the Global/International Level ...............................................30 Table 4 Possible TransLinks Outcomes at National Level .............................................................................30 Table 5 Possible TransLinks Outcomes at Sub-National Sites......................................................................31 Table 6 New Knowledge Generated by TransLinks .......................................................................................31 Table 7 TransLinks Budget Allocations by Activity .........................................................................................33 Table 8 Summary of TransLinks Outcomes ......................................................................................................36 PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page iii EXECUTIVE SUMMARY As part of the Evaluation Services IQC Task Order AID-OAA-TO-13-00040 awarded to International Business and Technical Consultants, Inc. (IBTCI), a final assessment of the program, Promoting Transformation: Linking Natural Resources, Economic Growth and Governance (TransLinks), was conducted from November 2013 to March 2014. This five-year program was implemented from 2007 to 2012 by a consortium led by the Wildlife Conservation Society (WCS) with the goal of increasing social, economic, biodiversity, resilience, and other environmental benefits through sustainable natural resource management. TransLinks focused on global knowledge generation and capacity building, principally through the documentation and dissemination of lessons learned in natural resource management. TransLinks worked to strengthen understanding of the relationships between the way ecosystems work (Nature), the livelihoods that depend on those ecosystems (Wealth), and the institutions that control access to natural resources (Power). The TransLinks evaluation framework addressed how the program achieved its objectives through seven evaluation questions that were used to examine results pertaining to performance outcomes, project design and management, and broader program dissemination: 1. How did WCS/Forest Trends use the lessons learned that they produced internally to influence their (or their international partners’) programs? 2. To what extent was new NRM knowledge generated from this work? (i.e., new peer reviewed articles, new grey literature produced, additional/ new lines of research questions explored) 3. What role did project design (and any changes/evolution in the design) play in the final results of the program? 4. What are the lessons from this partnership and/or consortium? 5. From TransLinks' results, what were the key determinants of success in documenting and disseminating the results of successful NRM tools for greater adoption? 6. Have (and if so, how) these case studies affected the development of the REDD+ mechanism? 7. Have (and if so, how) these case studies affected the development of innovations in PES? To address these questions, the evaluation methodology included a detailed review of TransLinks products and learning materials, program descriptions, annual work plans, budgets, and reports as well as key informant interviews and an e-survey of a broader selection of stakeholders. A Validation Workshop was held at which feedback on initial evaluation findings was received from members of the TransLinks implementing consortium. With regard to performance outcomes and the first two evaluation questions on using lessons learned and generating new natural resource management (NRM) knowledge, TransLinks enabled members of the consortium to significantly strengthen their capabilities in payments for environmental services (PES), particularly for forest carbon. Their programs reflect a stronger understanding of what it takes to make PES operational, an understanding which they are sharing with partners in government and civil society. A substantial body of new NRM knowledge was developed around the issue of land tenure and property rights in PES schemes. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page iv With regard to project design and management results questions 3 and 4—i.e., what worked in the design and what did not—the TransLinks program design was significantly changed twice. The first change was to replace pilot implementation at a limited number of developing county sites with a program to harvest experience and lessons learned through case studies at a much larger number of existing NRM interventions; at about the same time, a decision was made to focus on PES and related tenure issues. This change allowed TransLinks to make a solid body of new knowledge on PES accessible quickly and economically. The second change built on the first and, with the addition of new funds, TransLinks began to concentrate its work on PES for forest carbon. The result was that TransLinks’ impact in REDD+/forest carbon substantially outweighed that in other sectors. With regard to the broader program dissemination results questions 5, 6, and 7 on how land tenure and resource management or biodiversity conservation approaches were used, the assessment suggests that a well-defined communications strategy would have helped TransLinks make more of the evidence it harvested and the strong body of material it put together. As a consequence, it is the members of the TransLinks consortium that have served as the principal dissemination channel for the program’s results. TransLinks had the greatest impact where its knowledge generation, project development, and advocacy work came together to support the formation and implementation of policy in a single country. In Brazil, for example, TransLinks helped demonstrate that theoretical models of REDD+ can be made operational for the sale of carbon credits from sub-national significantly sites in voluntary markets. In at least one case, it has been possible to use progress in one country to show policy makers and practitioners from another country how they might proceed. TransLinks’ work on PES played a similar role. The focus has not been on innovation so much as it has been on showing how PES can be implemented in practice. A very important contribution has been to ensure that the critical question of tenure and property rights does not get overlooked in implementation. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page v ACRONYMS CCBA Climate, Community and Biodiversity Alliance CIFOR Center for International Forestry Research CBNRM Community-based Natural Resource Management CPR Common Property Resources CSO Civil Society Organization EI Earth Institute ESPA Ecosystem Services for Poverty Alleviation EWV Enterprise Works/VITA FT Forest Trend FTR Final Technical Report GEF Global Environment Facility IP Implementation Plan IUCN International Union for Conservation of Nature KII Key informant interview LMIC Lower Middle Income Country LRM Land Resource Management LTRM Land Tenure and Resource Management LWA Leader with Associate (Agreement) MVPs Millennium Village Projects NGO Nongovernmental Organization NI/LTC Nelson Institute for Environmental Studies/Land Tenure Center NR Natural Resources NRM Natural Resource Management NWP Nature, Wealth, and Power OH Outcome Harvesting PA Protected Area PD Program Description PDD Project Design Document PES Payment for Ecological Services PMP Project Management Plan REDD+ Reducing Emissions from Deforestation and Degradation RFA Request for Application RM Resource Management ROI Return on Investment ROSE REDD Opportunities Scoping Exercise SFCMI Standing Forest Conservation Market Initiative SOW Scope of Work TransLinks Promoting Transformation by Linking Nature, Wealth and Power VCS Verified Carbon Standard WCS Wildlife Conservation Society PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 1 A. INTRODUCTION Based on the statement of work (SOW) for the task order as part of the Evaluation Services IQC, International Business and Technical Consultants, Inc. (IBTCI) was commissioned to carry out performance evaluations for three program mechanisms supported by the Office of Land Tenure and Resource Management (LTRM) in USAID’s Bureau for Economic Growth, Education and the Environment (E3): (1) Global Sustainable Tourism Alliance (GSTA), (2) Promoting Transformation by Linking Nature, Wealth and Power (TransLinks), and (3) Property Rights and Resource Governance (PRRG). The overarching framework of the LTRM evaluations addressed how each program accomplished its objectives according to the evaluation questions set forth for it. This report presents evaluation findings for the TransLinks mechanism. 1. INTRODUCTION TO TRANSLINKS TransLinks was a five-year program implemented from 2007 to 2012 with the goal of “increasing social, economic, biodiversity, resilience, and other environmental benefits through sustainable natural resource management.” It focused on knowledge generation and capacity building, principally through the documentation and dissemination of lessons from experience in natural resource management. The program was implemented by a consortium led by the Wildlife Conservation Society (WCS). The other members were the Earth Institute (EI) at Columbia University, Enterprise Works/VITA (EWV), Forest Trends (FT), and the Nelson Institute for Environmental Studies/Land Tenure Center (NI/LTC) at the University of Wisconsin. The consortium was designed to offer expertise in the full suite of issues articulated by the TransLinks RFA. Since 2002, LTRM has worked to strengthen understanding of the relationships between Nature, the way ecosystems work; Wealth, the livelihoods that depend on those ecosystems; and Power, the institutions that control access to natural resource. As had the other mechanisms being evaluated, TransLinks was expected to work within this Nature-Wealth-Power (NWP) paradigm to simultaneously address resources, income generation and governance issues. 2. THE EVALUATION QUESTIONS The task order set forth the following questions for the evaluation of TransLinks:  Project Design and Management Results Questions: What worked in the design and what did not?  What role did project design (and any changes/evolution in the design) play in the final results of the program?  What are the lessons from this partnership and/or consortium? Broader Program Dissemination Results Questions: How LTRM or biodiversity conservation approaches were used:  From TransLinks’ results, what were the key determinants of success in documenting and disseminating the results of successful natural resource management (NRM) tools for greater adoption?  Have these case studies affected the development of the REDD+ mechanism? If so, how? PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 2  Have these case studies affected the development of innovations in PES (payment for ecosystem) schemes? If so, how? 3. EVALUATION APPROACH AND METHODOLOGY Evaluation implementation included a detailed review of the substantial portfolio of TransLinks products and learning materials; analysis of implementation documentation including program descriptions (PDs), annual work plans, budgets, and reports; a review of TransLinks outcomes using a modified outcomes harvesting approach, as reported by the implementing partners, key informant interviews with internal and external stakeholders, and an e-survey of a broader selection of stakeholders. Before drafting the final report, a validation workshop was held to seek feedback on initial evaluation findings from members of the TransLinks implementing consortium. There were no site visits. 4. STAKEHOLDER REVIEW AND E-SURVEY On June 18, 2012, TransLinks held a program results symposium to mark the end of the program. The evaluation used a list of those invited to the event to identify key informants who would be both independent and well-informed regarding TransLinks. The list included representatives of key institutions including the World Bank, Ford Foundation, World Wildlife Fund, The Nature Conservancy, and a number of universities. Emails were sent to all those on the list apart from USAID. Members of the WCS consortium were also asked to suggest possible key informants. Emails were also sent to team members of UN REDD (Reducing Emissions from Deforestation and Degradation) as potentially important stakeholders. Many of individuals contacted did not reply at all; others reported that they knew so little about TransLinks that they did not wish to participate. Appendix 1 lists those who did agree to an interview. Among those who responded, only those who had been directly involved with TransLinks felt able to respond to more than a small proportion of the questions. Their views are reflected at various points in the report to provide independent support for the assessment. An e-survey was carried out to reach a wider group, with a questionnaire emailed to participants of the TransLinks symposium and various workshops held around the world including in the United States, Vietnam, Philippines, Tanzania, and Ghana, for a total of 162 participants. 5. STRUCTURE OF THE REPORT The report is structured as follows:  Section A, Introduction, introduces TransLinks, reviews the evaluation questions and presents the methodology and tools used for the evaluation.  Section B, Findings, presents the evaluation’s findings in four subsections:  Subsection B.1, Project Design and Management Results, addresses the first two evaluation questions: the way the evolution of the project design and the management of the TransLinks consortium affected the final result.  Subsection B.2, TransLinks Knowledge Generation, reviews the portfolio of TransLinks knowledge products from case studies to tools and manuals, grouping them into four technical areas to identify where the program has generated the greatest critical mass of knowledge products. The program’s approach to knowledge generation and dissemination is also discussed. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 3  Subsection B.3, TransLinks Outcomes, draws on the responses of the implementing partners and external stakeholders to outline where the program’s work has led to significant developments in natural resource management. In doing so, it endeavors to answer the three evaluation questions on the program’s broader dissemination results.  Subsection B.4, Potential Return on the TransLinks Investment, presents an indicative assessment of how the benefits generated by TransLinks outcomes compare to the program’s cost. Section C presents the evaluation’s overall conclusions, structured as answers to the evaluation questions. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 4 B. FINDINGS 1. PROJECT DESIGN AND MANAGEMENT RESULTS This section sets out evidence relating to the following evaluation questions:  Project Design and Management Results Questions: What worked in the design and what did not?  What role did project design (and any changes/evolution in the design) play in the final results of the program?  What are the lessons from this partnership and/or consortium? TransLinks dates from September 2006, when USAID awarded $3,997,764 to WCS as support for the Equity, Sustainable Growth and Natural Resources Conservation program. WCS and its consortium partners were to fund a further $2,839,761 as a cost-share contribution. The five-year program was due to end in September 2011. After a short no-cost extension the program’s final end date was June 30, 2012. The program’s evolution can be tracked through three stages: 1. The initial Program Description issued with the request for applications (RFA), with the later version attached to the 2006 letter of award 2. The preparation of the Year 1 Implementation Plan, an early but substantial change of direction that was not recorded as a formal revision to the program direction 3. Introduction of a major new initiative in 2009, which led to a 2010 ceiling increase of $2,822,236 and was accompanied by limited revisions to the program description. The TransLinks Leaders with Associates mechanism (LWA) was awarded to WCS as consortium leader. The other members were the Earth Institute at Columbia University, Enterprise Works/VITA, Forest Trends, and the Nelson Institute for Environmental Studies/Land Tenure Center at the University of Wisconsin. The consortium was designed to offer expertise in the full suite of issues articulated by the TransLink RFA. The Initial Program Description The Program Description issued with the USAID RFA was broad. With an objective of realizing social and economic benefits through healthy ecosystems and sustainable resource management, it listed intermediate results ranging from improved resource management science and equitable natural resource (NR) governance to sustainable economic production. A fourth result was to provide “field support” and technical assistance to USAID operating units and local partners. The RFA was for LWA, designed to allow USAID country missions to buy into the core program by funding linked Associate Awards. The Program Description described how primary awards to the program leader would focus on governance, policy, research and synthesis, and institutional development and capacity building. The secondary Associate Awards could cover anything from information production and knowledge management to disaster and conflict prevention/mitigation, PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 5 gender, extractive industry transparency, and GIS implementation and training. Small grant projects, presumably to civil society organizations (CSOs) and others in-country, were intended. With a relatively small budget, it was inevitable that these broad ambitions would be reduced in the final Program Document annexed to the 2006 TransLinks Letter of Award. More challenging and specialized potential focus areas, such as extractive industry transparency and conflict prevention/mitigation, were left out. The focus was narrowed to a TransLinks objective of improving “natural resource management and governance to enhance biodiversity conservation and productivity for sustainable poverty reduction and economic growth.” Four pathways to this objective were defined, each comprising a number of activities as follows: 1. Knowledge generation through applied multisectoral research:  Convene advisory board  Elect four to six research sites through surveys of USAID mission staff, local partners, and a Global USAID mission web survey  Establish a local advisory committee at each site  Working with national partners, generate and disseminate lessons from a suite of relevant applied research actions  Clarify effective models as examples A clear set of research issues was identified based on the Nature-Wealth-Power triad. These included the way ecological potential, market conditions, and human capital determine the potential for NR resource enterprise; how sectors and institutions come together to constitute an effective NR governance system; and what factors determine the transition from unsustainable to sustainable resource use. 2. Decision-support systems for effective planning and adaptive management:  Global analysis of existing tools  Development of a diagnostic and decision support toolbox 3. Outreach, training, and capacity building for effective governance:  Draft training/capacity-building plans at each site and for regional and global training  Develop and field test curricula and modules  Establish a remote coaching system  Provide targeted training/knowledge building at existing conferences/meetings  Monitor knowledge growth annually and adapt training effort accordingly 4. Knowledge sharing—scaling up from farmer-to-farmer to global reach:  Establish web-based knowledge sharing  Establish “knowledge leadership process”  Identify key networks for knowledge dissemination  Ensure USAID gets the information it needs from TransLinks  Craft local knowledge plan at each site PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 6  Ensure local findings are incorporated into national and global learning Associate Awards were expected to allow TransLinks to explore models at additional sites and substantially increase the program’s reach. The timeline presented at the end of the Program Description included a number of activities intended to generate and support Associate Awards: surveys of USAID missions; proposals for USAID-funded site-based research; and training and coaching support for USAID Washington and missions. An Early Change of Direction Early discussions led to a decision not to focus on the generation of Associate Awards, as it was felt that not enough USAID mission staff were likely to request Associate awards under TransLinks. This assessment was later confirmed when the TransLinks team found little traction with mission staff and encountered skepticism about the value of the knowledge-generation effort. One aim of Associate Awards is to lower the transactions costs of contracting and provide mechanisms for missions to access services. The existence of two similar LWA programs in the environmental sector meant that TransLinks offered no clear added value in this regard. The Implementation Plan for FY 2007, i.e., TransLinks Year 1 (IP07) reflects a substantial change of direction from the Program Description. This change was apparently decided in planning meetings, but the evaluation could not locate a formal record of the changes made. IP07 maintains the titles of three of the four TransLinks pathways, but the content of each was quite different from that set out in the Program Description, especially for the first: Knowledge generation through applied multisectoral research. Instead of identifying four to six research sites to generate and disseminate “lessons from a suite of relevant applied research actions,” the IP07 approach appears to have been that lessons learned would be synthesized from the current “state-of-knowledge” and harvested from projects that were already being implemented, in many cases by members of the TransLinks consortium. After a workshop to identify knowledge gaps, comparative case study research at those existing projects would fill the gaps. Where the Program Description had expected the work to be done in close collaboration with USAID mission staff and local partners, this is given little emphasis in IP07. Subsequent implementation plans developed the approach set out. IP08 lists case studies at 17 different sites across Africa, Asia, and Latin America. Decision support systems for effective planning and adaptive management. The 2006 Program Description set out a two-step effort: Existing tools would first be analyzed, then the analysis would be used to develop a diagnostic and decision-support toolbox. The Year 1 IP might have been expected to concentrate on the analysis step as a basis for a subsequent development program. Instead it proposed an immediate start on developing four different tools. The work was divided among WCS consortium partners without a clear framework to integrate their efforts into a single toolbox. Outreach, training, and capacity building for effective governance. In the Program Description, this component was intended to result in tailored training/capacity-building plans for partners at sites at regional and global levels. In IP07, it was replaced with an activity for “cross-partner exchange, training, and capacity building.” This gave greater emphasis to building capacity within the TransLinks consortium, with a principal objective of allowing staff from consortium members and their local partners to participate in a number of high-level workshops on PES, economic options in biodiverse areas, livelihoods in protected areas (PAs), etc. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 7 Knowledge sharing. Scaling up from farmer-to-farmer to global reach: The Program Description described site-specific activities representing the “farmer-to-farmer” element of this component. These activities seem to have been lost in the implementation plans. IP07 proposes three activities: support to Forest Trends’ existing Ecosystems Market Place website; an annual “knowledge leadership conference” (as opposed to “a knowledge leadership process” in the Program Description); and the establishment of a TransLinks Advisory Group made up of “international opinion leaders.” The latter two of these activities were at first delayed (FY 07 Annual Report) and later apparently abandoned. They do not appear in IP08, which replaced them with panel discussions and meetings in New York and Washington and attendance by TransLinks staff at international conferences “to disseminate work, to gain feedback on the program and to stay abreast of current topics and issues in the field.” The original Program Description was ambitious and some reduction in scope was inevitable, especially once it was decided not to press for Associate Awards. That said, the implementation plans and reports for the first two years (FY 07s and 08) indicate that TransLinks had not found a clear strategic direction to replace the one set out in the Program Description. Instead, the planned work appears to be rather inward looking, driven principally by the research and implementation interests of TransLinks consortium partners. The Standing Forest Markets Conservation Initiative The Implementation Plan for FY 09 marks the second significant change for TransLinks. Three additional activities were shown, under the title Standing Forest Conservation Market Initiative (SFCMI). Unlike earlier plans, IP09 only presented a table listing the proposed activities; there was no text to explain the background or thinking behind this new initiative. However, a December 2009 Action Memorandum recommending a $2.4 million increase to TransLinks’ funding ceiling provided a succinct statement of the justification: “The SFCMI allows TransLinks to greatly expand its reach at a point in time when interest in [payment for ecosystem] services is exploding, in part driven by interest in carbon trading and tropical deforestation under the UN Framework Climate Convention.” As TransLinks already had some focus on PES, it was a logical place for a response to that explosion of interest. The evaluators understood that interest from the highest level within USAID contributed to this decision. SFCMI consisted of three additional TransLinks activities (formerly called pathways). The titles for the three activities were the same as those applied to the first three TransLinks activities since IP07: 1) knowledge generation; 2) tool development; and 3) cross-partner exchange and capacity building. However, there was a marked difference in style between the two groups. For the original TransLinks activities, almost all the planned outputs are either paper products—case studies, guidelines, etc.—or participation in workshops. With some exceptions, these outputs did not target a specific development outcome. “One paper of research results and one paper of guidelines/recommendations regarding land tenure conflict resolution” was a typical output statement. In contrast, it was much easier to infer the expected development outcome from the SFCMI output statements. The knowledge-generation activity presented a coherent package of country-scoping studies supported by background reviews of legal issues for PES in carbon and water services, all designed to enable greater use of PES in the countries concerned. This was carried through into the tools-development activity. The biggest difference was in the third activity: cross-partner exchange and capacity building. Outputs under the original TransLinks program were mainly paper products: workshop proceedings, case studies, training materials, and PowerPoint presentations. SFCMI outputs identify the ultimate beneficiaries—community organizations, government, and other partners in developing countries—with outputs were specified in terms of their capacities. “Community-based organizations will become informed and better-equipped to respond to REDD and other PES related opportunities,” and “Governments will be better-equipped to formulate and implement avoided deforestation … strategies,” are two examples. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 8 In the process of applying for a financial ceiling increase for SFCMI, the TransLinks Program Description was revised. However, the revisions were minimal, just a few paragraphs about the new work. The new program design (PD) presented all the original material about country programs of action research and associate awards unchanged. It did not mention the significant changes that had been made in that respect. For FY10 (TransLinks Year 4), the separation between TransLinks and SFCMI was removed and the plan reverted to four activities, although but the wording indicates a clearer focus on PES. The knowledge generation activity became knowledge generation and synthesis with two subactivities: i) designing PES and ecosystem service management programs to deliver multiple benefits to the rural poor, and ii) governance mechanisms that promote equitable benefits from payments for ecosystem service programs. Similarly, tools development was now tools development for PES. Under the cross-partner exchange and capacity-building activity, the major emphasis was on the Katoomba Workshop process. There was an explicit focus on forest carbon, with the aim of supporting the completion of project design documents as the final step in the validation of multiple forest carbon projects. To a substantial extent, therefore, IP10 can be seen as a takeover of TransLinks by the Standing Forest Markets Conservation Initiative or as a merger of these two programs under the TransLinks umbrella. As far as the evaluator has seen, it also marks the effective end of the TransLinks planning process. The available version of IP11 is incomplete and shows just two activities: synthesis activities and dissemination and training events. A number of activities have an end-of-project feel, for example an article for World Development, a TransLinks Video, and a TED talk. However, there is no discussion of an exit strategy setting out how TransLinks’ work might be carried forward. Consortium Structure and Management The WCS consortium was made up of three NGOs and two academic institutes. A summary of their mission statements gives an outline of the consortium’s capability and interests: NGOs.  Wildlife Conservation Society: “to save wildlife and wild places across the globe”  Forest Trends: “to promote sustainable forest management and conservation by creating and capturing market values for ecosystem services”  Enterprise/VITA: “to combat poverty through economic development programs based on sustainable, enterprise-oriented solutions” Academic Institutions.  The Earth Institute: “blending scientific research, education, and practical solutions, to help guide the world onto a path toward sustainability”  The Land Tenure Center: “to promote equitable and sustainable land stewardship” For a program working in the Nature Wealth Power framework, the WCS consortium offered a well￾balanced set of capabilities, qualified to effectively address all aspects: the scientific analysis of specific ecologies; governance and property rights arrangements affecting the ecosystem; and the sustainable exploitation of the economic opportunities it offers. When evaluating programs implemented by a consortium of this type, a central question concerns the extent to which the different partners come together to provide a service that is greater than the sum of its parts. For this, governance, particularly planning and budgeting arrangements, is critical. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 9 The 2006 Program Description sets out how the WCS consortium would be supervised by an Executive Committee, comprised of representatives from each of the members. The committee was to meet annually, with quarterly conference calls. Its duties were to include selecting sites/countries for applied research; appointing a lead partner for each site; reviewing progress reports and outputs; guiding tools development and dissemination; and setting priorities for training and capacity building. To support the committee, a gender-balanced Advisory Board was to be appointed from “renowned academics, practitioners, and USAID staff.” The board was to advise on initial design and site selection, carry out annual reviews, and ensure that TransLinks activities were needs-based and its products were of “both local and global value.” As previously noted, the Advisory Board was never appointed. It is not clear that the proposed Executive Committee was established as a formal body or met regularly as was expected. The evaluation found no records of meetings or minutes. The Implementation Plans record no discussion on the allocation of resources among the four TransLinks pathways, the selection of sites, and the nomination of the lead partner for each activity. Instead, the TransLinks Chief of Party advised that the Implementation Plans were prepared on the basis of proposals put forward by the different consortium partners, and agreed after discussion with the partners and USAID. The evaluation identified only one detailed annual budget, that for FY 2007, the first year of TransLinks. Table 1 shows the breakdown by activity and consortium partner. Assuming subsequent annual budgets were similar, it gives an idea of how the consortium worked. Table 1 TransLinks Budget Breakdown FY 07 Activities Leading On Activities Led Activities Involved In % USAID Funds Allocated EI 1.1 1 8 13.2 EWV 2.3, 3.2 2 7 11.6 FT 1.3, 2.4, 3.1, 4.1 4 6 6.7 NI/LTC 1.2 1 6 7.8 WCS 1.4, 2.1,2,2,3.3,3.4 4.2,4.3 7 15 41.1 WCS Administration 19.5 Total 15 - 100.0 The following points sum up this aspect of the consortium’s operations:  Although all consortium partners were active, only three activities involved all five: 1.1 state of knowledge briefs; 1.4 identification of knowledge gaps; and, 4.2 annual Knowledge Leadership Conference. As already noted, the last of these never happened.  Even where it was not lead, WCS was allocated staff time under every item, in some cases for a greater sum than that allocated to the lead partner. This was in addition to WCS’ administration charge.  Some lead partners, Forest Trends in particular, received very little, presumably because it was doing most of the work as a cost-share contribution. This snapshot from a single year may be misleading. It is, for example, known that the administrative cost over the four years to FY 10, was 10%, not the 19% shown above. Nevertheless, the less detailed budget data from later years shows a similar pattern, at least until the introduction of the SFCMI. It is of PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 10 a program where the funds were divided among consortium partners without a clear framework setting out how the work of each one did would contribute to an overall objective. It is notable that lead partners were identified for subactivities but not for the top-line pathways. Without, for example, a lead partner for the knowledge generation pathway as a whole, it is difficult to see how the large number of different activities were to be brought together. This is clearest in IP08. Under knowledge generation there are nine subactivities. The first two involve private sector payments for ecological services and property rights and resource tenure for income security and resource conservation, each billed as a cross-regional study. The first included seven case studies and the second three. The case studies were widely dispersed and diverse in character. The Implementation Plan treated them as standalone interventions, with separate outputs for each one. There was no provision for work to pull them together to provide a robust synthesis or a foundation for a set of guidelines and tools. It is perhaps indicative of the lack of integration that there was a case study on Uganda forests under Activity 1.1 and a separate subactivity, 1.4, which was also about Uganda forests. To sum up, in the early years at least, it appears that TransLinks became a grant fund, allocating finance to proposals put up by the consortium members than as an integrated strategic program working to deliver a clear set of top-level objectives. It may be that these more strategic objectives would have become clearer as the program developed. And indeed, the introduction of the SFCMI does seem to have created a stronger strategic focus. Nevertheless, any program will be more effective if it has focus from the beginning. This is especially true when the aim is to build on a strong framework that has already been developed, as was the case with TransLinks, which was intended as the next step in USAID’s earlier work on the Nature, Wealth, and Power framework. Consortium management is rarely easy and tensions over strategic direction, management style, and resource allocation are common. It is understood that there were difficulties over the introduction of the SFCMI, mainly it seems because the greater role given to Forest Trends upset the balance of the consortium. Differences in approach between the academic and non-academic partners were also mentioned as an issue. There is nothing to suggest that these difficulties were out of the ordinary. It is, however, worth pointing out that a stronger governance structure would have ensured that they were managed to the satisfaction of all partners. The planned Advisory Board and Executive Committee structure would have been particularly helpful in this respect. Stronger planning and budgeting processes would also have been useful. Performance Management Plan Results A spreadsheet entitled Final TransLinks Project Management Plan (PMP) Results presents a number of program output indicators. They can be summarized as follows: Indicator 1: Leverage. TransLinks generated only one Associate Award, in South Sudan. At a final figure of $30 million, this was far above the original target of $9 million in Associate Awards. However, it is understood that this was more the result of the special situation in South Sudan than it was a genuine response to what TransLinks was offering. (The evaluator did not review this award as per the SOW.) Without it, TransLinks’ leverage is reduced to the WCS consortium’s cost-share contribution of $2,839,760 and two small grants totaling $310,000: from UNDP-GEF (Global Environment Facility) and the U.S. Forest Service. Indicator 2: Training. Against a target of 2,841, TransLinks reported that it had trained 2,168 women and 3,010 men: 5,178 in total. These figures include attendance at workshops and seminars, several of them longer than three days. A substantial proportion of the total (1,400, or 27%) were recorded at just one workshop: the Katoomba meeting in Brazil. The other Katoomba workshops made up a further PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 11 23%. The next largest group was of people who attended workshops held in the United States: New York, Madison, and Washington. The Brazil Katoomba, for example, lasted one and a half days and was described as a public meeting. The very large audience included eight Brazilian state governors and speakers included important international figures such as the California Deputy Secretary for Energy and Climate Change. With five speakers in each 90-minute session, presentations were short. At the other extreme, a smaller two-day workshop in Rwanda included one day of technical training and introduced two environmental modelling techniques. Indicator 3: Changes in Implementation of Policies, Laws, and Regulations. Only one item is reported against this indicator, a Poverty Reduction Policy in Laos. Indicator 4: Special Studies Completed. Against a target of 127, the PMP file shows 184 studies, better described as knowledge outputs, given that the list includes briefs, reviews, primers, and guidelines as well as case studies. There is an element of double counting, with different language versions and different volumes of the same report treated as separate outputs. The Outcome Harvest (OH) method, discussed below, suggests that the PMP spreadsheet data for Indicator 3 does not adequately reflect TransLinks’ policy impact. Nevertheless, the overall picture is clear. The studies and other written material recorded against Indicator 4 mark by far the most substantial TransLinks output. How far that material has influenced policy and practice in NR management is the central question for this evaluation. Conclusion With a relatively small budget available, even with Associate Awards, it was inevitable TransLinks would have a narrower focus than that set out in the original Program Description. What that focus should be was a central question. Potentially it might have been any of the many topics covered under the Nature, Wealth, and Power paradigm. This treats NRM as an issue, which covers the widest possible range: from the conservation of springbok and Oryx in Namibia and Botswana to forest loss to farming in Madagascar and Mali, from soil depletion to urbanization, and so on.1 As TransLinks has shown, catchment management and marine and coastal management are just two of the many other potential areas for study and action under the umbrella heading of NWP. If the TransLinks program were to be realistic, a decision would have to be made on which NRM issues to address. It was also important to identify issues where TransLinks would be able to add real value in the form of new knowledge and lessons learned. Many NRM issues have been researched in great detail over many decades, in some cases since the first half of the 20th century, and there is limited scope to generate new knowledge. The Program Description annexed to the TransLinks letter of award was a practical interpretation of the RFA document. It set out a clear strategy of identifying four to six countries where programs of action research would generate and disseminate knowledge on NRM policy and practice. This would lead to “improved management and conservation across diverse landscapes.” The question of which specific NRM issues would be tackled was to be decided through an initial review to identify “an illustrative list of NWP questions that will likely form the core of the research.” The language— “illustrative,” “likely”—seems more designed to keep options open than to identify robust development outcomes as TransLinks objectives. 1 Nature, Wealth, and Power—Emerging Best Practice for Revitalizing Rural Africa, USAID 2002. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 12 Once the decision had been made not to focus on four to six countries and not to try to generate Associate Awards, the TransLinks team drew up a program based on case studies. By the end of the first year it had been decided that these would address two major NRM issues. These were summed up in the FY 07 Annual Report as: “payments for ecological services (PES) and resource tenure/property rights.” The two were seen to be linked. For PES projects, “sustainability and ability to deliver conservation and income generating benefits” . . . “will depend heavily on having the effective and equitable governance systems in place to deliver those benefits.” For IP08, 15 case studies were chosen: nine in a cross-regional study of PES and six in a cross-regional study of tenure/property rights. They covered a wide range of technical issues in eight countries: pastoralism; resettlement from protected areas; photographic/safari tourism; extension of commercial agriculture; charcoal production; and furniture production. Other issues in other countries were taken on as separate activities, outside the two cross-regional sets of case studies, adding to the list. There is no sign that TransLinks was seeking synergies between Nature, Wealth, and Power, as might have been expected; for example, no case studies were looked at under both the PES and the property rights activities. A case study on charcoal production in Uganda is not obviously linked to a separate activity reviewing deforestation and poverty in Uganda. It was not clear how the different sets of studies would be brought together into robust synthesis studies. Pastoralism is the clearest example of all. In addition to a case study covering pastoralists in Tanzania, Mali, and Mongolia, there were two separate FY 08 activities in the same sector and region: one on urban investment in rural cattle herds in Mali, the other on common property institutions and rangeland management in the Sahel. Taken together these might have formed a coherent piece of work. As stand￾alone initiatives, they were unlikely to add much. The Implementation Plan document argues that there had been relatively little previous work on pastoralism. Pastoralism and the management of semi-arid rangelands have, in fact, been researched extensively since the 1980s and before. To sum up, during the first two years of TransLinks’ operations, progress was made in developing a program structured around PES and property rights. However, the long list of activities undertaken in an ambitious attempt to cover a variety of ecosystems and a range of ecosystem services meant that the program remained relatively unfocused. The diversity also reflected the fact that study site selection had depended on the willingness of field staff to participate as well as on opportunities to leverage previous USAID support. Selection criteria were put before the technical nature of work being done at each site. The addition of the Standing Forests Conservation Market Initiative changed the balance of the TransLinks program. Although other work streams continued, it gave the program a stronger focus. For the first time, the FY 10 program presents a package of work that seemed targeted at specific development outcomes: preparing project design documents for forest carbon projects, developing subnational REDD programs, and raising awareness of PES, for carbon sequestration in particular, through Katoomba workshops, etc. All knowledge-generation programs face a critical, strategic choice between strategic direction and flexibility. The first may be better for delivering clear development objectives but it risks losing the ability to adapt to changing circumstances and exploit opportunities as they arise. The fact that TransLinks was in exactly the right place to respond to the growing interest in PES for forest carbon in 2009–2010 was clear justification for the flexible approach adopted in the earlier years. That said, it is arguable that TransLinks could have struck a better balance between direction and flexibility, and that the WCS consortium did not fully become more than the sum of its parts as a result. There were two main reasons: PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 13 The early revision abandoned the clear but still flexible strategy set out in the Program Description and did not define a clear replacement. The decision not to establish an Advisory Board and stick to the schedule of Executive Committee meetings meant there was no forum in which strategic direction could be reviewed. 2. TRANSLINKS KNOWLEDGE GENERATION This section sets out the evidence relating to the following evaluation question:  Broader Program Dissemination Results Questions: How LTRM or biodiversity conservation approaches were used: From TransLinks’ results, what were the key determinants of success in documenting and disseminating the results of successful natural resource management (NRM) tools for greater adoption? The TransLinks Final Technical Report sums up the program’s accomplishments as follows:  31 PES case studies  13 technical manuals and primers on PES  11 peer-reviewed journal articles  10 technical trainings, 22 workshops, and eight seminars on different aspects of PES, raising awareness and technical skills of 3,010 men and 2,168 women  Some 80 other products including 40 reports, 11 brochures, six videos and 32 technical briefs To map this large portfolio in a clear way, the next section describes the different types of products. They are then reviewed under four different headings: 1. PES - REDD 2. PES - Water services 3. Livelihood-biodiversity interfaces 4. PES - Cross-cutting issues A small group of products do not fit any of these categories. These outliers are reviewed in a separate section. A further section reviews knowledge generated on two critical cross-cutting issues: gender and social relations. This analysis has two aims. The first is to allow readers of this report to gain an overview of what TransLinks has produced. The second is to make an indicative assessment of what program outputs may have contributed to NRM outcomes in the broadest sense, i.e., to the adoption of new approaches to NRM and to improved models for development intervention in the sector. Product Typology The NRM portal on which the TransLinks products are stored does not classify them in exactly the same way as the final technical report. The following discussion is based on the classification used on the portal. Neither classification is entirely consistent. At various points adjustments that could help make for a more logical layout are noted. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 14 Case Studies. The majority of the case studies are relatively substantial reports (20 to 50 pages) on individual NRM sites.2 They are presented in a clear, attractive format describing the history of the NRM intervention being implemented at the site and discussing, to a greater or lesser extent, challenges encountered and lessons learned. Interventions are presented at a range of different stages: from preliminary scoping studies for potential projects through to situation reports on well-established projects with several years of implementation experience. Two synthesis papers are also presented under the heading Case Studies. However, neither of these draws on the 18 case studies presented. One reviews lessons learned on land tenure for REDD projects from nine other case studies apparently unrelated to TransLinks. The second looks at human wildlife conflict compensation schemes, combining theoretical analysis of micro insurance approaches and experience in another set of case studies not otherwise related to TransLinks. Briefs and Brochures. The 32 documents presented under this heading fall into two main groups. The first is a set of 13 briefs prepared by the Wisconsin University Land Tenure Center in its series of tenure briefs. In around 10 pages plus references, these present well-argued reviews of key issues such as human wildlife conflict or important sets of evidence—for example, an analysis of Mexico’s experience with PES for watershed management. Two other papers present similar material but not as part of the tenure brief series: one on tradeoffs for eco-labels and one on human rights and conservation. The second set is 10 two-page briefs prepared by the Earth Institute for TransLinks. These are concise summaries of experience and lessons learned at a varied range of case study sites in 10 countries: from a silvo-pasture scheme in Nicaragua to clearance of invasive species in South Africa and forest restoration in China. None of them are about the same sites as those reported under the heading Case Studies. The remaining seven documents present a mix of summary documents and brochures from individual NRM sites, advocacy material, and publicity for TransLinks tools. Similar material is also shown under the heading of Brochures: a set of seven brochures for different wildlife-friendly enterprises, wildlife friendly resource material, a PES brochure, etc. Two documents filed under Country Studies are better described as briefs. Prepared by International Union for Conservation of Nature (IUCN) for TransLinks, one is a study of legal frameworks for REDD and the other a study of legal frameworks for PES. 2 A number of studies are presented in more than one language. One paper is also classified more correctly as a brief, not a case study. If this double counting is eliminated, the NRM portal presents 18 case studies and two synthesis papers under the heading Case Studies. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 15 Tools/Country Studies. Apart from the two papers mentioned above, the material shown under the heading Country Studies all relates to the approach REDD Opportunities Scoping Exercise, or ROSE. As such it is grouped here with the other material classified as tools. The 16 documents under this heading include eight relatively substantial primers and manuals on specific issues, some sets of resource materials, not all of it well organized or accessible, and reports of experience implementing the ROSE tool. A number of the primers and manuals are available in two, three, or four languages. Journal Articles. The NRM portal lists seven articles published in a range of different journals. The final technical report lists a further 11 synthesis papers that were expected to be published. These include a set of six papers reviewing land tenure, forest carbon, and governance that have been accepted by World Development for publication in March 2014, and one which is in press with Ecosystem Services. Workshops/Seminars. The workshops and seminars shown on the Resource Management (RM) portal fall into three groups: 1. A series of major Katoomba events promoting PES in Tanzania, Ghana, Brazil, and Vietnam 2. Two value chain workshops promoting conservation marketing in Tanzania and the Philippines 3. Five practitioner workshops, mostly in the United States, on technical issues such as bundling and stacking and land tenure in PES schemes. The portal also records TransLinks’ involvement in meetings organized by others, such as an IUCN event in Barcelona and a UN event on carbon markets for climate change. PES - REDD The addition of the Standing Forests Market Initiative in 2009 moved TransLinks toward PES for forest carbon. The result is that the most substantial body of TransLinks work has to do with this topic. It can be summed up as follows: Activities in Developing Countries. Katoomba: Workshops in Brazil, Ghana, Tanzania, and Vietnam. Although these looked at PES in the broader sense, forest carbon was given the greatest emphasis. ROSE: The REDD Opportunities Scoping Exercise demonstrated in Tanzania, Ghana, and Uganda. Project Development: Support to the development of project design documents (PDDs) for REDD schemes in Cameroon, Cambodia, Guatemala, Nepal, and Brazil. Training: Courses on PES and REDD for community leaders and others in Peru, Honduras, and South Africa. Case Studies and Technical Analysis. Case Studies: A full study from Madagascar and reports on the PDD efforts in Nepal and Brazil. Short briefs on schemes in Nicaragua, Brazil, China, and Honduras. Technical Studies: a) A review of lessons for REDD that could be drawn from Mexican experience with watershed PES. Given TransLinks’ focus on relatively small, subnational voluntary schemes, its analysis of the potential for slippage and leakage was particularly relevant, as was its key finding that “avoided deforestation is best accounted for at a regional or national level”; b) a substantial volume of lessons on land tenure, forest governance, and REDD presenting nine case studies across six countries; c) a substantial report on National REDD Funding Frameworks looking at six countries; d) a paper on PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 16 nested approaches to REDD; and e) Legal Frameworks for REDD, an IUCN paper based on case studies in Brazil, Cameroon, Guyana, and Papua New Guinea. Technical Workshops/Seminars: Events in the United States on carbon markets, conservation and poverty, and land tenure and forest carbon management, as well as a training course on land change modelling for REDD. Articles: A set of six published articles on land tenure, forest carbon, and governance to be published shortly in World Development. Tools. Manuals on PES across all sectors are listed below. The RM portal shows three resources specific to REDD and forest carbon: a presentation listing a range of different resources available, principally on the Internet; a REDD Project Development Guide; and a substantial manual on social and biodiversity impact assessment for REDD+. The latter two are available in French, Spanish, and Portuguese as well as English. PES - Water Services PES has been established longer in water quality and watershed services than in other sectors. A famous early scheme was that set up by New York City in 1990 to protect its drinking water sources in the Catskill catchment. For that reason, PES for water/watershed services has provided models and identified important key lessons for other sectors like forest carbon. Activities in Developing Countries. Katoomba: The workshops in Brazil, Ghana, and Tanzania included one session on PES for water and marine ecosystems. Training: A workshop in Rwanda offered specialists from 15 African countries an opportunity to discuss watershed modelling and management. Case Studies and Technical Analysis. Case Studies: A full study on a scheme in Gabon and short briefs on schemes in the Danube Basin, Honduras, Indonesia, South Africa, and Mexico. Technical Studies: a) A review of lessons learned from Mexico’s watershed PES schemes (already noted above for REDD); b) a subsector analysis of rainwater harvesting in Uganda; c) State of Watershed Payments, a 2008 review of watershed PES schemes worldwide; d) Beyond Carbon, a 2009 review of watershed PES in Brazil (in Portuguese); e) PES—Legal and Institutional Frameworks, an IUCN paper including reports from Brazil, Bolivia, Colombia, and Peru. Articles: One article in Land Economics (Nov 2012)—Forest Conservation and Slippage: Evidence from Mexico’s National PES Scheme. Tools. The RM portal shows no specific tools for watershed PES. Livelihoods-Biodiversity Interfaces Formal PES is only one of various strategies designed to compensate communities for conserving the ecosystems they live in. There are close analogies to forest carbon and REDD schemes where communities conserve their resources in return for payment through an internationally certified scheme PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 17 such as the Climate, Community and Biodiversity Alliance or under a direct deal with tourism operators or protected area managers. Other strategies include wildlife-friendly certification of community products, compensation for predator damage, and managed subsidies to the community. The result is that the TransLinks material grouped under this heading is quite varied. To give some clarity it is sub￾divided in this evaluation into the following three groups. PES for Biodiversity. The majority of PES for biodiversity schemes are based on arrangements that employ community members on conservation work and share tourism and safari revenues with the community. There are also examples of protected areas that share international subsidies with the community. Direct sale of biodiversity credits that are analogous to REDD is rare. Compensation for losses to predators to encourage their conservation is a special case. Case Studies and Technical Analysis. Case Studies: Five full case studies in Guatemala, Indonesia, Tanzania, and Cambodia, and a two-page brief on a major protected area in Uganda. Technical Studies: a) Two full briefs on biodiversity monitoring to support incentive programs and on monitoring/enforcing PES; and b) a review of human wildlife conflict compensation/insurance. Articles: An article on compensation schemes for livestock losses to wolves. Wildlife-Friendly Certification. By winning markets and higher prices for its products, wildlife-friendly certification aims to compensate the community for conserving its ecosystem. This aspect received relatively more attention from TransLinks. Activities in Developing Countries. Workshops: Value chain workshops in Tanzania and the Philippines and a follow-up workshop in the Philippines on the rattan value chain. Case Studies and Technical Analysis. Case Studies: A full study on elephant pepper in Zimbabwe and a set of seven brochures on enterprises in Zimbabwe, Nepal, Zambia, Cambodia, Madagascar, Ecuador, and Kenya. Technical Studies: A full brief on market finance for biodiversity conservation. Technical Workshops/Seminars: TransLinks partners presented at a USAID Biodiversity and Forestry seminar and attended an IUCN conference in Barcelona. Articles: An article on strategic tradeoffs for wildlife-friendly eco-labels. Tools. The RM portal presents five substantial guidelines on wildlife-friendly enterprise: a) Conservation Marketing Equation: A Manual for Conservation and Development Professionals (also in Arabic); b) Green Marketing Trends—Guidance for Wildlife-Friendly Products; c) Web-based Marketing—How to Find a Market for Wildlife-Friendly Products; d) Product Marketing Chains for Conservation; and e) Guidance on Facilitating a Value Chain Actors’ Field Trip. The Livelihoods-Wildlife Interface. Some TransLinks materials look more widely at the interaction between community livelihoods and neighboring ecosystems and at community participation in NR management. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 18 Case Studies and Technical Analysis. Case Studies: Full studies on protected areas in Laos, DRC, Uganda, and Bolivia; participatory wildlife assessment in Mongolia; poultry production to substitute for bushmeat consumption in Tanzania; and community-based forest management in Tanzania. Technical Studies: a) An overview paper on human-wildlife conflict; and b) a tenure brief on collaborative land use planning in protected areas. Articles: An article on compensation schemes for livestock losses to wolves. PES - Cross-Cutting Issues TransLinks produced a substantial body of work on the broader principles of PES. Activities in Developing Countries. Workshops: Early in the TransLinks program a workshop was held in the Philippines on Nature, Wealth, and Power. A later workshop in Honduras was on compensation and payments for ecosystem services. Training: TransLinks supported a training event in South Africa on forests, climate change, and ecosystem services. Case Studies and Technical Analysis. Case Studies: One case study used a game-playing approach to test ideas about institutional controls on common property resources (CPR) in Cambodia. Technical Studies: Study topics included a) pro-poor rewards for ecosystem services in the United States; b) ecological complexity and CPR management; c) gender and water governance; d) incentive￾compatible institutional design; e) human rights in conservation; and f) tenure and property rights in PES for forest conservation. Technical Workshops/Seminars: Events in the United States covered: a) designing pro-poor rewards for ecosystem services; b) bundling and stacking for PES; and c) health as an ecosystem service. Training: Two courses were held at Columbia University, NY, on the ecological foundations of PES and on ecosystem services for conservation and poverty reduction. The first of these was a graduate seminar intended to support a major review of the science underlying PES schemes. Although a paper is shown in the final technical report, it is not available on the RM portal. Tools. PES tools presented on the RM portal include: a) Ecosystem Services: A Primer; b) PES: Getting Started; c) PES: Getting Started in Marine/Coastal Ecosystems; d) PES: A Training Resource Matrix; and e) A Guideline for Livelihoods Surveys for Conservation. Outliers A handful of TransLinks documents do not fit into the above categories. Three are journal articles on pastoralism and farmer-herder conflict in West Africa and a “New Pastoral Development Paradigm.” Pastoralism is not an issue that falls into the PES model nor does it touch closely on forest or biodiversity conservation and resilience. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 19 A fourth paper considers how far intensive crop production should be allowed to expand into the undeveloped steppe lands of eastern Mongolia. While this is clearly a major policy question, it does not link closely to TransLinks’ focus on PES. One of the TransLinks partners, the Earth Institute, is a major stakeholder in the Millennium Village Project (MVP) and efforts were made through TransLinks to give greater emphasis to environmental issues in the MVPs. However these issues have more to do with questions of erosion and soil nutrient cycles in areas of relatively high populations and intensive agriculture. These are largely unrelated to what became the core the TransLinks concern with places where small populations live in or beside areas of high biodiversity and relatively unchanged natural resources. It is understood that work in these areas was started before it became clear toward the end of the first year that TransLinks would focus on PES and tenure/property rights. Dissemination, Communication, and Advocacy It is well established that knowledge generation, however important or new the knowledge, will have limited impact if it is not supported by a robust communication strategy. At a minimum, it is essential to identify different target audiences and present new knowledge in formats that are tailored to the needs and capacities of each audience before dissemination. Many knowledge-generation programs go further and include a separate communications strategy. In its most intensive form, this will: a) include a policy advocacy function; b) require that each knowledge-generation activity includes a policy-uptake plan in the initial design; and c) actively promote program outputs through mailshots, policy briefs, and newsletters. Such an approach requires a separate budget and central direction for the communication function. TransLinks lacked such a strategy, nor did it fully recognize how the early program revision changed the target audience and made it necessary to pay more attention to communication. Before the revision, a key part of the audience was to be farmers, civil society, government, and USAID missions in the four to six research site countries. Communication with this audience would have occurred naturally through their involvement in TransLinks activities. By implication, the revision replaced that in-country audience with a broader, more global audience of NRM stakeholders. Ensuring that TransLinks would be fully heard among the many voices competing for attention with that audience required a focused communications effort. Although they were not working to a specific communications strategy, TransLinks partners were well aware of the need to produce high-quality material suited to different audiences. Products ranged from posters at international conferences to high-quality primers and manuals. Less successful formats such as the two-page case study briefs were dropped. The program included important advocacy efforts, most notably the series of Katoomba workshops. As discussed in the next section, these had real impact. The TransLinks web page on USAID’s Resource Management portal played a central role in providing access to the program’s knowledge outputs. Unfortunately, it is not possible to assess its outreach, as reliable statistics on views and downloads are not available. In an e-survey carried out for this evaluation, half of the respondents had used products available on the portal. The survey was small and directed at a group that would have been expected to know about TransLinks. It gives little indication of outreach to the wider community. Few of the external key informants spoken to seemed aware of the portal, and TransLinks partners reported they mostly used it as a reference resource. Although it is well presented and fronted by an attractive introductory video, the web page does not give casual visitors the guidance PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 20 they would need to find the one or two documents that could match their requirement out of a total archive of 200 or more. Overall, it was the TransLinks partners themselves that provided the most effective dissemination channel for the knowledge generated by the program, whether through their own programs, through participation in international conferences, through their staff, or through their own websites. Knowledge Uptake For a knowledge-generation program like TransLinks, uptake of knowledge products is a key indicator and a critical first step in the causal chain toward the achievement of development outcomes. Four sets of data have been analyzed to measure uptake:  Statistics of traffic on the TransLinks website  Internet data on citations of articles produced by TransLinks  Key informant assessments of TransLinks outputs  Responses to an e-survey question on TransLinks outputs Website Analytics. The evaluator was provided with data on views of TransLinks material recorded on the USAID RM portal between June 2012 and December 2013. This period starts approximately at the end of TransLinks, which means it should reflect a more lasting interest, not just activity generated by the program itself, and it covers the top 30 documents by number of views. Table 2 shows TransLinks website analytics for several documents. Statistics from TransLinks partners’ web pages are not included. Table 2 TransLinks Website Analytics Document Views (since June 2012) Comments TransLinks RM Portal Specialty Homepage 154,307 Finding the Linkages between Wildlife Management and Household Food Consumption in the Uplands of Lao People’s Democratic Republic: A Case Study from the Nam Et-Phou Louey National Protected Area (LAO) 27,428 This document is only found on the RM Portal, hence the relatively larger number of views Conservation Marketing Equation: A Manual for Conservation and Development Professionals - ARABIC 12,937 This document is only found on the RM Portal, hence the relatively larger number of views Final Document: Nested Approaches to REDD+ 12,444 Lessons on Land Tenure, Forest Governance and REDD+: Case Studies from Africa, Asia, and Latin America 11,715 PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 21 Document dissemination was not tracked in an organized way. Since many of these publications were posted prior to June 2012, those views are not included in the above data. A more strategic communications and marketing plan would have supported dissemination efforts. This result reflects the conclusion noted above: TransLinks’ main channel for dissemination was through its partners, who present the more important program outputs on their own websites and do not frequently refer to the RM portal. Citations of Journal Articles. The TransLinks Final Technical Report lists 10 articles, of which six have been published and one is in press. The RM portal shows a further seven earlier articles that have already been published with support from TransLinks. Perhaps the most important of these, a set of six on land tenure and forest carbon, have only just been published in a dedicated issue of World Development.3 These have already been cited between three and 10 times. Of the seven older articles on a range of topics, citations range from three to 40. Conclusion The early change in TransLinks’ strategy had a direct effect on the way it generated knowledge. Instead of gathering new evidence through “site-based applied research and surveys”4 in four to six countries, it harvested existing evidence from a broader range of sites and wrote it up to make it accessible to national and global stakeholders. With a growing body of field experience in PES and related approaches, and given the inevitable tendency of implementers not to make their experience available to the wider community, this change of strategy offered a rapid and economical way to put a solid body of new knowledge into the public domain. As explained in the SOW for this evaluation, this approach was reflected in a “purposeful” decision to keep the use of core USAID funds for the development of specific projects to a minimum. (To some extent, the second change of TransLinks strategy reversed this shift, when the introduction of the SFCMI included support for the preparation of project design documents in five countries.) Hindsight suggests that TransLinks could have made more of the evidence it harvested by:  Being more selective of the case studies it pursued  Targeting field research to deepen understanding at key case study sites  Giving more time to synthesis and analysis of evidence  Taking a more strategic approach to communication It can be argued that it was more important to make a large body of evidence accessible than it was to deepen the analysis and understanding. However, as discussed in the next section, it is clear that TransLinks has had its greatest impact where it has put together a critical mass of evidence and supported it with robust analysis and advocacy to put the results into the policy arena. The strong analysis of how land tenure affects the design of REDD+ projects is the outstanding example. 3 Volume 55, March 2014. 4 Program Description, p. 17. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 22 3. TRANSLINKS OUTCOMES This section sets out the evidence relating to the following evaluation questions: Broader Program Dissemination Results Questions: How LTRM or biodiversity conservation approaches were used:  Have these case studies affected the development of the REDD+ mechanism? If so, how?  Have these case studies affected the development of innovations in PES schemes? If so, how? At no stage in its evolution did TransLinks have a defined set of specific development objectives or outcomes. Neither the original RFA nor the Program Description annexed to the grant award included a logical framework or an equivalent model. An implicit, foreshortened theory of change can be derived from the Program Description as follows: Impact: Effective NRM for equitable and sustainable wealth generation Outcomes: - unspecified - Outputs: New Knowledge and Evidence, New Technical Solutions, Decision-Support Tools, Capacity Building The assumption is clear: Outputs would generate outcomes of value without any need to consider how or to define the kind of intended outcome. That assumption is carried through to the performance management plan and to the TransLinks Final Technical Report. Of the four indicators in the PMP, only one can be considered an outcome: changes in the implementation of policies, laws, and regulations. The technical report presents TransLinks’ accomplishments entirely in terms of knowledge products, i.e., outputs, with no consideration of the extent to which those products might have influenced NRM in practice. TransLinks’ emphasis on knowledge products was, perhaps, in line with the mandate of its sponsoring office. As a central technical bureau, LTRM’s principal role is to support USAID missions and host country governments with technical advice and backstopping for their NRM programs. It has also been pointed out that there is no objection in principle to a program that only focuses on delivering high￾quality knowledge outputs. This kind of program avoids an important risk: that researchers and implementers could add bias to their work, usually without realizing it, in their effort to deliver policy￾relevant or attractive outputs. Added to which, implementers and researchers are not always the best￾equipped to communicate the knowledge they generate to policy makers and other practitioners. The ultimate value of TransLinks must depend on the extent to which its outputs are taken up and lead to clear development outcomes through their influence on how natural resources are managed in practice. The evaluators were asked by USAID to use the principles of outcome harvesting (OH) to identify such outcomes, defined as “a change in the behavior, relationships, actions, activities, policies, or practices of an individual, group, community, organization, or institution.” With limited time available, it was not possible to implement a full OH approach.5 Instead, the evaluation used two tools structured on OH principles. The Chief of Party for the WCS consortium was asked to coordinate a response to an outcomes checklist. This linked two stages in the OH process: drafting outcome descriptions and engaging with change agents to formulate outcome descriptions. Appendix 2 presents the outcome checklist filled in by the WCS Consortium. The second 5 Outcome harvesting was not a requirement of IBTCI’s SOW. Appendix 6 presents the TransLinks evaluation methodology, which was drafted to integrate OH into the original methodology. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 23 tool was a key informant’s checklist, presented in Appendix 3 which sought KI assessment of possible outcomes and TransLinks’ contribution to those outcomes. It was hoped that this would give some degree of independent substantiation to the OH. The checklist asked KIs to consider possible TransLinks outcomes at three levels: 1. Global/international: Influence on, for example, international negotiations over REDD+ or on donor support for the forest carbon initiatives 2. National: Influence on national policy and practice in REDD+, PES, etc. 3. Subnational: Community progress with individual PES and other community NRM schemes Three important challenges must be noted. First, the TransLinks work was long term in nature and it may be some years before clear outcomes can be confirmed. Second, attribution of knowledge outcomes is particularly difficult in active fields. More often than not, progress results from the work of several different programs and it is not possible to separate the contribution made by each one. Finally, the evaluators found it difficult to identify external stakeholders who felt they knew enough about TransLinks to respond fully to the KI checklist. Despite these challenges, the evaluators are confident that the following represents a reasoned assessment of TransLinks’ outcomes. Nevertheless, stronger substantiation would make the assessment considerably more robust. Global/International Outcomes TransLinks did not make a targeted effort to influence global developments. Nevertheless, there were some activities at this level. One important example was TransLinks work with Center for International Forestry Research (CIFOR) to raise international awareness of the land tenure dimension of REDD projects at a side event to COP 17 in Durban, South Africa. TransLinks also supported the presentation of wildlife-friendly enterprises at an IUCN conference in Barcelona. Helping individual nations prepare their positions for participation in global meetings was a second channel of influence. TransLinks partners worked with government and civil society in Ghana to help them engage more effectively in international negotiations. In Brazil, several state governors attended a Katoomba meeting supported by TransLinks. It is understood that Brazil’s position at the COP 15 in Copenhagen reflected a letter they subsequently wrote to the President. The United States has a powerful voice in international fora and the TransLinks partners have a recognized position in the U.S. community. Strengthening the body of rigorous, well-argued evidence available to U.S. policy makers must have had an influence, most particularly for USAID’s own position. A clear example is the way TransLinks made it possible for the United States to respond effectively in 2009 to increased global interest in REDD+. More recently, USAID’s 2013 publication Nature, Wealth and Power 2.06 draws extensively on the evidence pulled together by TransLinks and the knowledge arising from it. TransLinks contribution to outcomes at this level is not measurable. It was one among many voices contributing to the discussion and many of those voices were more powerful and better funded. Nevertheless, few have been able to offer a well-argued case based on solid evidence from field experience. TransLinks may have had a greater impact than has been recognized. 6 Nature, Wealth, and Power 2.0: Leveraging Natural and Social Capital for Resilient Development. USAID, October 2013. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 24 PES - REDD TransLinks’ work in the forest carbon sector had the greatest focus on outcomes, particularly at the national and sub-national level. A number of results can be identified, although most can best be described as incipient or “pipeline” outcomes in that most remain short of a final, sustainable result. National and Sub-National Outcomes. TransLinks’ support contributed to the preparation of six projects for the sale of forest carbon credits in voluntary markets, i.e., markets in which credits are sold to private buyers, mainly corporations seeking to offset their own carbon emissions. While international negotiations on national programs remained stalled, the hope was that work with sub-national, voluntary schemes would maintain momentum behind the development of REDD+. Three key milestones mark the progress of a forest carbon scheme: preparation of a PDD; certification of the credit by one of the recognized registries for carbon credits; and, finally, sale of the credit. The first and most advanced of the projects supported by TransLinks was at Makira Forest in Madagascar. This has been certified by both VCS (Verified Carbon Standard) and by CCBA (Climate, Community and Biodiversity Alliance), with some 33 million tons of carbon credits for sale over the next 30 years. TransLinks supported the certification process, although WCS had begun work earlier. Winning the support of Government of Madagascar was critical to the success of this scheme, especially its agreement that the benefits be shared between the community, the management of the protected area, and the government. This was before the political unrest of 2009 and it is not clear how things will proceed. Of the five other schemes supported by TransLinks, the greatest impact was in Brazil where the Suruí indigenous community succeeded in registering its forest carbon project with VCS and CCBA, with an estimated emissions reduction of 247,460 tons a year (7.4 million tons over 30 years). The Suruí project was part of a Katoomba process which greatly raised the profile of REDD+ in Brazil. To some extent it is TransLinks’ flagship project. Even so, the TransLinks Final Technical Report describes how it still faces significant challenges from illegal logging, divisions in the Suruí community, and uncertainty over the regulatory framework. At the time of the final technical report, four other schemes were in full development: Seima Forest, Cambodia; Takamanda-Mone, Cameroon; Maya Reserve, Guatemala; and Dolakha, Gorhkha and Chitwan, Nepal. Critically all four had engaged successfully with governments and made progress in winning their support. In Cambodia, for example, the Seima Forest project is under validation as an official national demonstration model.7 TransLinks partners report how the program’s research, its advocacy through Katoomba and otherwise, and its support to the development of individual forest carbon schemes has influenced national policies. The Government of Madagascar has accepted that forest carbon benefits should be shared with the community and the Government of Brazil has recognized the principle that indigenous communities have rights in sales of forest carbon credits. Although the Government of Cambodia is reserving it position on this key principle, it has clearly recognized that sub-national REDD schemes can be made to work. Even in countries like Ecuador, which do not yet recognize community rights in carbon, key informants believe TransLinks has helped government officials understand how tenure and property rights affect the country’s major environmental program, Socio Bosque. 7 Judging from the fact that none of the four is yet registered by VCS or CCBA, there has not been much progress since the final technical report, apart from the Seima Forest PDD, which is under validation by CCBA. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 25 Even at the national level, care is needed in attributing particular outcomes to a TransLinks contribution. In almost all countries other programs are supporting national and sub-national REDD. And, as the Ecuador example shows, all of them have environmental ministries and established forest-protection programs. The fact that Cambodia’s Royal Government of Cambodia Forestry Administration registered a forest carbon scheme with VCS in 2013 makes the point that TransLinks was not alone in the field, making it difficult to pin down what it plausibly contributed to one or other outcome with the information available. Lessons Learned and Capacity to Deliver. Lessons learned, and the improved capacity to deliver which comes from that learning, are key results to be expected from a knowledge-generation program like TransLinks. If effectively disseminated, effective use of lessons learned will not only improve the way TransLinks partners work but also contribute to improved work by others. Members of the TransLinks consortium report the following as the most important lessons learned in PES for REDD: The Process: Completing all certification requirements for the sale of carbon credits is technically challenging and, as a consequence, slow and expensive. This can be difficult for communities and even for governments to accept. Key Principles: The principles of PES apply across all sectors and the transfer of lessons learned from successful experience with forest carbon projects can open up opportunities in other areas. In the Philippines for example, TransLinks drew on its REDD work in South/Southeast Asia to brief environmental grantees of the USAID mission and open opportunities in water PES. Managing Expectations: The complexity of the process makes maintaining community interest a key challenge. Tenure and Property Rights: Issues of tenure and property rights were fundamental in all countries. At some sites, community rights to the land itself and the division of those rights among members of the community were not defined. This is especially the case for indigenous communities holding land as common property. In addition, some governments have yet to decide whether rights to the land include the right to the carbon stored on it. PES - REDD Conclusion. Arising from the last point in the preceding section, one overriding lesson is clear: that even sub-national projects hoping to sell forest carbon in voluntary markets cannot succeed if the institutional and policy framework is not ready. As experience with the Suruí project has shown that even in countries like Brazil, where there has been significant progress, much remains to do. For that reason, progress toward completing the framework for REDD and forest carbon management will be the most important of all TransLinks outcomes. Conversely, if those frameworks are not put in place, most if not all TransLinks outcomes in PES for REDD will remain potential, not achieved. There is one other major barrier to the realization of these potential outcomes. Since before the end of TransLinks, international progress on the development of carbon markets has stalled. As nations fail to reach agreement at the inter-governmental level, potential private buyers of carbon credits are holding back on purchases and prices are falling. PES - Water Services TransLinks’ work on water PES was not at the same scale as for forest carbon. Only one national project was supported, in the catchment of the Mbé River, Gabon. Here TransLinks support to essential pilot work helped win GEF funds to bring the scheme to market. The evaluator was unable to establish PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 26 whether this scheme is operational, although it is understood that government wishes to proceed. GEF documentation makes it clear that it faces the same barrier as the REDD schemes discussed above: “The legislative and policy foundation for supporting financial payments to providers of ecosystem services is missing.” Until these barriers are overcome, TransLinks can contribute to few final outcomes. International registries provide a standard approach to the measurement and certification of forest carbon and international markets offer a standard price. Watershed services are more diverse in several respects. Potential buyers include both water and hydro-electricity utilities. The first is seeking pure water. For the second, reduced sediment loads are more important. Prices reflect local markets and the local balance of supply and demand; local political pressures are particularly strong when it comes to utility pricing. Measurement is challenging, reflecting the complex ecology, hydrology, and geology of each individual catchment. These factors have been reflected in TransLinks’ lessons learned:  Pricing: In Gabon, government policy for cross-subsidizing water and electricity prices tended to undermine the PES scheme.  Measurement of the Service: As the textbox from the GEF document emphasizes, measurement is critical. Yet TransLinks case studies as far apart as Tanzania and Indonesia have shown that it is extremely challenging in practice.  The Experiential Barrier: This phrase drawn from GEF sums up the need to inform all stakeholders on how PES works. It is, for example, “a very new approach to mobilizing finance for conservation in Gabon. Key stakeholders are unfamiliar with the potential of such innovative mechanisms.” Livelihoods-Biodiversity Interfaces As already discussed, TransLinks recorded experience from a diverse range of schemes to give communities an incentive to sustainably manage their natural resources. At some sites it also provided direct support for change. TransLinks partners reported the following significant outcomes. As part of a set of interventions in Cambodia, WCS is supporting community production of Ibis Rice. Wildlife-friendly certification of the rice wins a price premium, conditional on the community maintaining habitat for ibis and other large birds. WCS reports that TransLinks funding was critical in taking Ibis Rice to scale and allowing new producers to be certified. In the Philippines, TransLinks has supported two workshops and a process to engage value chain actors in non-timber forest products industries in analyzing how to manage the source forests sustainably. Starting with rattan from Palawan, the principles of sustainable management are now being extended by Philippine value chain actors to new sites and new products. A rather different program in the eastern steppe of Mongolia worked to make herder communities aware of conservation principles. TransLinks funded workshops to understand herder perceptions of biodiversity and conservation. An approach to establishing herder groups for NRM was developed and is now standard practice for the Mongolian authorities. Measuring Water Services “Up to now, the… value of ecosystem services in the watershed remains unknown. Without a thorough assessment of these values, a mechanism based on payments for their maintenance will be difficult to put in place, as M&E and verification systems, whose indicators rely on these values, are key tools for the establishment of trust between the parties involved in a PES scheme.” -- GEF Mbé Gabon Project Document PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 27 Key lessons from this diverse experience are also varied:  Design: No single structure of conservation incentives is a silver bullet. While in some situations payments to individuals can be effective, engaging the full community seems to work better, especially when it comes to managing free-rider issues. However, even communities may struggle to control poaching and other encroachments by outsiders.  Demand and Markets: Consumer demand for conservation products—e.g., birding—can be a limiting factor. While there is strong demand for eco-products, especially in developed country markets, the eco-label space is competitive, even crowded.  Access to Benefits: Can create inter-communal tensions (e.g., between men and women in Isoso, Bolivia) TransLinks Partner Programs and Capacities The members of the WCS consortium were an important channel for the dissemination of the knowledge TransLinks generated. In the same way, the consortium members’ increased commitment to and capacity for promoting workable PES is an important program outcome. WCS itself is the most important example. It reported that it was experienced with TransLinks, which convinced its board to make a much increased commitment under a new Ecosystem Services Program. From four forest carbon projects under TransLinks, the portfolio has expanded to 15. The Earth Institute’s work on the scientific underpinnings for PES is also continuing under a grant from the Keck Foundation. Consortium partners are carrying their increased capacities and knowledge forward into new work with large international programs such as the U.K.’s Ecosystem Services for Poverty Alleviation (ESPA) and Canada’s Ecosystems and Human Health. In the conservation enterprise sector, TransLinks support was critical to the establishment of the Wildlife Friendly Enterprise Network. This U.S.-based not-for-profit is being supported by a range of foundations and USAID to help put small producers from 12 developing countries in touch with U.S. demand for conservation-friendly products. It has been described as the first global certification network working to support and document how local communities develop find livelihoods that can take them out of poverty and yet coexist with wildlife and protect the habitat the wildlife needs to survive.8 Key Informant and e-Survey Responses Stakeholder Assessment of TransLinks Outputs and Outcomes. Telephone interviews with key informants, combined with an e-survey of a wider group, were central elements of the evaluation methodology. The objective was to seek stakeholder assessments of TransLinks outputs and outcomes. To do this both the KI checklist and the e-survey were structured to draw out their views on the following: a) TransLinks outputs under by categories: case studies, briefs, tools, workshops, seminars b) TransLinks outcomes at: 8 See http://wildlifefriendly.org/ PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 28 i. The global/international level: negotiations on REDD; commitments to conservation; donor funding levels; etc. ii. The national level, in terms of changes in policy and implementation iii. Individual sub-national sites c) TransLinks contribution to new knowledge and methods in each of the following: PES for forest carbon, PES for water services, and community management of biodiverse natural resources The KI checklist, which was also used for the e-survey questionnaire, is found in Appendix 3. Key Informant Interviews. On June 18, 2012, TransLinks held the Results Symposium: “Promoting Transformation by Linking Nature, Wealth, and Power.” The invitation list included representatives of a broad range of organizations including the World Bank; major philanthropic foundations such as Packard and Moore; universities; and NGOs specializing in conservation: WWF, TNC, IRG, RARE, etc. This offered a good spread of well-informed stakeholders who could be expected to be knowledgeable about TransLinks. To add to this list, the five members of the WCS consortium were asked to suggest other names to be interviewed. The evaluator also added names from UN-REDD as a key stakeholder in a major aspect of TransLinks’ work, and one informant known personally who had contributed to the original work on Nature, Wealth, and Power. (A list of those invited to take part is provided at Appendix 1.) The consortium partners were not able to suggest many new names and the response from the main list of symposium invitees was very low. Of 42 external stakeholders, 23 did not reply and eight declined the request for an interview. In all but one case this was because the respondent did not feel he or she knew enough about TransLinks to respond adequately.9 The result was that the evaluator was only able to talk to six stakeholders who were external to the WCS consortium and USAID. These fell into three groups. Two KIs had worked closely with TransLinks and knew particular aspects of the program well. Three work for conservation organizations with similar interests to the members of the WCS consortium. The last informant is a manager of a major program of research into ecosystem services funded by European donors. The two KIs who had worked with TransLinks were strongly supportive. One credited the program with much of the development to her organization. The second, speaking from a Latin American country, had a more nuanced view. He/she felt that the work of TransLinks on land tenure for PES and REDD had not led to any changes in policy but had “advanced the conversation” with government officials on an issue that affects a major national environmental program. The work on wildlife-friendly enterprise is less successful. If anything, international interest in biodiversity conservation is falling, according to this key informant. Despite these difficulties, this informant felt that the concept of PES came to life during TransLinks and that the program had “contributed an incredible amount to PES as an on-the-ground model.” The KIs from other conservation organizations were most aware of TransLinks where it had overlapped with their own work and through individual members of the team who they know personally. They rated some technical products highly, in particular the work on land tenure and property rights and on REDD+, but none of them felt they knew enough to name specific outcomes to which TransLinks had contributed. It was, for example, disappointing that an informant from an organization with extensive experience of PES and REDD in Brazil was not aware of the flagship TransLinks project for the Suruí 9 Although the list was old, there were only three bounces from unknown addresses indicating that it was still adequately current. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 29 indigenous people. More positively, research staff in the same organization consider the TransLinks Social and Biodiversity Impact Assessment and Livelihoods Surveys for Conservation Guidelines as important tools. The organizations these informants work for have substantial PES programs, particularly for water services, highlighting the fact that TransLinks was part of a broader movement. In the same vein, one KI noted the similarity between TransLinks and the Climate and Development Knowledge Network supported by the U.K. and Norwegian governments. He could not, however, suggest examples of any synergy between the two. By virtue of his position, the last informant had a good overview of the current global position on PES and was aware of TransLinks’ work. While he felt the program had focused on outputs, it had also made contribution to outcomes, though they are difficult to capture. This was, perhaps, built into the design of the program. TransLinks had not been encouraged to think about changing behavior or about capturing outcomes. Its major results will come from follow-on work under new programs in Europe, Canada, and the United States. Many of the KIs had views on how well TransLinks had integrated Nature, Wealth, and Power. One KI summed up the consensus view by suggesting that the PES community as whole, not just TransLinks, had not been up to the challenges presented by the underlying natural science. There was a perception that the science was well-known and that governance was the only issue that mattered. Similarly, issues of wealth and power within communities managing natural resources have not been given the attention they need. The e-Survey. To draw on a wider range of views, 162 stakeholders were invited to take part in an e￾survey. At the final count, there were 28 responses. The respondents fell into three groups: developed country NGO/research institute (9); LMIC country NGO/research institute/private sector (5); USAID staff (11); and TransLinks implementing partner (3). Of the survey takers, only 50% considered themselves “aware observers” of TransLinks’ work. Of the remainder, four were implementing partners and six participants, i.e., from organizations that had worked with TransLinks. A majority (18 of 28) said that they were interested in all aspects of NRM: REDD+, PES for water services, wildlife and biodiversity, and community-based natural resource management (CBNRM). Perhaps surprisingly the largest single interest was in the last of these: 22 of 28 with an interest in CBNRM. The following presents a summary of the responses to the survey. The full output is provided in Appendix 4. 1. TransLinks Outputs Just over half had used TransLinks material shown on the RM portal. They rated the material highly— “very useful” or “somewhat useful”—and there were no negative responses. TransLinks tools were rated most highly, followed by briefs and case studies. The workshops and seminars were more often given the lower rating of “somewhat useful.” But these differences were not large. Asked to identify any outputs they considered “outstanding,” no single document stood out. Instead, the 10 responses identified areas of work, such as land tenure, bundling and stacking, and Katoomba. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 30 2. TransLinks Outcomes Tables 3 to 5 summarize survey responses on possible TransLinks outcomes at the global, national, and sub-national levels. A brief paragraph after each table sums up the results it presents. Table 3 Possible TransLinks Outcomes at the Global/International Level 10 Possible Outcome Responses (External) Assessment of the TransLinks Contribution 1. Greater weight given to Forest Carbon in the UNCED process. 2 (1) Possibly through Forest Trends work; the Katoomba effort in Brazil 2. Greater weight given to biodiversity in international fora. 2 (1) GEF investment in enabling PES; possible impact on UN-IPBES 3. Donors give increased support to Forest Carbon initiatives. 2 (2) $20 million USAID in Cambodia, influence on wider USAID program 4. Donors give increased support to Biodiversity conservation initiatives 2 (1) “TransLinks materials frequently shared at CTF association network meetings (e.g., REDLAC)” 5. Better methods for the design and implementation of projects to trade forest carbon in voluntary markets. 5 (1) “Manuals have greatly influenced discussions about social safeguards for REDD” 6. Better methods for establishing Payment for Ecosystem Services. 6 (1) “One of the best archives of PES information” 7. Better methods for enabling communities to manage the interface between their livelihoods and neighboring areas of high biodiversity. 4 (1) “Helped our community in the province of Kalinga to engage in non-timber microenterprise using training facilitated by TransLinks” 8. Other (please add to the list if appropriate). 3 (1) “I represented the Philippines at COP 7, 8, 9 & 10, [and spoke] on topics I acquired during TransLinks training”. U. of California-Davis funded to work on health as an eco-system service after TransLinks workshop Table 4 Possible TransLinks Outcomes at National Level Outcome Responses (External) Assessment of the TransLinks Contribution 1. Better national policies for REDD+ implementation 4 (0) TransLinks legal study prepared way for Brazil 2012 law recognizing indigenous community rights in forest carbon 2. Better national policies for PES. 3 (1) “Increased Conservation Trust Fund capacity to engage in PES policy, e.g., Colombia. PES pilots helped shape national thinking.” 3. Better national policies for Community NR Management. 2 (0) Refers back to REDD/PES 10 Note: “External” respondents not directly involved in TransLinks as implementer or USAID AoR. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 31 Outcome Responses (External) Assessment of the TransLinks Contribution 4. Better national policies for Forest and Biodiversity conservation. 0 5. Other (please add to the list if appropriate). 0 Table 5 Possible TransLinks Outcomes at Sub-National Sites Outcome Responses (External) Assessment of the TransLinks Contribution 1. Sustainable deals to trade forest conservation for carbon credits. 5 (2) Outcomes described in Ch4. One external noted a number of other E. African projects, but was not aware of direct TransLinks influence. 2. Sustainable deals trading catchment conservation for water services. 4 (1) Outcomes described in Ch4 – Philippines and Gabon schemes in development. 3. Sustainable deals trading biodiversity conservation for community or private benefits. 5 (0) Outcomes described in Ch4 – especially Cambodia 4. Other (please add to the list if appropriate). 0 3. New Knowledge Generated by TransLinks The last section of the e-survey sought respondents’ views on the new knowledge TransLinks generated. Table 6 sums up the results. Table 6 New Knowledge Generated by TransLinks Outcome Responses (External) Assessment of the TransLinks Contribution 1. New concepts or new evidence about forest carbon and its better management. 1 Bundling and stacking, greater focus on biodiversity 2. New concepts/evidence about PES for water services. 3 (2) “One of the better sources of information on PES” 3. New concepts/evidence about community management of biodiverse natural resources. 5 (1) “TL funded data collection and analysis so that successes were documented and disseminated” 4. Other (please add to the list if appropriate). 1 (1) “Emerging insights about rangeland management, in N. Kenya in particular.” The e-survey response was too small to allow firm conclusions. That said, it seems to confirm TransLinks’ orientation toward outputs. A relatively large proportion of respondents have used the material made available on the RM portal and report they have found it useful. Critically, this included a number of external respondents. However, only a small minority felt they could identify outcomes to which TransLinks has contributed at any level; the majority of those who did had been directly involved in the program. There was little external substantiation, which is consistent with other evaluation results. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 32 Conclusion Measured by its budget, TransLinks was a small program. The U.K.’s ESPA alone has a budget about10 times larger, while just one of the World Bank’s many carbon funds has committed $90 million to forest and agro-ecosystem carbon sequestration. Against that scale, TransLinks’ chosen strategy of harvesting existing experience and making it accessible to stakeholders as quickly as possible seems justified. Despite its small budget, TransLinks has been able to influence and add momentum to the development of PES. In particular, it has helped:  Build momentum for PES, in particular REDD+, in several countries: Brazil, Nepal, Cambodia, Madagascar, Guatemala, Cameroon, Ecuador  Ensure that a start has been made on addressing key issues: tenure/property rights, the distribution of benefits, the high cost of certification and establishment  Create a solid body of tools and manuals, the best of which make complex issues accessible and ensure that the pluses and minuses of PES and other conservation models are fully understood In the mid-2000s, there was a great deal of excitement around the concepts of PES, especially PES for forest carbon. Some were predicting the development of huge international markets. This excitement— and the possibility for profit—drew in players sometimes described as “carbon cowboys,” some of whom were not mindful of technical complications or social impacts. At such a time the role of TransLinks as a balanced interpreter of the science base and field experience was particularly important. Even within USAID, TransLinks material helped enthusiasts to temper their enthusiasm and assisted skeptics in overcoming their doubts. Above all, TransLinks helped stakeholders at all levels start to understand how the theoretical models of PES could be made operational. As for outcomes, it should be recognized that progress in the development of PES and other conservation incentive models has been uneven and critically dependent on country will and capacity. When this is added to the continued uncertainty at the global and intergovernmental level, it is neither surprising nor a criticism to state that the larger part of TransLinks’ impact must still be measured in terms of potential rather than fully realized outcomes. PES is by its nature a mechanism with significant social implications in areas including land tenure, community institutions, and the distribution of wealth and livelihoods. The potential for negative social outcomes is significant. TransLinks focused relatively little attention to questions of gender and social exclusion. It is important to note, therefore, that it is not possible to provide a robust assessment of this aspect of TransLinks’ work. To make the point, neither of two forest carbon case studies in Madagascar and Nepal make any mention of gender and neither analyzes how forest use rights are distributed among the communities involved. 4. POTENTIAL RETURN ON THE TRANSLINKS INVESTMENT The evaluation SOW calls for an assessment of the return on USAID’s investment “where data is sufficient.” The financial records available to the evaluator are incomplete and do not show sufficient detail for any assessment of value for money. It is not, for example, possible to estimate the level of effort that has gone into each part of the program’s work, or calculate the unit rates charged for that effort. Table 7 summarizes the budget data available. There is no data for actual expenditures.11 11 It is understood that recipients of USAID awards like TransLinks are not required to present more detailed financial information. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 33 Table 7 TransLinks Budget Allocations by Activity12 Activity FY 07 FY 08 FY 09 FY 10 Total % 1. Knowledge Generation 193,775 333,552 812,435 468,271 1,808,033 33.9 2. Tool Development 56,929 119,171 222,001 225,775 623,876 11.7 3. Cross-Partner Exchange 46,953 482,169 921,946 319,687 1,770,755 33.2 4. Global Dissemination 76,814 21,896 60,721 139,755 299,186 5.6 5. Administration 90,720 144,527 199,527 129,661 564,435 10.6 - Indirect Costs 266,857 266,857 5.0 Totals 465,191 1,101,315 2,216,630 1,550,006 5,333,142 100.0 It is, however, possible to develop a set of assumptions to indicate the potential worth of TransLinks’ contribution to the development of PES. The assumptions are the following:  The principal return on the TransLinks investment will come from the six forest carbon schemes it has supported, if the schemes come to fruition.  Those schemes will sell the currently estimated quantity of carbon credits over 30 years, starting from 2016.  The sale price will be $7.5 per MtCO2e (metric tons of carbon dioxide equivalent), the 2012 average for voluntary markets.13  The net value of the credits, after subtracting implementation and opportunity costs, will be 20% of the gross.  Ten percent of that value can be attributed to TransLinks.  Including cost-share contributions, TransLinks’ $7 million cost was spread evenly between 2007 and 2011. Based on these assumptions, the return on investment (ROI) of TransLinks will be 5.1%.14 These are highly simplified assumptions. Nevertheless the calculation does indicate that if PES for forest carbon takes off, TransLinks’ effort to support it will have been well worth it, even more so if water PES and wildlife-friendly enterprise also progress. 12 This table, which refers only to USAID funds, has been constructed from the data in the Implementation Plans. It may not be correct. Complete data on matched funds is not available, nor are figures for FY 11. 13 Forest Trends: State of the Forest Carbon Market 2012. 14 The calculation is shown in the spreadsheet at Appendix 5. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 34 C. CONCLUSIONS The evaluation’s conclusions are presented below as responses to the seven evaluation questions set out in the SOW. 1. PROJECT DESIGN AND MANAGEMENT RESULTS Two questions were asked about project design and management: 1. What role did project design (and any changes/evolution in the design) play in the final results of the program? 2. What are the lessons from this partnership/consortium? The TransLinks program design was significantly changed two times. The first change was to replace pilot implementation at a limited number of developing county sites with a program to harvest experience and lessons learned through case studies at a much larger number of existing NRM interventions. At about the same time, a decision was made to focus on PES and related tenure issues. This change allowed TransLinks to make a solid body of new knowledge on PES accessible quickly and economically. The second change built on the first: With the addition of new funds, TransLinks began to concentrate its work on PES for forest carbon. This included efforts to develop projects in specific countries, moving at least some way back to the original concept of learning from pilot efforts at particular sites. The result was that TransLinks’ impact in REDD+/forest carbon substantially outweighed that in other sectors. The WCS consortium for TransLinks included three NGOs and two university institutes. Each of the five had separate specialist interests, and there were some tensions over which should have the lead in particular areas, particularly forest carbon. The different working styles of NGOs and academic organizations did not mesh entirely smoothly. It does not appear that the consortium became much more than the sum of its individual parts. From the beginning, TransLinks was not designed to work toward a specific set of outcomes. It had no logical framework and no theory of change. The first change in design left the strategic direction even less clear. At the same time the decision was taken not to set up the planned governance structure of an Advisory Board and Executive Team meeting regularly. Although consortium tensions are far from unusual, they become more difficult to manage without a clear strategy and a controlling governance structure. Without diminishing what it did do, a clearer strategy and stronger governance would have helped TransLinks achieve more. 2. PERFORMANCE OUTCOMES 1. How did WCS/Forest Trends use the lessons learned that they produced internally to influence their (or their international partners’) programs? 2. To what extent was new NRM knowledge generated from this work? (i.e., new peer-reviewed articles, new grey literature produced, new lines of research questions explored.) TransLinks has enabled members of the TransLinks consortium—especially the three practitioner NGOs—to significantly strengthen their capabilities in PES, particularly for forest carbon. WCS has PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 35 added an Ecosystem Services Program to its global initiatives and Forest Trends was able to extend the Katoomba process to five new countries. It is understood that their programs reflect a stronger understanding of what it takes to make PES operational, an understanding which they are sharing with partners in government and civil society. A substantial body of new knowledge was developed around the issue of land tenure and property rights in PES schemes. A forthcoming special edition of World Development on Land Tenure and Forest Carbon Management comprises six articles prepared with TransLinks support. Even prior to publication these articles are being cited. Other important areas explored by TransLinks included slippage and leakage in the conservation benefits from PES schemes, health as an ecosystem service, ways to “bundle and stack” mixes of ecosystem services, and the natural science foundations of PES. 3. PROGRAM DISSEMINATION RESULTS 1. From TransLinks results, what were the key determinants of success in documenting and disseminating the results of successful NRM tools for greater adoption? 2. Have (and if so, how) these case studies affected the development of the REDD+ mechanisms? 3. Have (and if so, how) these case studies affected the development of innovations in PES schemes? TransLinks did not have a well-defined communications strategy. The assessment suggests that such a strategy would have helped TransLinks make more of the evidence it harvested and of the strong body of material it put together. As a consequence, members of the TransLinks consortium provided the principal dissemination channel for the program’s results. TransLinks had its greatest impact where its knowledge generation, project development, and advocacy work came together to support the formation and implementation of policy in a single country. The clearest example is Brazil, where targeted research on legal and institutional issues, a major Katoomba meeting of stakeholders, and a successful pilot REDD project made significant progress in establishing the principles for PES schemes owned by indigenous communities. In Brazil and several other countries, TransLinks helped demonstrate that the theoretical models of REDD+ can be made operational for the sale of carbon credits from sub-national sites in voluntary markets. When the international REDD process regains its momentum, those countries will be well placed to develop their programs. In at least one case, it has been possible to use progress in one country to show policy makers and practitioners from another country how they might proceed. TransLinks’ work on PES has played a similar role. The focus has not been on innovation so much as it has been on showing how PES can be implemented in practice. Its most important contribution has been to ensure that the critical question of tenure and property rights does not get overlooked in implementation. Table 8 presents major outcomes identified by members of the TransLinks consortium to sum up how TransLinks has affected the development of REDD and PES. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page 36 Table 8 Summary of TransLinks Outcomes Field Site Status Update since TransLinks Ended Assessment of the TransLinks Contribution FOREST CARBON 1. Makira Forest – Madagascar 32.5 million tons of carbon credits for sale. Certified by VCS and CCBA. First ever government-backed sale of in Africa. Helped win certification from VCS and CCBA, won agreement with GOM over distribution of benefits. 2. Dolakha, Gorhkha & Chitwan – Nepal 919,500 million tons of carbon credits for sale. REDD Project Design Document ready, pending government policy. Assisted preparation of PDD, leveraged Nepal experience to brief Philippines government and civil society on REDD approaches. 3. Seima Forest – Cambodia 51.2 million tons of carbon credits for sale. First verification report due. The official national demonstration site. Helped show government REDD can work and to understand the effort needed. 4. Guatecarbon – Guatemala 24.0 million tons of carbon credits for sale. PDD ready. Assisted preparation of PDD. 5. Takamanda Mone – Cameroon 8.2 million tons of carbon credits for sale. PDD ready. Assisted preparation of PDD. 6. Surui People – Brazil 7.4 million tons of carbon credits for sale. Certified by VCS and CCBA. Helped win certification and GOB support for indigenous rights in forest carbon. WATERSHED SERVICES 1. Mbé Catchment – Gabon GEF funding is taking PES scheme forward. Supported pilot work to win GEF funding. WILDLIFE-FRIENDLY ENTERPRISE/BIODIVERSITY 1. Elephant Pepper – Zimbabwe Elephant pepper now selling in South Africa and U.S. Supported establishment of global Wildlife Friendly Enterprise Network. 2. Ibis Rice – Cambodia Certification expanding to new producers. Ibis Rice is purchased by major hotels. Critical support for getting Ibis Rice to scale. 3. Rattan Furniture – Philippines Rattan value chain model extending to other NTFPs in the Philippines. Engaging value chain members in sustainable harvest practices for high biodiversity forest. COMMUNITY-BASED NATURAL RESOURCE MANAGEMENT 1. Okapi Reserve – DR Congo Community zoning work a model valid for other CARPE landscapes. Supported research to complete study. 2. Eastern Steppe – Mongolia Community Conservation Association working to establish and train herder community groups. Demonstrated how to work with herder perceptions of wildlife abundance, wildlife use, and natural resource management capacity. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page i APPENDIX 1. KEY INFORMANTS CONTACTED BY THE TRANSLINKS EVALUATION Name Organization Response/Interview Date The WCS Consortium David Wilkie (TransLinks CoP) WCS 26/11/13 Michael Jenkins Forest Trends 2/12/13 Shaheed Naim Earth Institute 4/12/13 Lisa Naughton Nelson Inst./LTC 9/1/14 Ann Koontz Enterprise Works Ray Victurine WCS 13/1/14 USAID Staff Michael Colby AoR TransLinks 21/11/13 Diane Russel Asst AoR TransLinks 29/11/13 Megan Hill CoR TransLinks evaluation 21/11/13 External Stakeholders Peter Riggs Ford Foundation Bounce Chuck Bennett Aveda Bounce Paul van Gardingen ESPA 8/1/14 Kai Lee Packard Foundation Declined Guillermo Castilleja Moore Foundation No reply Jorgen Thomsen MacArthur Foundation No reply Peter Veit WRI No reply Bob Winterbottom WRI No reply Michael Mascia WWF Declined Robin Naidoo WWF Declined Amy Rosenthal WWF No reply Gretchen Daily Stanford University No reply Steve Polarsky University of Minnesota No reply Taylor Ricketts University of Vermont No reply Peter Kareiva TNC No reply Jim Rieger TNC 20/12/13 Jessica Musengezi TNC 20/12/13 Heather Tallis Stanford University Bounce Agi Kiss World Bank No reply Stefano Pagiola World Bank No reply Eva Garen Conservation Intl No reply Keith Alger RARE 7/1/14 Rodrigo Martinez RARE Missed call Herman Daly Univ. Maryland Declined Andy White Rights and Resources No reply Theo Dillaha Virginia Tech No reply Scott Hajost FCMC/TetraTechARD No reply Gabriel Thuomi FCMC/TerraGlobal Capital No reply PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page ii Name Organization Response/Interview Date Asif Shaikh IRG No reply Ann Lewandowski IRG Declined Rosa Andolfato UN-REDD No reply Denise Martinez Breto UN-REDD No reply Tim Clairs UN-REDD Declined Julie Greenwalt UN-REDD No reply Catherine Schloegel Fundación Cordillera Tropical, Bolivia 31/1/14 William Sunderlin CIFOR No reply Bruce Campbell CGIAR Declined Manolo Morales Ecolex, Ecuador No reply John Bruce Declined Julie Stein WFEN 7/1/4 Bhishma Subedi Nepal No reply PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page iii APPENDIX 2. OUTCOMES CHECKLIST Completed by WCS TransLinks Consortium 1. Introduction This checklist is to set a framework for identifying TransLinks outcomes. It groups those outcomes into four areas:  PES for forest carbon services  PES for watershed services  Wildlife/biodiversity-friendly enterprise  Community NR management The aim of the checklist is to identify the outcomes of TransLinks’s work in the field on case studies, PDD development, and other areas. To keep the evaluation manageable, only three field sites are identified under each heading. The ones shown are the evaluator’s suggestion. If there is a consensus that other sites are more representative or more informative, they may be substituted as long as the number reviewed does not increase. A status update has been sought for each site, principally to gain an understanding of how TransLinks’ involvement has contributed to continued progress and a sustainable outcome. Two additional questions are asked under each heading:  What three lessons have been learned from the TransLinks experience?  What are three examples of new knowledge generated or disseminated by TransLinks that has been taken up in policy or practice? 2. PES for Forest Carbon Services A. Update and Assessment at Three Field Sites Field Site Status Update Since TransLinks’ End Assessment of TransLinks’ Contribution 1. Makira Forest – Madagascar 705,588 certified carbon credits for sale. 32.5 million tons of carbon. First-ever government￾backed sale of carbon credits in Africa. Sept 17, 2013. http://science.time.com/2013/09/1 7/a-new-deal-for-carbon-raises￾hopes-for-threatened-forests-in￾madagascar/ USAID funding helped Makira REDD+ Project become validated and verified by the Verified Carbon Standard (VCS), and a Gold level validation by the Climate, Community, and Biodiversity Alliance. Sept 3, 2013 PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page iv Field Site Status Update Since TransLinks’ End Assessment of TransLinks’ Contribution 2. Dolakha, Gorhkha, and Chitwan – Nepal Technical and social information completed; PIN and PDD developed but on hold until government can decide on REDD+ policy. As of December 2013, held workshop with government to advance this deal and others in Nepal that are pending. While actual forest carbon deal is still pending, TransLinks materials and experience from this site were used to expand other PES development in the Philippines. Government and NGO officials visited the Nepal site in 2013. Training in carbon development using TransLinks-developed material was used in the Philippines to complete PIN and community￾managed carbon monitoring plots. 3. Seima Forest – Cambodia Being validated by SCS Global Services. FPIC obtained from 20 Phnong communities. First verification report due March 2014. Now the official national demonstration site. TransLinks funds helped show government that REDD can work but involves significant effort. B. Three Lessons Learned from the TransLinks Experience about PES for Forest Carbon Services Lesson 1: The process of certification is much more complicated and time-consuming than originally thought. The slowness of getting carbon to market is a major impediment to getting community and government buy-in. Barriers to entry are huge and require significant capital and expertize to overcome. Lesson 2: TransLinks helped expand and take PES beyond carbon and the politics of REDD and REDD plus. The TransLinks video was used for training all environmental grantees of the Philippines mission. At the request of the mission, EnterpriseWorks led the session; grantees commented the training helped them understand that PES deals were within their reach. Lesson 3: Ii is extremely difficult to raise awareness in rural communities about what REDD and carbon credits really mean. This is partly due to the hiatus between initial awareness raising and actual verification and marketing of carbon credits. As a result, maintaining awareness during the long verification process is critical. C. Three Examples of Uptake or Influence on Policy or Practice from TransLinks Work on PES for Forest Carbon Services Example 1: The Government of Madagascar agreed to an equitable benefit sharing process with local Makira forest stakeholders. Example 2: The Government of Madagascar became the first government in Africa to sell certified carbon credits. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page v Example 3: The Government of Cambodia was reticent to finalize a benefit-sharing protocol as it has yet to sell any carbon. 3. PES for Watershed Services A. Update and Assessment at Three Field Sites Field Site Status Update Since TransLinks’ End Assessment of TransLinks’ Contribution 1. Mbé Catchment – Gabon Funds received from GEF to help get the Payment for Watershed Services scheme to market. Pilot work was essential to developing the GEF proposal. 2. Ruvu River – Tanzania I do not believe we funded any PWS work here. May have been a presentation at a Katoomba meeting. 3. RURES – Indonesia I do not believe we funded any PWS work here. May have been a presentation at a Katoomba meeting. B. Three Lessons about PES for Watershed Services Learned from the TransLinks Experience Lesson 1: Transfer pricing (hydro-electricity use fees to subsidize drinking water fees) mandated by the Government of Gabon was identified as a policy challenge that might undermine the proposed Payment for Water Services scheme for the Mbé watershed. Lesson 2: (No response given) Lesson 3: (No response given) C. Three Examples of Uptake or Influence on Policy or Practice from TransLinks work on PES for Watershed Services Example 1: A water PES deal with a private sector fruit company has been concluded with an indigenous group in Mindanao, Philippines. Talks are ongoing with two other major multinationals for similar water PES deals in the Philippines. Example 2: The Government of Gabon is moving ahead with a Payment for Water Services system for the Mbé Watershed. Uptake No 3: (No response given) PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page vi 4. Wildlife/Biodiversity Friendly Enterprise A. Update and Assessment at Three Field Sites Field Site Status Update Since TransLinks’ End Assessment of TransLinks’ Contribution 1. Elephant Pepper – Zimbabwe Elephant pepper now sells two varieties of sauces and four spice grinders to markets in South Africa. It recently obtained shelf space in Whole Foods in the U.S. TransLinks did not directly support this enterprise. Rather support went to promoting the Wildlife Friendly Enterprise Network. 2. Ibis Rice – Cambodia Ibis Rice certification continues to expand to new producers in the northern plains. Ibis Rice is now purchased by nine major hotels in Siem Reap. USAID funding was a critical component of getting Ibis Rice to scale. 3. Rattan Furniture – Philippines The work with Cebu Furniture Manufacturers Association in Rattan has allowed the same model and interventions to be applied to other NTFPs in the Philippines. Assistance has been extended to northern Mindanao sites. USAID funding was an important component in engaging industry members for expanded use of a variety of NTFPs and supporting sustainable harvest practices to conserve the Philippines high biodiversity forest. B. Three Lessons Learned from the TransLinks Experience with Wildlife/Biodiversity-Friendly Enterprise Lesson 1: Experience from Cambodia showed that different payments schemes result in different ecological and livelihood outcomes. We should not expect a single PES silver bullet. Lesson 2: Some PEs schemes, such as ibis tourism, cannot be scaled up, because demand is limited and once many birders have seen a rare species and added it to their life list they are not interested in a return visit. Lesson 3: With so many certification schemes around (organic, fair-trade, FSC, wildlife-friendly, shade￾grown, certified humane), there is now competition for label space and producers must decide which certification labels to prioritize. C. Three Examples of Uptake or Influence on Policy or Practice from TransLinks work on Wildlife/Biodiversity-Friendly Enterprise Uptake No 1: (No response given) PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page vii Uptake No 2: (No response given) Uptake No 3: (No response given) 5. Community Based Natural Resource Management A. Update and Assessment at Three Field Sites Field Site Status update since TransLinks end Assessment of the TransLinks Contribution 1. Okapi Reserve – DR Congo Work continues in this turbulent region of Africa. The zoning work in OWR was considered a model worth replicating in other CARPE landscapes Provided the author with the time and resources to completed the case study 2. Eastern Steppe - Mongolia EMCCA the Eastern Mongolia Community Conservation Association continues to use the WCS approach for establishing and training Herder Community Groups TransLinks funding was essential for conducting the workshops that assess herder perceptions of wildlife abundance, wildlife use and natural resource management capacity. 3. Gran Chaco - Bolivia This was a case study of prior work with the Isoseno’s and not part of an ongoing project Provided the author with the time and resources to completed the case study B. Three Lessons About CBNRM Learned from the TransLinks Experience. Lesson 1: The extraordinary financial success of the Women of Isoso project was a significant benefit to Isoso families but cause internal political problems within CABI (the representative organization for the Isosenos) as the men, who dominate CABI were jealous of the women’s success. Lesson 2: (No response given) Lesson 3: (No response given) C. Three Examples of Uptake or Influence on Policy or Practice from TransLinks work on CBNRM Uptake No 1: The TransLinks supported approach to establishing Community Herder Groups is now the standard practice used by EMCCA in Mongolia Uptake No 2: (No response given) Uptake No 3: (No response given) PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page viii APPENDIX 3. KEY INFORMANT CHECKLIST 1. Identification Name: Date: Organization: Position: Main Responsibilities: What are your main environmental interests? Tick all that apply.  REDD+/forest carbon  PES watershed services  Wildlife/biodiversity conservation  Community NR management  Other (explain) _____________________________ 2. Contact with TransLinks How would you describe your involvement with TransLinks? Implementing Partner / Participant / Beneficiary / Aware Observer / None Have you used any of the TransLinks products made available on USAID’s Natural Resources Management and Development Portal? (http://rmportal.net/library/content/translinks) How would you rate the different sets of products shown on the portal on a scale of 1 to 5, where 1 = Very Useful and 5 = Not Useful? Used/Not Used Rating Case Studies _________ _______ Briefs _________ _______ Tools _________ _______ Workshops _________ _______ Seminars _________ _______ PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page ix Are there any products on the portal you would rate as exceptional? Which are they and why? __________________________________________________________________ Have you attended any TransLinks events or training? What were they? ___________________________________________________ How useful were they? (1-5) __________ 3. Recent Outcomes in Natural Resource Management and Governance Our evaluation aims to identify major developments in NR management and governance over the last six years (since 2007) and assess what the work done by TransLinks has contributed to those outcomes. At the Global/International Level Outcome Details of the Development Assessment of the TransLinks Contribution 1. Greater weight given to forest carbon in the UNCED process. 2. Greater weight given to biodiversity in international fora. 3. Bilateral and multilateral donors give increased support to forest carbon initiatives. 3. Donors give increased support to biodiversity conservation initiatives 4. Better methods for the design and implementation of projects to trade forest carbon in voluntary markets. 5. Better methods for establishing payment for ecosystem services. 6. Better methods for enabling communities to manage the interface between their livelihoods and neighbouring areas of high biodiversity. 7. Other (please add to the list if appropriate). At the National Level Outcome Details of the Country and the Development Assessment of the TransLinks Contribution 1. Better national policies for REDD+ implementation. 2. Better national policies for PES. 3. Better national policies for community NR management. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page x Outcome Details of the Country and the Development Assessment of the TransLinks Contribution 4. Better national policies for forest and biodiversity conservation. 5. Other (please add to the list if appropriate). At Individual Sub-National Sites Outcome Details of the Location and the Development Assessment of the TransLinks Contribution 1. Sustainable deals to trade forest conservation for carbon credits. 2. Sustainable deals trading catchment conservation for water services. 3. Sustainable deals trading biodiversity conservation for community or private benefits. 5. Other (please add to the list if appropriate). In Knowledge about NR Management Outcome Details of the Concepts and Evidence Assessment of the TransLinks Contribution 1. New concepts or new evidence about forest carbon and its better management. 2. New concepts/evidence about PES for water services. 3. New concepts/evidence about community management of biodiverse natural resources. 4. Other (please add to the list if appropriate) PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xi APPENDIX 4. TRANSLINKS E-SURVEY SUMMARY Note: Responses are taken verbatim from the surveys and have not been corrected for spelling or grammar errors Question 1: Please indicate your name, company and your position Answer Options Percent Response Response Count Name: 100.0% 28 Company: 100.0% 28 Position: 100.0% 28 answered question 28 skipped question 0 List of Organizations Represented  Cebu Furniture Industries Foundations, Inc.  EnterpriseWorks – Relief International  EWW Philippines  IRG  Kalinga Mission for Indigenous Children and Youth Development, Inc. (KAMICYDI)  Natural Capital Project/WWF  Perkumpulan KABAN  Rare  Relief International  The Nature Conservancy  UN Foundation  University of the Philippines Los Banos  USAID (including USAID/AFR/SD: Africa Bureau, USAID/PPL/SPP)  Virginia Tech  Wildlife Conservation Society (WCS) Question 2: Please describe your main responsibilities at your current position Answer Options Response Count answered question 26 skipped question 2 List of Answers:  Environmental policy  Manage staff and strategic partnerships/fundraising for a portfolio of site based biodiversity conservation projects employing social marketing and totalling ~ $8-10M/yr.  Business Development and Project management PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xii  Manage, direct and supervise the implementation of programs, projects and services of the organization. Assist the BOT in strategic direction, systems installation and knowledge management techniques.  As Chair, I help the Board of Directors set the strategic direction of the organization. I also directly supervise the CEO.  Teaching, research and extension work  As the past executive director of the organization for the last 20 years and now its current consultant, I advise the staff and board on matters pertaining to the management of the organization as a business support organization as well as making sure that plans and programs are implemented according to expectations.  To assist people organization and to coordinate field event to support people and community organizing in Sentarum Lake National Park and Siawan Lake, Kapuas Hulu, West Borneo  Provide forestry and climate expertise  Ecosystem services, science-policy interface  I support RI's projects in the EnterpriseWorks Division, which focus on NRM, rainwater harvesting, and improved cook stoves. We employ for profit solutions to poverty  Lead initiatives on resource management  I support US debt reduction programs used to finance conservation trust funds in qualifying developing countries for the purpose of tropical forest conservation.  Oversee Social Enterprise Global Portfolio  Senior Advisor on the Program Cycle and other USAID programming policies.  Office works to strengthen critical links between biodiversity conservation, natural resources management, improved livelihoods and economic growth, and good governance throughout Africa. It provides leadership on African development issues through analysis, strategy development, program design, technical assistance, advocacy, and information dissemination.  I manage NRM projects in the Philippines, mostly forestry ones  I just retired but prior to that I coordinated several USAID contracts for Virginia Tech for 10 years or so in addition to my university teaching and research responsibilities.  Environmental policy analyst across the development spectrum, esp. as to env. & social soundness, focusing on sub-Saharan Africa.  Manage programs on biodiversity, climate change and research. Advise on gender, social science, knowledge management and various other tasks.  I represent the USA on international forestry matters related to trade and governance; manage a public private partnership, and I provide TA to USAID Missions.  Director of the Wildlife Conservation Society (WCS) - Cambodia Program. Management of >200 staff and a budget of >$2.5 million.  I support a variety of TNC's interactions with USAID in Latin America, support our USAID projects, and write proposals for USAID funding. I also coordinate our Latin America Climate Adaptation activities.  I direct all central office technical support to our field programs  Design and manage agricultural development projects. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xiii Question 3: What are your main Environmental Interests? Click all that apply. Answer Options Percent Response Response Count REDD+/Forest Carbon 69.2% 18 PES Watershed Services 65.4% 17 Wildlife/Biodiversity Conservation 69.2% 18 Community NR Management 84.6% 22 Other (please specify) 14 answered question 26 skipped question 2 Question 4: How would you describe your involvement with TransLinks? Answer Options Percent Response Response Count Implementing Partner 15.4% 4 Participant 23.1% 6 Beneficiary 0.0% 0 Aware Observer 50.0% 13 None 11.5% 3 answered question 26 skipped question 2 PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xiv Question 5: Have you used any of the TransLinks products made available on USAID’s Natural Resources Management and Development Portal? (http://rmportal.net/library/content/translinks) Answer Options Percent Response Response Count Yes 56.0% 14 No 44.0% 11 answered question 25 skipped question 3 Question 6 How would you rate the different sets of products shown on the portal on a scale of 1 to 5, where 1 = Very Useful and 5 = Not Useful? Answer Options Very Useful Somewhat Useful Neither Useful nor Not Useful Not Very Useful Not At All Useful N/A Rating Average Response Count Case Studies 7 6 1 0 0 1 1.57 15 Briefs 8 6 0 0 0 1 1.43 15 Tools 9 4 0 0 0 2 1.31 15 Workshops 5 8 1 0 0 1 1.71 15 Seminars 5 7 1 0 0 2 1.69 15 answered question 15 skipped question 13 PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xv Question 7: Is there any products on the portal you would rate as exceptional? Which are they and why? Answer Options Response Count answered question 10 skipped question 18 List of Answers:  Publications by Lisa Naughton and her colleagues. These are not easily absorbed by policy makers, but they are asking the right questions.  Linkages to other sites. The diversity of the ecosystem of products  I do review the video presentations on the portal. I often download the documents. My personal interest is in PES, however, the private sector investment in TransLinks projects such as COMACO show exceptional outcomes.  The tools section since it brings together items needed under nature, wealth and power in one place.  Nature Wealth and Power 2.0  Do not recall specific examples  NWP 2.0, N. Kenya Rangeland carbon sequestration work  The selection of products on land tenure, the workshop and products on bundling and stacking, the Katoomba materials from West Africa  PES Primers (original and marine); Conservation Marketing Equation; WCS Case Studies on Cambodia (3), Guatemala (Turkeys), Tanzania (Simanjiro), Madagascar (Makira); LTC Brief #11; LTC Lessons about Land Tenure, Forest Governance & REDD; Social & Biodiversity Impact Assessment Manual for REDD+ Projects; Surui documents; Peer-reviewed Journal articles; Nested REDD+; State of Biodiversity Markets Report; State of Watershed Payments Report; too many others  Closeout video, Conservation Marketing Equation, PES case studies, PES Getting Started, REDD project development guide Question 8: Have you attended any TransLinks events or training? Answer Options Percent Response Response Count Yes 60.0% 15 No 40.0% 10 answered question 25 skipped question 3 Question 9: What was the TransLinks event or training that you attended? Answer Options Response Count answered question 14 skipped question 14 PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xvi Question 10: If you have attended any TransLinks event or training, how useful was it? Answer Options Percent Response Response Count Very Useful 57.1% 8 Somewhat Useful 42.9% 6 Neither Useful nor Not Useful 0.0% 0 Not Very Useful 0.0% 0 Not Useful at All 0.0% 0 answered question 14 skipped question 14 Question 11: Outcome: Greater weight given to Forest Carbon in the UNCED process Answer Options Percent Response Response Count Details of the Development 100.0% 2 Assessment of the TransLinks Contribution 100.0% 2 answered question 2 skipped question 26 Question 12: Outcome category: Greater weight given to biodiversity in international fora Answer Options Percent Response Response Count Details of the Development 100.0% 4 Assessment of the TransLinks Contribution 100.0% 4 answered question 4 skipped question 24 Details of the Development Assessment of the TransLinks Contribution GEF investment in PES enabling conditions in Latin America Raised the profile of the approach with national conservation groups Private Industry in Asia better understanding biodiversity in their supply chains Direct result of private sector inclusion in Translinks workshops and site visits; changed procurement practices I don't know TL not really set up to do this I don't see this as a direct goal of TransLinks. but TL may have some impacts on the UN-IPBES program PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xvii Question 13 Outcome category: Bilateral and multilateral donors give increased support to Forest Carbon initiatives Details of the Development Assessment of the TransLinks Contribution I don't know TL was not set up to do this Contracting of the $20 million Supporting Forests and Biodiversity (USAID-SFB) project by USAID in Cambodia The pilot work by WCS and Winrock International on the Seima REDD+ project stimulated greater interest in REDD+ in Cambodia. The SFB project was specifically designed to build on the TransLinks-funded pilot work and take it to the next stage. USAID gave more support to Forest Carbon after TransLinks precedents, though it would have done so anyway. TL did have some impact on how this was done in E3 (global FCMC project) and LAC's new Regional Community-oriented REDD program, the competition for which a TL partner won (its first USAID project as leader). I would like to have seen it have more impact on the USAID Sustainable Landscapes program writ large. Question 14: Outcome category: Donors give increased support to Biodiversity conservation initiatives Answer Options Percent Response Response Count Details of the Development 100.0% 4 Assessment of the TransLinks Contribution 50.0% 2 answered question 4 skipped question 24 Details of the Development Assessment of the TransLinks Contribution GEF investment in species programs such as AZE alliance Not sure Global Conservation Trust Funds (CTFs) are grantmakers and, therefore, local donors. The TransLinks materials were frequently share at CTF association network meetings (e.g. the Annual Regional Network of Latin American and Caribbean Environmental Funds Network - RedLAC). While the strategic investments may have been influenced by TransLinks products, the principal interest was how the CTFs could play a role as intermediaries (e.g. REDD benefits distribution) in PES schemes. I don't know TL was not set up to do this I don't see this as a direct goal of TransLinks. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xviii Question 15 Outcome category: Better methods for the design and implementation of projects to trade forest carbon in voluntary markets Answer Options Percent Response Response Count Details of the Development 1 6 Assessment of the TransLinks Contribution 1 6 answered question 6 skipped question 22 Details of the Development Assessment of the TransLinks Contribution Surui REDD+ experience Established a positive case study Range of community-led forest carbon monitoring options defined and tested TransLinks supported access to a range of options and support for their testing and documentation I don't know The Surui case as a model, also lessons and models developed in Katoomba meetings Methodological guidance for REDD+ Projects TransLinks helped to develop a REDD+ Primer, and provided technical guidance on the methods used for Manuals for Social and Biodiversity Impact (SBIA) Assessment of REDD+Projects The manuals developed first under TransLinks have been refined and adopted by other NGOs outside of TransLinks, and by USAID's own FCMC project, and have greatly influenced discussions about "Social Safeguards" for REDD. WCS carbon projects have template for development TransLinks helped developed best REDD development practices Question 16: Outcome category: Better methods for establishing Payment for Ecosystem Services Outcome category: Better methods for establishing Payment for Ecosystem Services Answer Options Percent Response Response Count Details of the Development 1 7 Assessment of the TransLinks Contribution 1 7 answered question 7 skipped question 21 Details of the Development Assessment of the TransLinks Contribution Conserved our forest and used fast regenerating non-timber materials in producing products Help our community in the Province of Kalinga engage in micro-enterprise using non-timber materials as a result of my training facilitated by TransLinks A set of lessons about stacking and bundling Quite good Communities and other donor implemented projects consider PES in broader terms beyond carbon TransLinks video and trainings allowed local project to develop locally applicable PES beyond carbon (water, eco-tourism, products) PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xix Details of the Development Assessment of the TransLinks Contribution Sponsored a series of workshops, conferences, publications, tools, etc. on PES around the world Significant. One of the best archives on PES information. I don’t know More expertise in WCS. EnterpriseWorks and USAID Improved understanding of the design of PES projects TransLinks funded research into PES in Cambodia, which has led to 4 papers in peer-reviewed journals, 3 case studies, and has stimulated research and thinking into PES both in Cambodia and globally. Two of the Cambodia PES projects have won international awards. Forest Trends developed training materials and conducted training for community leaders in several countries – Brazil, Ecuador, Peru, Honduras, Tanzania, South Africa, Uganda Coupled with their precedent-setting Surui Forest Carbon project in the Amazon and their SBIA Manuals, Forest Trends helped set a new standard for how to engage with communities on forest carbon so that they benefit from REDD rather than becoming victims of another international scheme for their lands, which also led to a new LAC Regional project. In addition to #17, this also contributed a vision and methods for enabling communities to manage the interface between their livelihoods and neighboring areas of high diversity. WCS how has PES program and growing portfolio Lessons learned from TL avoids pitfalls Question 17: Outcome category: Better methods for enabling communities to manage the interface between their livelihoods and neighboring areas of high biodiversity Answer Options Percent Response Response Count Details of the Development 1 5 Assessment of the TransLinks Contribution 1 5 answered question 5 skipped question 23 Details of the Development Assessment of the TransLinks Contribution Conserved our forest and used fast regenerating non-timber materials in producing products Help our community in the Province of Kalinga engage in micro-enterprise using non-timber materials as a result of my training facilitated by TransLinks Explicit tools for integrating enterprise and biodiversity TransLinks tools used in other USAID and EU projects in Asia, Africa and Latin America to combine biodiversity and livelihoods. I don't know TL case studies detail ways to improve PES approaches to biodiversity conservation PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xx Details of the Development Assessment of the TransLinks Contribution Improved understanding of the impact of biodiversity conservation initiatives on human wellbeing TransLinks funded research into the interaction between human livelihoods and biodiversity conservation interventions, leading to recommendations for improved design of initiatives. These results have been published in peer-reviewed journal articles and have been widely cited. Conservation Marketing Equation Tool/methodology This tool combined for the first time the Nature, Wealth, and Power framework with the value chain methodology for natural resource-based enterprise development, and was used in several workshops in Asia (Cambodia, Philippines, Nepal) and Africa (Tanzania, Uganda). Also see #16. Question 18: Other (please add to the list as appropriate) Answer Options Response Count answered question 3 skipped question 25  Open-Ended Response:  The topics on biodiversity conservation which I acquired during the TransLinks Training in Cebu City was one of the ideas I shared in the International Meetings of the Conference of Parties (COP) 7, 8, 9, and 10 when I was invited as one of the panel speaker.  I do not have sufficient information to make meaningful comments. I support many environmental orgs with personal contributions, but my not for profit responsibilities have only partial links to the environment.  The Health as an Ecosystem Service Workshop led to funding from other donor (I don't know the details) and an ongoing research program called "One Health", I think housed at University of California at Davis, which includes closer linkages with the Global Health Bureau's efforts to manage and prevent epidemics from emerging disease with ties to wildlife (e.g., SARS, tuberculosis, HIV). Question 19: Outcome category: Better national policies for REDD+ implementation Answer Options Percent Response Response Count Details of the Country and the Development 1 4 Assessment of the TransLinks Contribution 1 4 answered question 4 skipped question 24 PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xxi Details of the Country and the Development Assessment of the TransLinks Contribution Brazil indigenous rights to carbon Cambodian Government National REDD+ Readiness Programme TransLinks directly helped with the development of REDD+ policy by the Royal Government of Cambodia, and the establishment of the Seima REDD+ pilot project Brazil Surui project - led to a new national law signed in the Spring of 2012 recognizing the rights of indigenous forest communities to the forest carbon in their Indigenous Reserves, an area totaling about 105 million hectares, a huge precedent for Brazil and possibly, other countries. TransLinks funded the legal study that led to this recognition. Cambodia, Madagascar adopt benefit sharing policies TL funds used to encourage these policy reforms Question 20: Outcome category: Better national policies for Payment for Ecosystem Services Answer Options Percent Response Response Count Details of the Country and the Development 1 3 Assessment of the TransLinks Contribution 0.333 1 answered question 3 skipped question 25 Details of the Country and the Development Assessment of the TransLinks Contribution Once again, the CTF community often has a significant voice in national government policy development. With the use of TransLinks materials, a number of CTFs had increased capacity to engage their government PES and REDD policy dialogs. (e.g. Colombia) I don't know Cambodian Government PES policies The Cambodia PES projects, piloted under TransLinks, have helped to shape national thinking on PES in Cambodia PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xxii Question 21: Better national policies for Community Natural Resource Management Answer Options Percent Response Response Count Details of the Country and the Development 1 2 Assessment of the TransLinks Contribution 1 2 answered question 2 skipped question 26 Details of the Country and the Development Assessment of the TransLinks Contribution Nepal - REDD+, Philippines expanded PES Direct case study and workshops support for the expanded PES Brazil indigenous rights to carbon and REDD+, more community based REDD+ action Question 22: Outcome category: Better national policies for Forest and Biodiversity conservation Answer Options Percent Response Response Count Details of the Country and the Development 1 1 Assessment of the TransLinks Contribution 0 0 answered question 1 skipped question 27 Details of the Development: Assessment of the TransLinks Contribution: I don't know Question 23: Other (please add to the list as appropriate) Answer Options Response Count answered question 3 skipped question 25 Open-Ended Response  Better community-based resource management systems  See #14  Sorry for all the “I don’t knows” but I don’t have the details of outcomes of specific countries and fora PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xxiii Question 24: Outcome category: Sustainable deals to trade forest conservation for carbon credits Answer Options Percent Response Response Count Details of the Country and the Development 1 6 Assessment of the TransLinks Contribution 0.667 4 answered question 6 skipped question 22 Details of the Country and the Development Assessment of the TransLinks Contribution Ghana stoves project -- Relief International Four projects in East Africa I am aware of, recently studied in a paper by FCMC (ARD/TT), comprise: 1. Trees for Global Benefits project (TFGB) implemented by Environmental Conservation Trust of Uganda (ECOTRUST), a Ugandan non￾governmental organization (NGO): 2. Humbo Assisted Natural Regeneration Project (HANRP) in Ethiopia, implemented by World Vision through its Australian and Ethiopian affiliates; 3. The International Small Group and Tree Planting Program (TIST) in Kenya implemented by Clean Air Action Corporation (CAAC) and Institute for Environmental Innovation (I4EI); and 4. East Aberdare/Mount Kenya Forest Rehabilitation Project (EAMK-FRP) implemented by Green Belt Movement (GBM), a Kenyan NGO. Not aware of any directly I don't know Cambodia: Seima REDD+ Project TransLinks funded WCS to develop the Seima REDD+ project for the voluntary carbon market. Surui Indigenous Forest Carbon project, Brazil - first to achieve dual verification and validation and actual carbon sales Would not have happened without TransLinks seed funding for Forest Trends' involvement. Makira carbon credits validated TL funds facilitated this process PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xxiv Question 25: Outcome category: Sustainable deals trading catchment conservation for water services Answer Options Percent Response Response Count Details of the Country and the Development 1 5 Assessment of the TransLinks Contribution 0.6 3 answered question 5 skipped question 23 Details of the Country and the Development Assessment of the TransLinks Contribution Entergprise Works llc "bob" rain water capture device Philippines, Water PES between private sector ag company and indigenous community in uplands Tools and workshops on water PES from TransLinks got the initiative started and was completed with other donor financing I don't know This actually was never a major objective of TransLinks, but may yet lead to a project in Gabon TransLinks leader WCS did a feasibility study that pointed out that government policies on water pricing were a major roadblock, which may have shifted government policy to enable a project to go forward. Forest Trends' publication of the "State of Watershed Payments" report also provided useful global analysis and highlighted this burgeoning field. GEF investment in PWS in Gabon TL case study provided critical initial analysis Question 26: Outcome category: Sustainable deals trading biodiversity conservation for community or private benefits Answer Options Percent Response Response Count Details of the Country and the Development 1 5 Assessment of the TransLinks Contribution 1 5 answered question 5 skipped question 23 Details of the Country and the Development Assessment of the TransLinks Contribution Wildlife Friendly enterprise in Africa, Asia, and LAC include explicit benefits for communities and biodiversity Support from TransLinks allowed Wildlife Friendly to link deals around the work so they could learn from each other and gain greater access to markets and technical assistance PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xxv Cambodia Contributed to Ibis rice deals, maybe other contributions in other countries Cambodia: Tmatboey Community-based Ecotourism Project and Ibis Rice TransLinks funded the design and research into these two PES projects, both of which linked biodiversity conservation outcomes to community and private benefits (payments to communities and households). The Tmatboey, Cambodia "Ibis Rice" program that actually combines three different local revenue generation systems in return for conservation of endangered bird species. TransLinks funded some of the data collection and analysis of this program, leading to dissemination of various lessons coming from WCS' and EW/V's patient approach to PES, eco￾tourism development, eco-agriculture, and premium-price marketing. Expansion of conditional conservation enterprises at WCS Lessons learned from TL helped demonstrate feasibility and value of this approach Question 27: Other (please add to the list as appropriate) Answer Options Response Count answered question 1 skipped question 27 Open-Ended Response  I’d like to know more about what TransLinks has accomplished in the above respects, regarding “sustainable deals” and PES Question 28: Outcome category: New concepts or new evidence about forest carbon and its better management Answer Options Percent Response Response Count Details of the Country and the Development 1 1 Assessment of the TransLinks Contribution 1 1 answered question 1 skipped question 27 Details of the Country and the Development Assessment of the TransLinks Contribution More knowledge about bundling and stacking, Nature, Wealth and Power as it applies to REDD+, more focus on biodiversity case studies and other publications PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xxvi Question 29: Outcome category: New concepts/evidence about PES for water services Answer Options Percent Response Response Count Details of the Country and the Development 1 4 Assessment of the TransLinks Contribution 0.5 2 answered question 4 skipped question 24 Details of the Country and the Development Assessment of the TransLinks Contribution for profit sale of rainwater harvesting devices Philippines - At President's level for water PES Introduced how to explore and model water PES and allowed key actors to attend trainings and workshops Dissemination of information on lessons learned about PES in many countries. One of the better sources of information on PES. Never a significant goal of TransLinks. Question 30: Outcome category: New concepts/evidence about community management of biodiverse natural resources Answer Options Percent Response Response Count Details of the Country and the Development 1 5 Assessment of the TransLinks Contribution 1 5 answered question 5 skipped question 23 Details of the Country and the Development Assessment of the TransLinks Contribution Complemented our traditional knowledge & indigenous knowledge systems of community-based biodiversity management Acquired during the training by TransLinks Philippines, Paraguay, Wildlife Friendly enterprises in 10 countries Brought key country level actors together to incorporate new concept and evidence-based interventions that integrate community management of biodiversity Wildlife Friendly Enterprises TL supported the development of a whole new type of certification PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xxvii Cambodia: community-based PES research TransLinks funded a long-term research program into the design of community PES projects to maximize biodiversity conservation and poverty reduction goals. This research has been published in peer-reviewed literature and case studies, and has been heavily publicized. The papers have helped to stimulate improved knowledge into the design of community PES projects. Tmatboey, Cambodia together with Nepal Forest User Groups showed that it was possible to stack multiple sources of payments or revenues from improved CBNRM and Wildlife-Friendly Enterprise Development for biodiversity in a developing country. TransLinks did not fund the main development activities in either location, but it funded data collection and analyses so that the successes were documented and disseminated. Question 31: Other (Please add to the list as appropriate) Answer Options Response Count answered question 1 skipped question 27 Open-Ended Response  Emerging insights about rangeland management and enhanced (drylands) grazing systems and the endogenous benefits deriving for water, vegetative cover, biodiversity & livelihoods (evidence in Northern Kenya especially) Question 32: Any other comments Answer Options Response Count answered question 7 skipped question 21 Open-Ended Response  I am not a user of TransLinks so am unfamiliar with its impact  I have limited knowledge on the other programs and services of TransLinks which give me limited opportunity to answer your questions. However, I am aware that you are doing great in the field of NRM. Good luck!  Other than the training I've attended, I did not become aware of the other initiatives of TransLinks I do not know how the Philippines has benefited from its programs, if there are any  Apologies for not being able to provide concrete "evidence" or application outcomes of the TransLinks tools and publications. They were, however, made available to players (CTFs) who often have substantive roles for application of these materials locally. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xxviii  The sheer body of work makes it difficult to parse out all potential impacts. It will be really important to get input from non-USAID and non-TL partners, even if you have to call and interview them. So this would be GCC Office at USAID, land tenure experts (contractors working on land tenure for USAID), those working on wildlife friendly enterprise, etc. But many potential interviewees will not know what the TL contribution is.  Sorry, I could not comment on the above, as I really did not know what the outcomes were, or the details or project activities. I was aware of a Standing Forest Initiative that TransLinks supported for us in Brazil, but don't know the outcomes. I never go on the FRAME Portal, but I did always grab a hard copy of TransLinks Publications, because PES is a new and emerging area, and this seemed to be the best source of information. However, I do not know what impacts the outreach had, although I know that thanks to some projects in Vietnam and Ecuador USAID has been a leader on this topic. I just don’t know the TransLinks role or performance well enough.  I really had not heard of the TransLinks program until I was contacted by the evaluators. Please do not regard this as a negative perspective of the TransLinks program -- there are many USAID programs that I do not know about. Project Total Per Annum Value @ $7.5 Net @ 20% TL Cont @ 10% Makira 33,166,337 1,105,545 8,291,584 1,658,317 165,832 Seima 51,207,340 1,706,911 12,801,835 2,560,367 256,037 Guatecarbon 24,030,000 801,000 6,007,500 1,201,500 120,150 Takamanda Mone 8,284,710 276,157 2,071,178 414,236 41,424 Nepal 919,500 30,650 229,875 45,975 4,598 Surui 7,423,800 247,460 1,855,950 371,190 37,119 Total 625,158 PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xxix APPENDIX 5. DISCOUNTED CASHFLOW ANALYSIS OF TRANSLINKS-SUPPORTED REDD+ SCHEMES Table A5.1 shows the calculation of an estimated benefit attributable to TransLinks from its support to six forest carbon schemes. Table A5.1 Calculation of Estimated Forest Carbon Benefits Attributable to TransLinks REDD+ Scheme Carbon Credits – mt Tons Per Annum Value @ $7.5/Ton Net @ 20% TL Cont @ 10% $ Per Annum Makira Forest, Madagascar 3,166,337 1,105,545 8,291,584 1,658,317 165,832 Seima Forest, Cambodia 51,207,340 1,706,911 12,801,835 2,560,367 256,037 Guatecarbon, Guatemala 24,030,000 801,000 6,007,500 1,201,500 120,150 Takamanda-Mone, Cameroon 8,284,710 276,157 2,071,178 414,236 41,424 Dolakha, Gorhkha, Chitwan, Nepal 919,500 30,650 229,875 45,975 4,598 Surui People, Brazil 7,423,800 247,460 1,855,950 371,190 37,119 Total $ Per Annum 625,158 Note: The schemes are briefly described in Table 8 of the main report. The assumptions underlying the calculation are as follows:  The six forest carbon schemes will sell the currently estimated quantity of carbon credits over 30 years, starting from 2016.  The sale price will be $7.5 per MtCO2e (metric tons of carbon dioxide equivalent), the 2012 average for voluntary markets.  The net value of the credits, after subtracting implementation and opportunity costs, will be 20 percent of the gross.  10 percent of that value can be attributed to TransLinks.  Including cost-share contributions, $7 million cost of TransLinks was spread evenly between 2007 and 2011.  That there are no benefits from any other aspects of TransLinks’ work and that all the costs of the program must be absorbed by the six forest carbon schemes. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xxx Table A5.2 summarizes the discounted cash flow analysis used to calculate the Internal Rate of Return on the TransLinks investment on this basis. Table A5.2 TransLinks Discounted Cash Flow Model CASH FLOWS 2007 $ 2008 $ 2009 $ 2010 $ 2011 $ 2012 to 2015 From 2016 $ To 2045 $ TransLinks Cost 1,400,000 1,400,000 1,400,000 1,400,000 1,400,000 Carbon Benefits 625,158 <625,158> 625,158 Net Cash Flow -1,400,000 -1,400,000 -1,400,000 -1,400,000 -1,400,000 0 625,158 <625,158> 625,158 Internal Rate of Return 5.1% PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xxxi APPENDIX 6. EVALUATION METHODOLOGY 1. Introduction This paper outlines the planned methodology for an evaluation of the TransLinks program funded by USAID between 2006 and 2012. The consultancy IBTCI has been commissioned to carry out the evaluation as part of a work package which covers two other USAID-supported programs: Global Sustainable Tourism Alliance and Property Rights and Resource Governance. Data collection and analysis mechanisms common to all three programs are described in Section A of the main methodology document. The following sections set out how they will be applied for the evaluation of TransLinks. The evaluation has been commissioned by USAID’s Office of Land Tenure and Resource Management (LTRM), which will be the primary user of the evaluation findings. The purpose of the evaluation is to analyze the LTRM’s investment in TransLinks in order to: 1. Compare planned outputs and outcomes to those which were achieved; and 2. Where data is sufficient, determine the return on investment and key drivers of success (or failure). Mechanism Performance Outcome Questions Project design and Management Results Questions What Worked in the Design, and What Did Not? Broader Program Dissemination Results Questions How LTRM or Biodiversity Conservation Approaches Were Used TransLinks How did WCF Forest Trends use the lessons learned that they produced internally to influence their (or their international partners’) programs? To what extent was new NRM knowledge generated from this work? (i.e., new peer reviewed articles, new grey literature produced, additional new lines of research questions explored) What role did project design (and any changes/evolution in the design) play in the final results of the program? What are the lessons from this partnership and/or consortium? From TransLinks’ results, what were the key determinants of success in documenting and disseminating the results of successful NRM tools for greater adoption? Have (and if so, how) these case studies affected the development of the REDD+ mechanism? Have (and if so, how) these case studies affected the development of innovations in PES schemes? The Table shows the Evaluation Questions laid down in the LTRM Task Order for the evaluation. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xxxii 2. TransLinks The Program Description, which formed part of the original TransLinks Grant Award, set out the objective: to “help improve natural resource management (NRM) and governance to enhance biodiversity conservation and productivity for sustainable poverty reduction and economic growth” Four categories of integrated activity were planned: 1. Knowledge generation through applied research, 2. Using knowledge to build on and add to a suite of decision support tools, 3. Training to enhance the capacity of local and national partners to integrate the inter-dependent concepts of NWP into natural resources enterprise and biodiversity conservation planning, implementation and adaptive management, and 4. Widely disseminated technical solutions that provide clear guidance on applying the concepts of NWP in economic development and NRM decision making. TransLinks was implemented by a consortium of six institutions with extensive experience and specialist knowledge of NRM: – Wildlife Conservation Society (WCS), Earth Institute (EI), Enterprise Works/VITA (EWV), Forest Trends (FT) and the Nelson Institute for Environmental Studies/Land Tenure Center (NI/LTC). WCS was the consortium leader. An initial review of the 180 odd TransLinks products shows that the primary focus has been on projects that have applied the Payments for Ecosystem Services (PES) approaches to support people living in threatened ecosystems through conservation and sustainable use of the natural resource base on which they depend. Key evaluation questions revolve around how these PES incentive mechanisms have worked, and how development practitioners and policy makers have increased their ability “to promote more equitable natural resource governance that conserves ecosystems and the vital services they contribute to the welfare and livelihoods of poor families.”15 To answer the six TransLinks evaluation questions set out in the SoW, the evaluation team will use the general approach and methodology described in Section A. The remainder of this section sets out a more detailed approach, identifying the main steps that will be completed to prepare a draft TransLinks Evaluation Report. Of the six Evaluation Questions, the one on Program Design and Management Results has to do with the TransLinks Process. The other questions relate to different aspects of the TransLinks Outcomes: what the program has achieved. In drafting the methodology, particular attention has been given to identifying the most effective way to measure program Outcomes. TransLinks is a complex program with multiple implementers working across many countries. The central objective, i.e., better NRM leading to biodiversity conservation and sustainable poverty reduction, is broad and multi-facetted. The program has many potential Outcomes which cannot be tightly specified in advance. Those Outcomes will form part of a broader package of work by other agencies, national governments, and civil society. Direct attribution of a measured impact to TransLinks will not be possible. Table 1 15 TransLinks Final Technical Report, July 2012 PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xxxiii Outcome Harvesting (OH) is a technique which has been designed for the evaluation of this kind of program. The key principle is that “Outcome Harvesting does not measure progress towards predetermined outcomes or objectives, but rather collects evidence of what has been achieved, and works backward to determine whether and how the project or intervention contributed to the change.”16 Outcome Harvesting is a highly participatory tool and it consists of six iterative steps:17 1. Design the Outcome Harvest and identify useable questions to guide the harvest 2. Gather data and draft outcome descriptions 3. Engage change agents (i.e., implementers) in formulating outcome descriptions 4. Substantiate and validate findings 5. Analyze and interpret the data and provide evidence-based answers to harvesting questions 6. Identify how evaluation users (i.e., those who commissioned the evaluation in order to make decisions about the program) might use the findings. To implement OH fully takes time and relatively intensive discussion with implementing partners (“change agents”) and independent stakeholders (“substantiators”). The approach set out below adapts OH to fit with the budget and time allowed and also to include some more traditional evaluation techniques. The work will have three stages: 1. Document Review: Preparation of a comprehensive but concise statement of TransLinks achievements to provide a sound basis for the next stage; 2. Stakeholder Consultation: Key informant interviews with representatives of different stakeholders, supported with targeted e-surveys; and, 3. Partner Workshop: To test and add perspective to the findings of the first two stages, and bring them together into a robust final synthesis. Document Review Product Mapping: From a thorough review of the 180 odd TransLinks products, a matrix will be developed grouping the products by topic and country, along the lines of the following. The final shape of the matrix will be determined by the breakdown of the product portfolio. PES – Wildlife/BioD PES – Catchment REDD+ NR Science Tenure/ Governance Other Brazil Nepal Zambia Etc. 16 R. Wilson-Grau & H. Britt, 2012 – Outcome Harvesting, Ford Foundation, Cairo 17 Ibid. (Wilson-Grau and Britt 2012) PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xxxiv PES – Wildlife/BioD PES – Catchment REDD+ NR Science Tenure/ Governance Other Etc. Global This exercise will make it possible to:  Judge how far the different research, dissemination, training and advocacy elements of the program have come together to provide a coherent package of support on each issue, in a particular country or globally.  Identify the major areas in which new knowledge and lessons learned were expected to be generated. This will be used to define the evaluation questions, or ‘Useable questions’ in the OH approach: questions which will elicit answers which will allow the users of the evaluation report to make actionable decisions.  Summarize the conclusions reached in each area, to draft a first set of Outcome Descriptions, and to prepare semi-structured interview formats for discussion with stakeholders on how TransLinks has contributed to different Outcomes, and by how much.  Focus the search for potential TL Outcomes on areas where TL outputs have the greatest critical mass. In the time available, it will be necessary to limit the Harvest to a manageable number of Outcomes. The extent to which implementers and policy makers have used, or are aware of the research knowledge and practitioner tools developed under the program will be an important indicator of how TransLinks has contributed to outcomes in NWP. These research results and tools are made available on the USAID NRM and Development Portal.18 Thirteen technical manuals and primers on ecosystem services and market-based approaches to PES, 31 case-studies documenting ecological and livelihood benefits that may have accrued from PES initiatives and the challenges encountered in implementing them, 11 peer-reviewed journal articles and over 80 other reports. The product map, broken down by type (e.g., case studies, tools, journal articles), will be used to ask stakeholders which ones they found most useful. If web analytics are available, they will identify which of the products on the TransLinks page of the RM Portal have been most popular, in terms of views and downloads. Downloads to developing countries will be an important indicator of TL outreach to those who are most likely to put the material into use. Web-based citation analytics will also be sought, as an indicator of what impact the TransLinks peer reviewed publications have had. Design Review: The starting point will be the Program Description which is Attachment B to the Tran TransLinks slinks Letter of Award. From there the way the program has developed will be tracked through the annual work plans. The objectives set in those documents will be compared with the End of Project situation, as reflected in the product portfolio and the tables of results against the Performance Management Plan. 18 http://rmportal.net/TransLinks PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xxxv This analysis will provide a basis for discussion with TL implementers and USAID staff on how the original project design, and the way it has changed, has affected the final results. The Program Description indicates that a TransLinks Advisory Board was to be appointed and an Executive Committee formed representing the implementing partners. How these key design elements worked to shape the way the program developed will be part of the review. Stakeholder Consultation This will be at the heart of the evaluation. The majority of the discussions will be held over the telephone/Skype. A central Key Informant Questionnaire will be used to guide discussion of the main evaluation questions. However, some elements will be tailored to the interests and knowledge of different stakeholder groups as follows: TransLinks Implementing Partners: As well as being an important source on how TransLinks has generated new knowledge and lessons learned, program design and the way it has developed will be a primary topic for discussion with this group. A third key objective will be to capture their in-the-field experience of the way NWP works in practice. Drawing on the implementing partners’ knowledge to transform the draft Outcome Descriptions prepared in the Document Review into the final versions will be a central objective of this set of interviews. For the Chief of Party and other key actors, two rounds of discussion may be needed. In the first, apart from fact checking and filling any gaps in the data available, the evaluator will be seeking any guidance these key actors can give on how best to approach the evaluation. The second round of discussion will come after the completion of the document review, when the Key Informant Questionnaire will be completed. At the same time, implementing partners will be asked to confirm or correct initial conclusions emerging from the document review. Implementing partners will also be asked to identify other stakeholders who should be included in the consultation. USAID TransLinks Supervision Staff: As well as the main evaluation questions, the TransLinks supervision team will be principal informants on program design and development, in particular the way TransLinks has reflected US Government policies and priorities. USAID In-Country Staff: As a Leader with Associate program, TransLinks was expected to generate up to $9 million of local and regional activities sponsored by USAID missions in-country. Training, decision￾support, and other services were planned to help missions identify appropriate activities. In country staff views will be sought on how TransLinks has met their needs and influenced their programs. They will be asked to assist in identifying potential TransLinks outcomes in each country. TransLinks Event Participants: TransLinks has supported over 50 events, including training, workshops, seminars, and larger conferences. Just over 5,000 men and women have attended them. Some, but not all, TransLinks event reports include lists of those attending. These will be reviewed to see whether it will be practical, and useful, to contact a sample of participants for interview, or possibly through an e￾survey. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xxxvi TransLinks Project and Case Study Participants: The communities involved in the five REDD+ projects partly supported by TransLinks and the 31 PES case studies are representatives of the program’s primary beneficiaries. The document review of project and case study reports will allow it to be decided if it will be practical to include the community leaders from a sample in the stakeholder consultation. The NWP Expert Community: Although still subjective, the most independent assessment of TransLinks’ contribution to new knowledge and lessons learned on NWP will that of other specialists working on PES, REDD+, etc. A small panel of such experts will be invited to contribute to the stakeholder consultation. If an appropriate Listserv can be identified for the NWP community, an e￾survey may be possible. All groups apart from the implementing partners will be act as Outcome “Substantiators,” providing their informed opinion on how, and how much, TransLinks has contributed to each principal outcome. Partner Workshop If more time was available, it would have been useful to hold a partner workshop earlier in the process, to work together with them on the Outcome Harvest. That will not be possible, so this critical stage will be completed through the Key Informant Interviews. This later Partner Workshop will, therefore, roll three different parts of the process into one:  To review the Outcome Descriptions one more time  To allow the partners to respond to the Substantiators’ assessment of Outcome credibility  To allow them to check the analysis of how program design has evolved and the way that has affected the results  To allow them to respond to the proposed answers to the Evaluation Questions. Table 4 summarizes data which will be gathered to implement the TransLinks Evaluation approach and address the Evaluation Questions for this program. Although not put as a specific Evaluation Question, the evaluation Task Order does require a determination of the value of TransLinks as a USAID investment, “where data is sufficient.” While it may be possible to estimate a quantified Return on Investment from the data provided in some of the Case Study reports, for most program activities it is not expected that this will be possible nor, as a consequence, for the program as a whole. Two separate approaches will be used to provide a reasoned assessment of the potential return from TransLinks:  A bottom-up review of the cost involved in producing each TransLinks result, or set of results for comparison with the potential value of those results in terms of development and environmental conservation outcomes.  A top-down assessment, based on available literature, of the economic value of major outcomes in NRM management – from carbon sequestration to bio-diversity conservation. These will be used as a yardstick for the value of a TransLinks contribution to those outcomes, even where it is estimated that this contribution may be relatively small. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xxxvii Timing The evaluation draft report must be submitted by mid February 2014. To meet this deadline, the Partner Workshop is a critical date. The aim is to hold this between 28 January and 9 February. The schedule of Key Informant Interviews will start in December 2013 and continue through January 2014. Table 1: TransLinks information needs EVALUATION QUESTION TYPE OF ANSWER/ EVIDENCE REQUIRED COLLECTION METHOD SOURCE OF DATA SAMPLING/ SELECTION DATA ANALYSIS METHODS Performance Outcome Questions How did WCS/ Forest Trends use the lessons learned that they produced internally to influence their (or their international partners') programs? - List of Lessons Learned - Breakdown of TL Lessons Learned promotion activities - List of promotion audiences - Product Map - Review of dissemination activities - Review of tools - KIIs -Document review -Outcome Harvest -TL implementer reports - KII responses - 100% of TL products -Triangulation across Outcome Harvest and other sources To what extent was new NRM knowledge generated from this work? (i.e., new peer reviewed articles, new grey literature produced, additional/ new lines of research questions explored) - List of New Knowledge Outcomes - Breakdown of journal articles and other research outputs - Journal citations - Web analytics - Product map - Review of research outputs - KIIs - Document review - Outcome Harvest - TL implementer reports - KII responses - Web Analytics for NRM Portal/ TransLinks - Google Scholar etc. citation data - 100% of TL products -Triangulation across Outcome Harvest and other sources Project Design and Management Results Questions: What worked in the design and what did not? PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xxxviii EVALUATION QUESTION TYPE OF ANSWER/ EVIDENCE REQUIRED COLLECTION METHOD SOURCE OF DATA SAMPLING/ SELECTION DATA ANALYSIS METHODS What role did project design (and any changes/ evolution in the design) play in the final results of the program? - Track of design changes - Monitoring data on progress against plan and budget - Review of program award documents, Annual Work Plans, implementer reports - KIIs with USAID & implementers - Document review - KII responses Comprehensive - Narrative of program development supported by timeline of key events, deliverables and other milestones What are the lessons from this partnership and/or consortium - List of strengths and weaknesses - List of issues arising - Lessons learned from above As above As above As above Building on the preceding, analyze strengths and weaknesses and opportunities for improvement Broader Program Dissemination Results Questions: How LTRM or biodiversity conservation approaches were used From TransLinks’ results, what were the key determinants of success in documenting and disseminating the results of successful NRM tools for greater adoption? - List of TransLinks’ Outcomes - Clear result to Outcome attribution track for each TL result - Evidence of uptake for TL tools and research products - Outcome Harvest - Partner workshop - KIIs As above + Partner workshop proceedings 100% of TL products Tabulation of results and crosstabs with dissemination approaches Have these case studies affected the development of the REDD+ mechanism? If so, how? - Analysis of REDD+ development generally and in case study countries - TL REDD+ outcomes - Outcome Harvest - Partner workshop - KIIs As above 100% REDD+ case studies Qualitative description with some tabulation for frequency of case study use. PROMOTING TRANSFORMATIONS: TRANSLINKS PERFORMANCE EVALUATION Page xxxix EVALUATION QUESTION TYPE OF ANSWER/ EVIDENCE REQUIRED COLLECTION METHOD SOURCE OF DATA SAMPLING/ SELECTION DATA ANALYSIS METHODS Have these case studies affected the development of innovations in PES schemes? If so, how? - Analysis of PES development generally and in case study countries - TL PES outcomes As above As above As above As above