This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Integra Government Services International, LLC. The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. LESTARI INDONESIA MID-TERM EVALUATION May, 2018 iii LESTARI MID-TERM EVALUATION USAID.GOV CONTENTS TABLE OF ACRONYMS V EXECUTIVE SUMMARY 1 THE PROJECT 1 THE EVALUATION 1 PROGRESS 1 VALIDITY OF STRATEGIC APPROACHES AND THE RESULTS FRAMEWORK 3 PERFORMANCE 3 LESSONS 4 RECOMMENDATIONS 4 CONCLUSION 5 I. EVALUATION PURPOSE AND EVALUATION QUESTIONS 6 EVALUATION QUESTIONS 6 TECHNICAL COMPONENT 1: FOREST AND LAND USE GOVERNANCE AND ADVOCACY 6 TECHNICAL COMPONENT 2: IMPROVED MANAGEMENT OF CONSERVATION AREAS 7 TECHNICAL COMPONENT 3: IMPROVED PRIVATE SECTOR PRACTICES 7 PROGRAM DESIGN AND PERFORMANCE 7 MANAGEMENT EFFECTIVENESS AND SUSTAINABILITY 7 II. PROJECT BACKGROUND 8 III. EVALUATION METHODS AND LIMITATIONS 10 EVALUATION TOOLS AND DATA SOURCES 11 DATA ANALYSIS AND SYNTHESIS 11 Q1: TO WHAT EXTENT ARE LESTARI OUTREACH ACTIVITIES BUILDING CONSTITUENCIES IN SUPPORT OF BIODIVERSITY AND FOREST CONSERVATION? 13 Q2: TO WHAT EXTENT HAS LESTARI BEEN ABLE TO PREVENT CONVERSION OF CRITICAL AREAS TO OTHER PURPOSE AREAS? 15 Q3: TO WHAT EXTENT ARE LESTARI POLICY RECOMMENDATIONS INCORPORATED INTO GOVERNANCE DOCUMENTS AT THE DISTRICT, PROVINCE, AND NATIONAL LEVELS? 17 Q4: TO WHAT EXTENT ARE MULTI-STAKEHOLDER FORUMS INFLUENCING LAND USE DECISION￾MAKING, MONITORING AND PLANNING? 18 Q5: TO WHAT EXTENT HAVE LESTARI ACTIVITIES LED PROVINCIAL AND DISTRICT GOVERNMENTS TO EFFECTIVELY MONITOR LAND USE AND LICENSING? 19 Q6: TO WHAT EXTENT IS LESTARI EFFECTIVELY ENCOURAGING MANAGEMENT OF CONSERVATION AREAS (E.G. CO-MANAGEMENT, FMUS, METT, ETC.)? 21 iv LESTARI MID-TERM EVALUATION USAID.GOV Q7: TO WHAT EXTENT ARE SMART PATROLS PREVENTING WILDLIFE TRAFFICKING AND POACHING? 23 Q8: TO WHAT EXTENT ARE LESTARI PUBLIC-PRIVATE PARTNERSHIPS SUCCESSFUL IN INCREASING ENVIRONMENTALLY SUSTAINABLE COMMODITY PRODUCTION? 25 Q9: TO WHAT EXTENT ARE THESE PARTNERSHIPS REDUCING DEFORESTATION AND FOREST DEGRADATION? 28 Q10: TO WHAT EXTENT HAS LESTARI INCREASED ACCESS TO AND USE OF LONG TERM FINANCING FOR CONSERVATION AND RESTORATION? 29 Q11: HOW IS THE IMPLEMENTATION STRUCTURE AND MANAGEMENT OF LESTARI CONTRIBUTING TO ITS SUCCESSES AND ARE THERE AREAS FOR IMPROVEMENT? 30 Q12: TO WHAT EXTENT ARE LESTARI ACTIVITIES UNDERTAKEN WITH A FOCUS ON SUSTAINABILITY OF RESULTS AFTER THE PROJECT IS CLOSED? 31 Q13: TO WHAT EXTENT IS THE THEORY OF CHANGE STILL VALID? 32 Q14: TO WHAT EXTENT ARE LESTARI ACTIVITIES ALIGNED WITH THE THEORY OF CHANGE? 33 Q15: TO WHAT EXTENT IS THERE BUY IN FOR LESTARI INTERVENTIONS ACROSS THE GOI (NATIONAL, PROVINCIAL, LOCAL, COMMUNITY)? 38 Q16: TO WHAT EXTENT IS USAID MANAGEMENT OF LESTARI CONTRIBUTING TO ITS SUCCESSES AND ARE THERE AREAS FOR IMPROVEMENT? 38 5. CONCLUSIONS 40 GOVERNANCE AND ADVOCACY 40 IMPROVED MANAGEMENT OF CONSERVATION AREAS 40 PRIVATE SECTOR ENGAGEMENT 41 6. RECOMMENDATIONS 43 A) TECHNICAL AREA: GOVERNANCE AND ADVOCACY 43 B) TECHNICAL AREA: IMPROVED CONSERVATION MANAGEMENT 43 C) TECHNICAL AREA: PRIVATE SECTOR ENGAGEMENT 44 D) MANAGEMENT EFFECTIVENESS AND SUSTAINABILITY 46 ANNEX I: EVALUATION SCOPE OF WORK 47 ANNEX II: LESTARI MTE FINAL WORKPLAN AND APPROVED ADDENDUM 60 ANNEX III: LESTARI MTE SUMMARY OF FINDINS BY LANDSCAPE 76 ANNEX IV: EVALUATION MATRIX 113 ANNEX V: TEAM COMPOSITION 116 ANNEX VI: ITINERARY 120 v LESTARI MID-TERM EVALUATION USAID.GOV TABLE OF ACRONYMS ADS United States Agency for International Development (USAID) Automated Directive System BAPPEDA Regional Development Planning Board (Bappeda Kota Lhokseumawe) BAPPENAS National Development Planning Ministry BIJAK USAID Build Indonesia to Take Care of Nature for Sustainability project (Bangun Indonesia untuk Jaga Alam demi Keberlanjutan) BKPRD Spatial Planning Coordinating Board BKSDA Nature Conservation Agency of Indonesia (Balai Konservasi Sumber Daya Alam) BUMDes Village-owned enterprise (Badan Usaha Milik Desa) BRG Peat Restoration Agency of Indonesia (Badan Restorasi Gambut) CARPE Central Africa Regional Program for the Environment (USAID) CCLA Community Conservation and Livelihood Agreement FGD Focus Group Discussion FPIC Free prior and informed consent FMU Forest Management Unit FSC Forest Stewardship Council (certification body) GHG greenhouse gasses GOI Government of Indonesia HCV High conservation value (applied to forests) IFACS USAID/Indonesia Forest and Climate Support Project IP Implementing Partner(s) KII Key Infomant Interview METT Management Effectiveness Tracking Tool MoEF Ministry of the Environment and Forestry NCBA National Cooperative Business Association (affiliate of CLUSA International) NRM Natural resources management PADIATAPA Agreement on Early Information (Persetujuan atas Dasar Informasi di Awal Tanpa Paksaan) PPP Public/private partnership PTSP Provincial One-Stop Service Agency (Pelayanan Terpadu Satu Pintu) RIL-C Reduced impact logging for reduced GHG emissions RPJMDes Village Level Medium Term Development Plan (Rencana Pembangunan Jangka Menengah Desa) SFB Sustaining Forests and Biodiversity (USAID/Cambodia project) SIMTARU Spatial Planning Information Management System (Sistem Informasi Tata Ruang) SMART Spatial Monitoring and Reporting Tool SST Sustainability Screening Toolkit STEWARD Sustainable and Thriving Environments for West African Regional Development (USAID) UNOPS United Nations Office for Project Services USAID United States Agency for International Development USFS United States Forest Service WRI World Resources Institute vi LESTARI MID-TERM EVALUATION USAID.GOV 1 LESTARI MID-TERM EVALUATION USAID.GOV EXECUTIVE SUMMARY THE PROJECT USAID’s LESTARI project supports the Government of Indonesia to reduce greenhouse gas (GHG) emissions and conserve biodiversity in carbon rich and biologically significant forest and mangrove ecosystems. USAID LESTARI applies a landscape approach, integrating forest and peatland conservation with low emissions development on other, already degraded land. USAID LESTARI activities are targeted in six strategic landscapes across Aceh, Central Kalimantan, and Papua. The overall goal of LESTARI is that, at the end of five years, USAID assistance will have 1) reduced land-based greenhouse gas (GHG) emissions, and 2) conserved biodiversity in carbon-rich and biologically significant forest and mangrove ecosystems within selected landscapes. LESTARI is based on a systems approach, integrating governance, risk management, institutional development, and private sector engagement. It has made a substantial contribution to stakeholder engagement by integrating Free Prior and Informed Consent (FPIC) into a range of activities involving land use. THE EVALUATION The mid-term evaluation of LESTARI is required under Automated Directive System (ADS) 201 as part of the project and programming cycle. For LESTARI, evaluators were asked to examine the following issues: • Assess progress towards achieving LESTARI goals; • Assess the validity of program strategies, approaches, and assumptions; • Assess program performance by Implementing Partners; • Identify lessons learned • Recommend actions to improve performance A team of six experts in biodiversity, forest management, private sector engagement, and social dimensions of conservation conducted the evaluation, working in two teams of three. The evaluators visited five of the six LESTARI landscapes; Leuser, Katingan-Kahayan, Cyclops, the Lorentz Lowlands, and Mappi-Bouven Digoel. They omitted the Sarmi landscape because of its limited access and relatively low number of beneficiaries and observable activities. PROGRESS Free Prior and Informed Consent (FPIC) has the potential to transform the relationships between the government, civil society, and the private sector. For example, LESTARI has made significant progress in the use of participation in village-level, medium-term development planning (RPJMDes). In turn, citizen participation is helping shift the focus of government development investment from strictly infrastructure to a more holistic approach that includes better management of forests and other natural resources, taking into account local perspectives. 2 LESTARI MID-TERM EVALUATION USAID.GOV LESTARI’s outreach and advocacy efforts are raising public awareness of the linkages between resource management and rural vulnerability. In the focal areas, we found significantly increased understanding that unsustainable natural resource management--including deforestation, wetland drainage, and illegal hunting—increase the vulnerability of communities and families. Armed with this knowledge, communities are now becoming effective constituencies in advocating for better land use policy. LESTARI strikes a responsive chord in focal landscape communities, since they have experienced environmental disasters such as floods and fires. The evaluation team found that LESTARI has been agile in taking advantage of the public concern and demand for action to advance restoration and improve land management in response to the 2015 fire emergency. LESTARI has also laid foundations for improvements in forest management through improved information for decision-making and targeted policy support at the provincial and district levels. Land-use monitoring efforts have had some influence on high-level decision-making. LESTARI is working to build government capacity to manage forests through engaging with Forest Management Units (FMUs) in LESTARI landscapes, including through the implementation of toolkits developed with the support of the US Forest Service. Institutional development of Indonesia’s forest management system is clearly a worthy goal. This major challenge requires a deliberate, stepwise process based on USAID’s Human and Institutional Capacity Development Policy, including planning for financial sustainability. While the evaluation team was unable to find a strong correlation between the use of the spatial monitoring and reporting tool (SMART) and reduction of wildlife crime, they did observe that the collection and use of SMART data is contributing to protected-area management in enforcement planning, and in the documentation and mapping of important data points such as encroachment incidents and wildlife sightings. LESTARI is improving the management of conservation areas. One example is the use of the Management Effectiveness Tracking Tool (METT). METT is a self-assessment scorecard for conservation area managers, which encouraged managers to focus on continuous improvement. LESTARI’s co-management activities, such as community ranger patrols, are giving communities a sense of ownership of conservation areas, thus contributing to improvements in management. Using domesticated elephants for perimeter patrols in the Rawa Singkil Wildlife Reserve is helping to resolve serious human/wildlife conflicts involving crop predation by wild elephants. The patrol elephants have helped to contain the wild population within the conservation area boundaries, resulting in greater support for conservation on the part of the communities and better protection for the wild elephants. LESTARI’s progress on private sector engagement has been slow. Successes in this domain often involve small scale livelihood-focused interventions with community-based producers, for example organic cacao in Leuser and sustainable rubber production in Katingan-Kahayan. Reduced impact logging (RIL-C) is also making headway. More ambitious initiatives, such as payment for ecosystem services schemes involving water or ecotourism, remain largely on the drawing board at the halfway point in the project. 3 LESTARI MID-TERM EVALUATION USAID.GOV A breakthrough appears possible from current discussions with the industrial conglomerate Korindo, which holds major oil-palm concessions in Papua. Korindo is feeling the heat from the international conservation community for its controversial forest-burning for land clearance in 2015. LESTARI is promoting a FPIC-based approach to concession management in the Mappi-Bouven Digoel landscape, involving substantial forest set-asides for cultural and conservation purposes, in which communities would work with concession managers to develop and implement a land-use plan. Such an approach would be beneficial to Korindo in meeting financial and conservation targets for certification of its palm oil by the Roundtable for Sustainable Palm Oil (RSPO). If successful, it could establish good practice for the oil palm industry, and if replicated, protect substantial amounts of at-risk, high-conservation value (HCV) forest as well as culturally significant forests for communities in and around concessions. Since many forest and oil palm concessions in Papua have yet to become operational, the timing is propitious. Any risks associated with engagement with the powerful oil palm industry must be weighed against the risks of not meeting ambitious project targets. VALIDITY OF STRATEGIC APPROACHES AND THE RESULTS FRAMEWORK This theory of change articulates a demand-driven approach to conservation, based upon awareness of linkages between goals and long-term interests of stakeholders, participation in management of resources, and private sector engagement. This is consistent with USAID good practice and in response to top-down approaches to natural resource governance. While the theory of change is valid, hidden assumptions are identified in this report concerning continuity of government policy, institutional sustainability and private sector motives, that produce some leaps of faith in the implementation. Prospects for long-term sustainability of some project gains remain unclear. Stakeholders appear to be in some denial about the fact that LESTARI will end in the foreseeable future. LESTARI leadership is mindful of this issue and of the importance of their sustainability planning -- including a local system for maintaining and updating databases, as well as training staff and volunteers in their use. Communication of the steps in the transition will be important to effective transfer of responsibilities to counterpart organizations. PERFORMANCE The performance of USAID’s Implementing Partner in the management of LESTARI has been solid, with a senior management team that is well-regarded by counterparts in the field and by the staff, careful work planning using adaptive management practices, and strong relations with counterpart organizations. LESTARI is a manifestation of the long-term Comprehensive Partnership between the USA and the Republic of Indonesia. The Joint Statement issued on October 26, 2015, on the occasion of the state visit of President Joko Widodo to Washington, affirmed the preservation of peatlands and other high￾carbon landscapes as a priority area of cooperation between the two countries. USAID management of LESTARI reflects the high profile of this activity as the implementation of the commitment between the two countries. USAID has been very supportive of LESTARI, both in terms of securing high-level commitment from the government, and in terms of technical assistance provided by the Forest and Biodiversity office of USAID as well as from the Department of the Interior and the Department of Agriculture. The evaluation team noted levels of involvement in direct project management beyond what is considered appropriate for contract implementation; this issue was substantially addressed before the evaluation was completed. 4 LESTARI MID-TERM EVALUATION USAID.GOV LESSONS Taking success to scale is an important challenge for LESTARI. Interventions in improving private sector practices are still largely limited to pilot projects. While generating important lessons and providing proofs of concept, by themselves they will have negligible impact against the powerful drivers of deforestation and biodiversity loss. RECOMMENDATIONS Governance and Advocacy One possible strategy for taking success to scale is to mainstream the lessons of LESTARI through the network of the Bupatis with whom LESTARI has enjoyed a strong working relationship. This may help to consolidate and institutionalize LESTARI innovations even as the forest management structure of government evolves. LESTARI can maximize its sustainability and impact by capitalizing on the successes of the RPJMDes process. Participating in this process has increased the confidence of villagers to actively shape their future. Success can be capitalized upon through communication, replication, and integration. Better support for market participation of those communities engaged in livelihood activities is recommended, with targeted capacity building to improve market awareness. Improved conservation management Independent, third-party assessment using the METT scorecard would help to provide a more accurate picture of management effectiveness in conservation areas. If BAPPENAS, in cooperation with relevant line ministries, were to lead the METT process, substantial internal learning could take place within the Government of Indonesia, and a similar scorecard approach could be adapted elsewhere. LESTARI should continue SMART as a management tool, beyond the original purpose of combating wildlife crime. If combating wildlife crime is to continue to be a priority outcome, alternative strategies consistent with LESTARI’s landscape-level approach are available, such as situational crime prevention. Private sector engagement The evaluation team observed poor understanding on the part of local partners, including grantees, of feasibility and risk in economic development activities generally. To increase the relevance of commodity production on sustainable livelihoods, LESTARI should develop a thorough assessment of local economic development opportunities and provide corresponding technical advice in terms of feasibility, risk, and tradeoffs for any commodity production promoted. Information produced through the Project Evaluation Tool to assess ecotourism feasibility in year one apparently did not propagate throughout the LESTARI network and additional work with this tool is not reflected in the year 2 annual report. LESTARI should increase efforts to engage with the private sector and with not-for-profit entities working to promote environmental and social governance and green investment. This provides an important opportunity to advance its integration of FPIC into land use planning. LESTARI should also be willing to abandon non-productive initiatives to mobilize conservation finance. The lack of progress on 5 LESTARI MID-TERM EVALUATION USAID.GOV payment for ecosystem services schemes suggest that they should be re-examined and dropped if there is no likelihood of achieving measurable impacts. Important progress may be underway in the plans for a fund for the Lorentz Lowlands. LESTARI should pay particular attention to the governance dimensions of this fund, to ensure that it does promote community buy-in and does not facilitate “policy capture” or undue influence of private sector partners in conservation decisions. Management effectiveness and sustainability LESTARI may need to improve responsiveness with partners, including response time on decisions. This can be done through establishment of standards and boundaries around the amount of time allowed to elapse before LESTARI responds to a formal request from a partner. Activities that may not be sustainable should be assessed and discontinued or refocused. The evaluation team is concerned about the uptake of the spatial planning tools, in particular. CONCLUSION The bottom line: LESTARI has already helped stakeholders and partners in the focus landscapes to systematize public involvement in planning, design, and implementation of natural resource management activities. The next step in a holistic landscape approach is to tie together the strands of spatial planning, economic development, and conservation that too often appear disconnected, and to take pilot initiatives to scale. LESTARI can accomplish this through the development of a clearly articulated strategy and timetable for taking pilot programs to scale that integrate these strands. Strands that cannot be integrated should be considered for discontinuation. LESTARI was built upon the foundations of the processor activity, the Indonesia Forests and Climate Support activity (IFACS). During IFACS’ implementation, concerns were raised as early as the mid-term evaluation about the lack of effective engagement with the government agencies that control large areas of forest. LESTARI attempts to strike an appropriate balance between interventions at local, regional, and national levels. Maintaining an appropriate balance is essential if LESTARI is to build upon its successes in forest governance, much of which occurs at the local level. As a result of LESTARI’s contributions, USAID has an opportunity to engage with the Government of Indonesia and other stakeholders to establish forest conservation as a critical strategy for development, including for disaster risk management. In the rapidly shifting policy environment, disaster risk management will remain a constant for Indonesia. Awareness of the risks from land use and land cover change may help to counterbalance economic strategies based on deforestation and land use change. 6 LESTARI MID-TERM EVALUATION USAID.GOV I. EVALUATION PURPOSE AND EVALUATION QUESTIONS This mid-term evaluation was conducted to provide USAID/Indonesia and the Government of Indonesia’s Ministry of Environment and Forestry (MoEF) with an independent review of progress toward the program’s targets for reductions in greenhouse gas emissions and conservation of forests and biodiversity. The evaluation is intended to help USAID/Indonesia, Government of Indonesia counterparts at national and subnational levels, implementing partners and key stakeholders to assess the program and incorporate recommendations into the strategy, implementation and monitoring of LESTARI. The evaluation focused on program design and implementation, program performance and progress, program management and coordination, prospects for sustainability, and recommendations for improving the design, performance and management of the program. The LESTARI program has three technical components, with eight specific strategies derived from the Results Framework for the program. The objectives of the midterm evaluation are: • Assess progress towards achieving LESTARI goals and objectives; • Assess the validity of LESTARI’s strategic approaches and results framework; • Assess program performance by Implementing Partners; • Identify lessons learned • Recommend actions to improve performance, strategy, and management EVALUATION QUESTIONS Evaluation questions covered each technical component of the LESTARI Midterm Evaluation workplan as accepted. At the inception meeting, the evaluation team and USAID/Indonesia modified the questions from the initial scope of work for improved clarity and relevance. They are clustered around the three LESTARI technical components, management effectiveness, and program design and performance. TECHNICAL COMPONENT 1: FOREST AND LAND USE GOVERNANCE AND ADVOCACY 1) To what extent are LESTARI outreach activities building constituencies in support of biodiversity and forest conservation? 2) To what extent has LESTARI been able to prevent conversion of critical areas to other purpose areas? 3) To what extent are LESTARI policy recommendations incorporated into governance documents at the district, province, and national levels? 4) To what extent are Multi-Stakeholder Forums influencing land use decision-making, monitoring and planning? 5) To what extent have LESTARI activities led provincial and district governments to effectively monitor land use and licensing? 6) To what extent is LESTARI effectively encouraging management of conservation areas (e.g. co￾management, FMUs, METT, etc.)? 7 LESTARI MID-TERM EVALUATION USAID.GOV TECHNICAL COMPONENT 2: IMPROVED MANAGEMENT OF CONSERVATION AREAS 7) To what extent are SMART patrols preventing wildlife trafficking and poaching? 8) To what extent are LESTARI public-private partnerships successful in increasing environmentally sustainable commodity production? 9) To what extent are these partnerships reducing deforestation and forest degradation? TECHNICAL COMPONENT 3: IMPROVED PRIVATE SECTOR PRACTICES 10) To what extent has LESTARI increased access to and use of long-term financing for conservation and restoration? 11) How is the implementation structure and management of LESTARI contributing to its successes and are there areas for improvement? 12) To what extent are LESTARI activities undertaken with a focus on sustainability of results after the project is closed? PROGRAM DESIGN AND PERFORMANCE 13) To what extent is the theory of change still valid? 14) To what extent are LESTARI activities aligned with the theory of change? MANAGEMENT EFFECTIVENESS AND SUSTAINABILITY 15) To what extent is there buy-in for LESTARI interventions across the GOI (national, provincial, local, community)? 16) To what extent is USAID management of LESTARI contributing to its successes and are there areas for improvement? 8 LESTARI MID-TERM EVALUATION USAID.GOV II. PROJECT BACKGROUND USAID Indonesia has a long history of supporting natural resource management (NRM) and forest conservation, from the NRM and NRM II projects through USAID’s Indonesia Forest and Climate Support (IFACS) project, which ended in September 2015. In response to the new era in forest governance, decentralization, and regional autonomy in Indonesia, USAID began to work more closely with 13 District Governments in the Provinces of Aceh, West Kalimantan, Central Kalimantan, and Papua. The Agency identified eight focus landscapes (see below) that contain both 1) areas of high biodiversity, including habitats of key species such as the orangutan, and 2) carbon-rich, forested areas under immediate threat from conversion and degradation. The majority of these sites represented new areas for USAID assistance. Work under this project has now built the foundation for sustained USAID investment in these landscapes by: • Establishing relationships with key local stakeholders; • Building basic government capacity for low emissions development; and • Establishing institutional working arrangements with districts, the private sector, and communities. Through IFACS, USAID sought to engage key members of the private sector, district-level government, and communities to ensure that all stakeholders were represented in Multi-Stakeholder Forums (MSF); and that decisions were made transparently. In addition, the project delivered significant levels of training and technical assistance to district governments and stakeholders to develop Strategic Environmental Assessments (SEA). These SEAs make use of modern geospatial technology for improved and more transparent planning and decision￾making. USAID also helped Conservation Management Area Units (National Parks and the Nature Conservation Agency, known by its Indonesian initials – BKSDA) improve the capacity of forest ranger special units (SPORC) in monitoring and combating wildlife crime. The Agency also facilitated collaborative, conservation-area-management initiatives between communities and park management. Within villages adjacent to forest, USAID combined alternative community livelihoods development in forest-friendly products such as cocoa, nutmeg, and patchouli with conservation, through the promotion of Community Conservation and Livelihood Agreements (CCLA). USAID also made initial forays into engaging with the private sector by facilitating agreements to manage high conservation value areas in 13 timber concessions, 2 palm oil concessions, and 1 mining concession. USAID’s IFACS project was effective in initiating improved land use planning processes within district governments, building a basic understanding of the importance of terrestrial conservation and climate change among a variety of stakeholders, engaging private sector partners, and enhancing conservation￾oriented community livelihoods in improved land-use planning. USAID seeks to build on and sustain these results through LESTARI. However, the mid-term evaluation of IFACS highlighted various challenges and gaps, including an overambitious geographic reach, the lack of a true landscape approach because of the exclusion of conservation areas, and the need to enhance stakeholder participation and advocacy. Therefore, while 9 LESTARI MID-TERM EVALUATION USAID.GOV building on the USAID IFACS project’s foundational successes, USAID’s LESTARI Project focuses on a smaller number of landscapes, expanding its reach to cover key conservation areas within those landscapes, and emphasizing advocacy, co-management, and community participation. In following the Agency’s emphasis on sound design through evidence-based programming, clear cause￾and-effect linkages, and improved monitoring for necessary adaptive management and measurement of impacts, USAID/Indonesia has developed an overall LESTARI project situation model. The LESTARI model identifies the intended ecosystem focal interests and associated pressures, drivers and illustrative interventions. Each intervention pathway is considered a “Development Hypothesis” that can be more closely examined to define specific intermediate subtasks that need to be taken. This is consistent with best practice in the implementation of the USAID Program Cycle and is based on the Open Standards for the Practice of Conservation, which is the result of USAID work with the conservation community over the last few decades. Because mitigating land-based carbon emissions and conserving terrestrial biodiversity both ultimately aim at conserving key terrestrial ecosystems that are under severe threat, the conceptual model works for both objectives. Since the majority of LESTARI’s funding comes from Congressional earmarks for biodiversity conservation, the Contractor must explicitly show its strategies, targets, and measurements to achieve biodiversity objectives, with a particular focus on targeted areas of high biodiversity within conservation areas. The overall goal of USAID’s LESTARI project is that, at the end of five years, USAID assistance will have 1) reduced land-based greenhouse gas (GHG) emissions, and 2) conserved biodiversity in carbon-rich and biologically significant forest and mangrove ecosystems within selected landscapes. • Key results that must be achieved by the completion of the project are: • At least 41 percent of total CO2 equivalent emissions reduced from land use, land use change and deforestation averaged across all landscapes within the project scope, based on the use of the IPCC Good Practice Guidance for Land Use, Land-Use Change and Forestry, and taking into consideration appropriate relevant national or subnational methods; • At least 8.42 million hectares of primary or secondary forest, including orangutan habitat, under improved management; • Management of at least six conservation areas improved, resulting in the conservation of valuable orangutan and other key species habitat, and the reduction in poaching of threatened and endemic species; • At least ten public-private partnerships (PPPs) promoting low-emissions conservation oriented development established; • Funding leveraged from public and private sources, representing co-investment in project outcomes; • Increased commitment of key private sector, government, and community stakeholders regarding the positive benefits of conservation and sustainable use of forests and the species they encompass; and • Policies, laws, regulations, and procedures in support of low emission development and forest conservation and management increased, promulgated, and enforced at all levels. 10 LESTARI MID-TERM EVALUATION USAID.GOV • Model(s) for successful integration of district, provincial, and national low emissions development and forest conservation strategies developed and shared at all levels of government and with other key stakeholders. III. EVALUATION METHODS AND LIMITATIONS This evaluation focuses on LESTARI performance with respect to expected results and objectives by technical component; program design and management; the prospect of long-term sustainability; and practical recommendations for performance improvement and strategic planning. Integra employed a non-experimental evaluation design and a mixed-methods approach, combining qualitative and quantitative research methods and analysis. This design is intended to allow USAID and its stakeholders to understand the effectiveness of LESTARI programming and build a knowledge base to inform decision making, enabling USAID to test the key hypotheses behind the overarching results framework’s theory of change. As part of the evaluation design deliverable, the evaluation team produced a detailed evaluation matrix (Annex IV). This matrix linked each evaluation question with proposed data sources, including required documents for review, targeted key informants and stakeholder groups for interviews and focus groups and the justification and constraints on the evaluation’s analytical methods and data. Data collected was intended to establish, to the extent possible, progress towards LESTARI’s overall goals and objectives, as well as the soundness of the strategic approaches and additional questions related to operational efficiency and effectiveness of LESTARI and its implementing partners. Analysis sought to capture the quantitative changes in objective outcomes across all three technical evaluation components, the qualitative changes in perceptions around conservation, land use and economic opportunities, and the processes that have shaped the implementation of the activity. Given the logistical challenges associated with travel in Indonesia, field visits were managed between two groups, ensuring adequate time at each landscape to gather robust data from various perspectives on each evaluation question. The evaluation focused on meeting with implementing partner teams; national and local government representatives and parks authorities; the private sector; civil society representatives working in the area of land rights, economic growth, biodiversity and conservation; community members in selected landscapes; and academic institutions, where applicable, to gather relevant information that was used for triangulation to obtain confirmation of findings and results. An additional limitation is that USAID’s specifications for the evaluation team called for senior level experts in technical fields such as finance. This approach conflicts to some extent with the data collection process because of the interests and biases of the experts, resulting in a focus on areas of professional interest and unevenness in overall data collection. This was mitigated to some extent through coaching during the inception of the enumeration period. In the future, it may be prudent to engage technical experts only in the design of the evaluation instrument and in interpretation of findings and leave the data collection to trained generalists or skilled enumerators, where available. 11 LESTARI MID-TERM EVALUATION USAID.GOV 8 7 8 4 10 Focus Group Discussions per Landscape Leuser Katingan MAPPI-Boven Cyclops Lorenta EVALUATION TOOLS AND DATA SOURCES The evaluation produced qualitative information and quantitative data from the following sources: • Interviews with Key Informants at various levels of interlocutors, including local government representatives; • Document Review including LESTARI and GoI reports and assessments, academic literature, international environmental NGOs and expert sources in Indonesia; • Focus Group Discussions with direct project beneficiaries and community members, providing an opportunity to assess gender equality and women’s empowerment aspects of the evaluation; and • Institutional Analysis, supported by information gleaned through a variety of data sources including direct observation and secondary data. All data provided by the various tools were triangulated with contextual and institutional information, with the quantitative data supporting the qualitative context under which LESTARI is implemented. The numbers of interviews, focus groups and other collected data are shown by landscape and in total in Table 1. Table 1: Data Sources TOTAL Leuser Katingan Kahayan Cyclops Lorentz Mappi Bouven Number of Key Informant Interviews (KIIs) 83 23 22 15 10 13 Focus Group Discussions 37 8 7 8 4 10 Narrative reports of unstructured interviews 32 Literature review – number of documents ~40 DATA ANALYSIS AND SYNTHESIS Data was analyzed and synthesized through a structured integration of quantitative indicators/data with qualitative data and direct observations supported by data from LESTARI’s baseline, monitoring plan, secondary documentation and expert inputs. For evaluation questions that address the sufficiency of activities or the effectiveness of various interventions, quantitative targets established in LESTARI’s annual work plans and monitoring plans were assessed by the team’s technical specialists to determine the progress in achieving desired outcomes, and the sufficiency of activities. Table 2, below, summarizes evaluation question responses by landscape, taking into account the sources identified per question in the evaluation matrix, and 12 LESTARI MID-TERM EVALUATION USAID.GOV Key Informant Interviews by Sector Government NGO Company Community/Indig Group Media Implementing partners Others (religious, academic) submitted as part of the evaluation design and workplan. These are scores of progress toward meeting targets. For example, most of the questions in technical component 3, Improved Private Sector Practice, are shaded yellow in the Papuan landscapes (Cyclops, Lorentz, and Mappi) because these activities started later than did the activities in Leuser and Katingan, and are still in the early stages of development, where there is not sufficient evidence to determine how well these activities will perform. It is important that the reader not interpret the scorecard as a “pass or fail” judgment. That is not the purpose of this evaluation. To the extent possible, the evaluation team used quantitative data, and endeavored to demonstrate the methods for interpretation of opinions and perceptions from interviews and group discussions. The evaluation team used a gender and social inclusion expert in the design of the evaluation to ensure that gender, indigenous community and socioeconomically disadvantaged group impacts are integrated into the evaluation findings. Table 2 summarizes findings of the evaluation team by question. • RED cells indicate that work has not been initiated, or, if initiated, that the evaluation team identified challenges that are discussed in the narrative. • YELLOW cells indicate only that the work is still in initial stages, and otherwise provides too little information to effectively evaluate • GREEN cells indicate that the work is underway and on track. • GREY cells indicate that a question is not applicable to the respective landscape Table 2: Scorecard for the Evaluation, by question Evaluation Questions Leuser Katingan Cyclops Lorentz Mappi Technical Component 1: Forest and Land Use Governance and Advocacy 1 To what extent are LESTARI outreach activities building constituencies in support of biodiversity and forest conservation? 2 To what extent has LESTARI been able to prevent conversion of critical areas to other purpose areas? 3 To what extent are LESTARI policy recommendations incorporated into governance documents at the district, province, and national levels? 4 To what extent are Multi-Stakeholder Forums influencing land use decision-making, monitoring and planning? 5 To what extent have LESTARI activities led provincial and district governments to effectively monitor land use and licensing? 6 To what extent is LESTARI effectively encouraging management of conservation areas (e.g. co-management, FMUs, METT, etc.)? N/A Technical Component 2: Improved Management of Conservation Areas 7 To what extent are SMART patrols preventing wildlife trafficking and poaching? N/A 13 LESTARI MID-TERM EVALUATION USAID.GOV 8 To what extent are LESTARI public-private partnerships successful in increasing environmentally sustainable commodity production? 9 To what extent are these partnerships reducing deforestation and forest degradation? Technical Component 3: Improved Private Sector Practices 10 To what extent has LESTARI increased access to and use of long￾term financing for conservation and restoration? 11 How is the implementation structure and management of LESTARI contributing to its successes and are there areas for improvement? 12 To what extent are LESTARI activities undertaken with a focus on sustainability of results after the project is closed? Program Design and Performance 13 To what extent is the theory of change still valid? 14 To what extent are LESTARI activities aligned with the theory of change? Management Effectiveness and Sustainability 15 To what extent is there buy-in for LESTARI interventions across the GOI (national, provincial, local, community)? 16 To what extent is USAID management of LESTARI contributing to its successes and are there areas for improvement? Q1: TO WHAT EXTENT ARE LESTARI OUTREACH ACTIVITIES BUILDING CONSTITUENCIES IN SUPPORT OF BIODIVERSITY AND FOREST CONSERVATION? Evaluation Questions Leuser Katingan Cyclops Lorentz Mappi Technical Component 1: Forest and Land Use Governance and Advocacy 1 To what extent are LESTARI outreach activities building constituencies in support of biodiversity and forest conservation? RESPONSE LESTARI has been successful at building constituencies in support of biodiversity and forest conservation. The key question is whether or not these constituencies can scale during the project to achieve the primary project objectives of improved conservation management and reduced greenhouse gas emissions. The evaluation team found strong evidence that constituency building is scaling in most landscapes. Significantly, LESTARI is helping advocates to integrate biodiversity and forest conservation in the context of livelihoods, resource rights, and customary use. FINDINGS LESTARI has developed media partners that are using mass media for awareness-raising and conservation advocacy. Methods and tools vary by landscape to take into account local capacities and interests. In addition, support is provided in the form of training and technical assistance for journalists and advocacy NGOs, engagement of multistakeholder groups, and community empowerment. Journalism training, for example, was provided in the Leuser landscape for 40 journalists from 5 districts in the Leuser landscape, In Aceh, LESTARI supports a women-led social media campaign. In Katingan￾Kahayan, education and awareness is being developed through articles and advertisements in mass media, radio, as well as social media. For example, LESTARI has supported a regular radio talk show about free prior and informed consent, land use, and fire prevention. In addition to awareness-raising through mass media and social media, LESTARI outreach includes direct engagement with priority communities, primarily in areas at risk of forest degradation or deforestation, through community planning. LESTARI’s approach links the economic impacts of poor resource 14 LESTARI MID-TERM EVALUATION USAID.GOV management to extreme weather events, which is manifested in in the form of flood and fire. In the Leuser and Katingan-Kahayan landscapes in particular, RPJMDes planning has increased understanding of greater environmental issues and disasters they face. One such environmental issue is the prevalence of serious fires. These have become both more visible and politicized, both nationally and internationally. LESTARI took strong advantage of this in building constituencies for habitat/peatland restoration. “Fire prevention” is LESTARI’s entry point for building awareness of biodiversity in the Katingan; for example, for orangutan conservation, a video of fire and its effects on orangutans sent a powerful message. Katingan-Kahayan community outreach, through RPJMDes planning, increased understanding of environmental issues to disasters, and linkages between disaster risk management, conservation, and economic improvement (e.g. through rubber and sengon1 production). Cyclops is marked as yellow in this evaluation question. There has been outreach to the general public through radio and newspapers. The project has conducted some targeted outreach to priority communities such as internal migrants from other parts of Papua. Because community outreach work is limited in scope at this point, occurring only through SMART Patrols and a vanilla production initiative, evaluators determined that there was not yet sufficient evidence that this work is on track. In the Lorentz Lowlands, LESTARI has supported an active coalition involving a civil society group (Papua Animal Care), the local office of a government agency (BKSDA), and the private sector (PT Freeport Indonesia) on wildlife conservation in Mimika. Mappi’s progress in this area is relatively recent and there is not enough information to evaluate progress in answering this question. It is therefore marked as yellow. LESTARI’s ability to follow up on outreach with technical assistance for concrete actions that increased local control of resources, either directly or through small grants and partnerships, resulted in highly mobilized constituencies. The process of introducing social forestry has increased the communities’ understanding of conservation as it relates to their everyday interests. DATA SOURCES Data included literature review, specifically reporting on indicators, key informant interviews with community leaders, journalists, and NGOs, and focus group discussions with community groups. 1 A softwood tree in the Mimosa family grown in plantations for plywood production. 15 LESTARI MID-TERM EVALUATION USAID.GOV Q2: TO WHAT EXTENT HAS LESTARI BEEN ABLE TO PREVENT CONVERSION OF CRITICAL AREAS TO OTHER PURPOSE AREAS? Evaluation Questions Leuser Katingan Cyclops Lorentz Mappi Technical Component 1: Forest and Land Use Governance and Advocacy 2 To what extent has LESTARI been able to prevent conversion of critical areas to other purpose areas? RESPONSE It is premature to draw conclusions about the ability of LESTARI to prevent conversion of critical areas at the mid-term mark for the project. Evidence from remote sensing2 shows continuing forest degradation throughout the landscapes, but this doesn’t tell the whole story. There are many drivers of conversion, including, among other things, national economic policies (e.g., regarding mining and road construction), extreme weather events (especially the drying linked to the 2015 fires), internal migration and the chronic demand for land by agricultural communities and by industry. Some of these factors are outside the sphere of influence for USAID and the LESTARI program (e.g., changing political priorities); others require long-term solutions, including improved forest management capacities at all levels, and complex multisectoral negotiations are required to reverse decades of land use policy that produced significant degradation, particularly in the case of policies that encourage drainage of peatlands. Improving capacity is a significant objective of LESTARI, and an important part of the solution. Capacity building is necessary, but by itself insufficient, to stop conversion. It is important to note that continued improvement in capacity at all levels to manage drivers of forest and biodiversity loss is a long-term undertaking that must extend well beyond the LESTARI period of performance, and LESTARI efforts must be prioritized to address what can be accomplished by the conclusion of the project. FINDINGS A substantial amount of work to prevent conversion was observed across the LESTARI landscapes. The process to restore peat forest through the closure of drainage canals in the Katingan-Kahayan landscape of Kalimantan is noteworthy for its application of FPIC, the Padiatapa approach, to wetland restoration. It links the social process of FPIC with peatland restoration as part of a fire prevention and habitat restoration strategy. This approach has overcome obstacles encountered in prior attempts to close canals, by better engaging communities around issues of land use change, and the impacts such change can have on daily life. The successful pilot closure demonstrates that the approach can generate the local support necessary to ensure the sustainability of the wetland restoration work. If LESTARI and its partners can replicate the 2 Hansen, M. C., P. V. Potapov, R. Moore, M. Hancher, S. A. Turubanova, A. Tyukavina, D. Thau, S. V. Stehman, S. J. Goetz, T. R. Loveland, A. Kommareddy, A. Egorov, L. Chini, C. O. Justice, and J. R. G. Townshend. 2013. “High-Resolution Global Maps of 21st-Century Forest Cover Change.” Science 342 (15 November): 850–53. Data available on-line from:http://earthenginepartners.appspot.com/science-2013-global-forest.Accessed via Global Forest Watch. on 2/22/2018. www.globalforestwatch.org 16 LESTARI MID-TERM EVALUATION USAID.GOV success of the pilot phase, it will produce a major impact on conversion of critical areas, including a substantial amount of expected greenhouse gas emissions. This process, driven by an effective multistakeholder forum, has been unanimously praised as useful by respondents at all levels; representing governments, village representatives, disaster management units, and government planning representatives. Also within the Katingan-Kahayan landscape, LESTARI has intervened to accelerate the issuance of village forest permits in the Tangkahan district in order to prevent the conversion of land to palm oil production. In Cyclops, the main threats to land cover change are encroachment by migrants and urban sprawl around Jayapura. Here, co-management is increasing the acceptability of the Cyclops Natural Reserve by the adjacent communities, while increasing the conservation area management capacity. The support package created by LESTARI and partners for pilot/priority villages in Cyclops involves the production of a co-management plan for natural resources in the Cyclops buffer zone, support for the development of village conservation regulations, and the establishment of a community ranger program as a pathway to community-based conservation. The villagers of Necheibe are committed because conservation supports customary mandates. The full package has been applied only in two pilot villages so far, both of which are comprised of local people (i.e., no migrant settlements). A vanilla cultivation initiative in Cyclops targets villages with high threats where local and migrant people are living. Its activities are still in the early stages in four villages covering 180 members. It is not yet possible to assess its potential for preventing conversion. One of the farmer groups has agreed with the draft of the conservation agreement where they agree not to clear land inside the Nature Reservation. In the Lorentz landscape an agreement has been concluded with LESTARI support with two villages in Asmat, and crab traders to co-manage an 61,000-ha mangrove ecosystem. It has not been operationalized, however, leading to the yellow rating. In Mappi-Bouven Digoel, where community engagement is only now beginning, one clan has already succeeded in initiating the process of protecting its land from Korindo, with LESTARI support. Also, in Mappi-Bouven Digoel, Adat councils in both districts supported LESTARI on SEA-spatial plan reviews, but there was no evidence of their participation in the SEA-spatial plan and HCV mapping on their own initiative. (They engaged only when invited by LESTARI.) DATA SOURCES Data on land degradation and deforestation comes from satellite remote sensing, particularly using the World Resource Institute/University of Maryland Global Forest Watch dataset. Data on factors at play in causing deforestation come from the literature review (notably, in the case of palm oil, from Chain Reaction Research) and from key informant interviews with government officials, NGOs, and the private sector. 17 LESTARI MID-TERM EVALUATION USAID.GOV Q3: TO WHAT EXTENT ARE LESTARI POLICY RECOMMENDATIONS INCORPORATED INTO GOVERNANCE DOCUMENTS AT THE DISTRICT, PROVINCE, AND NATIONAL LEVELS? Evaluation Questions Leuser Katingan Cyclops Lorentz Mappi Technical Component 1: Forest and Land Use Governance and Advocacy 3 To what extent are LESTARI policy recommendations incorporated into governance documents at the district, province, and national levels? RESPONSE LESTARI policy recommendations and supporting materials are well regarded and used at the district and provincial level. National level policy inputs are being handled by a parallel USAID project, the Build Indonesia to Take Care of Nature for Sustainability project or Bangun Indonesia untuk Jaga Alam demi Keberlanjutan (BIJAK)3; the evaluation team did not pursue this line of inquiry. FINDINGS LESTARI has worked with technical partners such as UNOPS on fire management and peatland restoration, where LESTARI’s added value has been in establishing a policy framework for support. This complementarity mode is repeated throughout LESTARI’s work. Examples include RPJMDes throughout the landscapes, where communities and government officials attested that LESTARI showed them to effectively make use of this process leading to new capacity to leverage government/village funds for conservation. The evaluation team found that LESTARI policy recommendations were widely incorporated into governance documents at the district level. In Aceh, Lestari worked to harmonize national and provincial regulations (Qanum number 7 and 2 National Forestry Regulations). In Papua, the Spatial Planning Information Management System (Sistem Informasi Tata Ruang or SIMTARU) has been operationalized for land use planning and monitoring but is only being fully used by BAPPEDA at this stage. Some recommendations have also been incorporated into provincial level governance documentation. For example, in Katingan-Kahayan, the harmonization of fire regulations has been a major policy contribution of LESTARI in the last two years. Recommendations for repealing some inconsistent laws and regulations are now being implemented through the legislative process. LESTARI’s support for policy development is widely appreciated by government partners, including LESTARI’s coordination with other technical support activities such as those under the aegis of the UN Office for Project Services (UNOPS). In the Lorentz Lowlands landscape, LESTARI has helped to produce the Lorentz Lowlands zoning plan, the FMU long-term management plan, and the SEA-Spatial plan review to change the classification of Rawa Baki into protected forest/Essential Ecosystem Area. In Asmat, the district secretary agreed to support changing the classification4 of Rawa Baki from one of limited production forest to one protected 3 With the exception of policies related to Reduced Impact Logging, which LESTARI addresses directly. 4 The Sekretaris Kabupaten does not have legal authority to change a national land use designation within the forest estate 18 LESTARI MID-TERM EVALUATION USAID.GOV area (Essential Ecosystem Area), based on LESTARI recommendations, according to key informants. The IFACS final report had already noted that the World Wildlife Fund had lobbied for this designation. The FMU VI (Mimika) has agreed to protect sacred and culturally important areas for the Adat (indigenous) people and integrate that protection into the long-term forest management plan (RPHJP). Current legal mandates for land use decision-making at the national level (e.g. concessions for mining by ESDM – the Ministry of Energy and Mineral Resources), or at the provincial level (e.g., concessions for plantations by provincial forestry authorities) may conflict with legal mandates at local levels (spatial planning, RPJMDes). Since LESTARI worked primarily at local and provincial levels, the evaluation found less evidence of LESTARI influence at the national level; where sister USAID project BIJAK is playing the leading role. DATA SOURCES Review of policy documents, interviews with government personnel, and LESTARI reporting provided the basis for these findings. Q4: TO WHAT EXTENT ARE MULTI-STAKEHOLDER FORUMS INFLUENCING LAND USE DECISION-MAKING, MONITORING AND PLANNING? Evaluation Questions Leuser Katingan Cyclops Lorentz Mappi Technical Component 1: Forest and Land Use Governance and Advocacy 4 To what extent are Multi-Stakeholder Forums influencing land use decision-making, monitoring and planning? RESPONSE The influence of the Multi-Stakeholder Forums (MSFs) is inconsistent, depending upon the context and the personalities involved in forum leadership. LESTARI has concluded that the forum approach is obsolete and, based upon successes like the MSF in the Katingan-Kahayan landscape, is adapting in favor of more action oriented multi-stakeholder partnerships. FINDINGS Multi-Stakeholder Forums (MSFs) are a legacy of the previous IFACS project and there is considerable variation in their history and their record on influencing land use since that time. In Leuser, for example, the long-standing MSF process has over time been captured by a faction promoting its own agenda, and is no longer functioning as a participatory forum. Taking this into account LESTARI has moved away from Forums (narrowly construed) towards initiatives with a common interest among multiple stakeholders, the best example of which is in the Katingan-Kahayan landscape. With LESTARI support, the Pulang Pisau MSF is using FPIC through multi-stakeholder consultations as part of its fire prevention and peatland restoration strategy. (such as limited production forest), only to propose changes. Any proposals must be approved at provincial and national levels before the changes are made official. 19 LESTARI MID-TERM EVALUATION USAID.GOV This effort in Pulang Pisau is viewed as a success story by all levels of respondents: governments, village representatives, disaster management units, and government planning representatives. In particular, a subgroup called Tim 9 enjoys strong all-around buy-in and demonstrates a strong grasp of the positive impact of fire prevention. Tim 9 drives the FPICs process and encourages BRG (Badan Restorasi Gambut – the Peatland Restoration Board) to allocate activities and programs in fire prevention through drainage canal closures. This has been a result of the convergence of events and demands made on the group in the wake of the 2015 fires. In the Lorentz Lowlands, the MSF is regarded as highly effective in working with a civil society coalition to combat wildlife trafficking, according to key informants from diverse institutions. The collaboration has led to the creation of a fund for conservation. Formal Multistakeholder forums have not yet come on line for Cyclops and Mappi, resulting in their yellow ratings. DATA SOURCES Key informant interviews with MSF participants and LESTARI staff, and LESTARI reporting, provided the basis for this finding. Q5: TO WHAT EXTENT HAVE LESTARI ACTIVITIES LED PROVINCIAL AND DISTRICT GOVERNMENTS TO EFFECTIVELY MONITOR LAND USE AND LICENSING? Evaluation Questions Leuser Katingan Cyclops Lorentz Mappi Technical Component 1: Forest and Land Use Governance and Advocacy 5 To what extent have LESTARI activities led provincial and district governments to effectively monitor land use and licensing? RESPONSE The uptake of LESTARI support for monitoring land use and licensing at the district and provincial level is uneven, and the prospects for continuation of this work post-LESTARI are unclear. FINDINGS LESTARI activities on land use monitoring and licensing at the district and provincial levels have had mixed results. In the Leuser landscape, Bappedas Blankejeran reported that land clearances within the district have gone down based on their internal monitoring system developed with LESTARI support. The Wildlife Conservation Society, a LESTARI partner, indicated that LESTARI has helped to harmonize land use monitoring. In Papua, provincial government plays a significant role in licensing decisions at the district level. In Lorentz, the SIMTARU has been installed, is operational, and is now in use by the provincial BAPPEDA to monitor land use and licensing. Using this, the governor of Papua rejected a proposal from the Bupati for Mimika to convert protected forest for a smelter plant. In another case, the spatial planning coordination board (BKPRD) of Mimika suspended the operating license of a sago processing plant in a protected forest unless the investors secure reclassification of the land, as the result of an MSF intervention. LESTARI is now moving to restore impacted wetlands. 20 LESTARI MID-TERM EVALUATION USAID.GOV On the other hand, the Bupati (head of district) holds nominal responsibility for issuing location permits and environment licenses, but none of the district governments have been using SIMTARU to support the Bupati in permits and licensing. The Bupati depends on input from the Environment Impact Assessment team (which is organized by the Environment Agency of Papua Province) for making decisions on environment licenses. This agency does not use SIMTARU and the Sustainability Screening Toolkit (SST). The one-stop licensing service for the provincial government (PTSP) does not fully use SIMTARU and still has no clear understanding about the function of the SST. LESTARI efforts to support SIMTARU and SST in provincial government through technical assistance and training have been somewhat effective. Apparently, however, success in monitoring of land use and licensing has not necessarily translated to protection against conversion. In the example from Mimika above, the response to incompatible use was to reclassify the land. In the Mappi-Bouven Digoel landscape, LESTARI works with local governments and with the private sector to assess high conservation value (HCV) lands and to undertake Strategic Environmental Assessment spatial plan reviews with a view to enlarging protected areas. Two district governments indicated that they appreciate LESTARI support on SEA-spatial plans and will be able to increase protected area coverage in the spatial plan as a result. However, SIMTARU and SST are not in use in this landscape. As in the Lorentz Lowlands, the Bupati relies upon the Environment Agency of Papua Province for technical support, and this agency does not use SIMTARU and SST. Given the investment required in keeping these systems current and maintaining trained staff, sustainability is uncertain. Efforts to explore this investment by the evaluation team were generally interpreted by partners as invitations to lobby for LESTARI continuation, suggesting a lack of an exit strategy on the part of some LESTARI partners. DATA SOURCES Key informant interviews with government officials and LESTARI staff, supported by documentation, provided the bases for these findings. 21 LESTARI MID-TERM EVALUATION USAID.GOV Q6: TO WHAT EXTENT IS LESTARI EFFECTIVELY ENCOURAGING MANAGEMENT OF CONSERVATION5 AREAS (E.G. CO-MANAGEMENT, FMUS, METT, ETC.)? Evaluation Questions Leuser Katingan Cyclops Lorentz Mappi Technical Component 1: Forest and Land Use Governance and Advocacy 6 To what extent is LESTARI effectively encouraging management of conservation areas (e.g. co-management, FMUs, METT, etc.)? N/A RESPONSE LESTARI is successful in encouraging more effective management of conservation areas using self￾assessment tools and data collection tools. Technical assistance and training is being provided to Forest Management Units (FMUs). The sustainability of the FMUs over the long term is a subject of concern, and there are limits to what LESTARI can do; a comprehensive national capacity development strategy for FMUs is necessary and beyond the remit and resources of the current project. FINDINGS The Management Effectiveness Tracking Tool (METT) self-assessment scorecards provide protected area managers with baseline data on their management effectiveness. With targets for improvement, the evaluation team found a serious commitment wherever METT was applied to improve scores. Self￾assessment has its limitations, however. Third party validation, performed in Sebangau National Park, indicated lower ratings than the self-assessment. But it also showed improvement over the baseline. In the Trumon corridor of the Leuser landscape, tamed elephants patrol the area composed of both production forest and APL. Elephants and mahouts or rangers go on patrol for 7-10 days over the 2,700 ha. The tamed elephants are used in perimeter patrols of park boundaries, where, as an added benefit, they also contain wild elephants. This also prevents human/wildlife conflicts with wild elephants, a major threat to agriculture and to support for conservation. These elephants appear anecdotally to advance community support for conservation, based upon interviews with stakeholders. With LESTARI support, the Wildlife Conservation Society runs a seven-point program on wildlife management in the Trumon corridor; it is reported to have lowered both the rate of human wildlife conflict and the temperature of community emotions, which can run high in such situations. LESTARI has supported the establishment of co-managed community forests for several villages in the Katingan-Kahayan landscape. Dayak tribes living in the concession area are encouraged to co-manage the high conservation value areas. LESTARI subcontractor Tropical Forest Foundation provides training to reduced impact logging (RIL-C) for the Dayak communities and concessionaire. Reduced impact logging is important to Dwima and other concessionaires for its Forest Stewardship Council (FSC) certification of sustainable production. Dwima expressed satisfaction with the reduced impact logging because it saves money while fostering prospects for community buy-in and continuity. 5 The evaluation team interpreted this question to refer to the Conservation Co-Management Thematic Area, including both conservation areas and forest management areas, including specific reference to FMUs. 22 LESTARI MID-TERM EVALUATION USAID.GOV In Papua, conservation village regulations provide innovative ways to integrate customary law and formal law to increase local people’s acceptance of conservation measures. The effectiveness of this recent development cannot yet be measured. In Cyclops, LESTARI is working with a partnership of the provincial government, Jayapura district government and the national Nature Conservation Agency (BKSDA) to support conservation planning, codification of village conservation regulations, community mapping, development of a cadre of community rangers, and improved forest management in the buffer zone. Support is being provided for the organizational and human capacity of BKSDA/FMU. This includes the establishment of conservation area management through METT, and training on participatory mapping, geographic information systems, multidisciplinary land use assessment (MLA), and SMART patrolling. In the Lorentz Lowlands, two communities, crab traders, and the district government have agreed to cooperate to support forest conservation in a 61,000-ha mangrove forest area in Asmat. LESTARI has not yet been able to reach an agreement with timber and oil palm concessionaires to cooperate to improve management practices in Mimika. A community land use plan has been incorporated into the draft of the long-term forest management plan for Mimika VI. There are 9 villages with conservation village mid-term development plan to leverage local government and village funds. LESTARI is now in discussion with the industrial conglomerate Korindo, which holds a major oil palm concession in Mappi-Bouven Digoel. Korindo has been the target of a disinvestment campaign by forest conservation pressure groups as a result of its controversial forest clearance with fire in 2016 in Papua. The pressure is apparently influencing Korindo’s business practices. LESTARI is promoting a FPIC-based approach with Korindo for the management of its concession in the Mappi-Bouven Digoel landscape. This would potentially involve substantial forest set-asides for cultural and conservation purposes. Through the approach that LESTARI is promoting, communities will work with concession managers to develop and implement a land-use plan. Such an approach would be beneficial to Korindo in meeting financial and conservation targets for certification of its palm oil by the Roundtable for Sustainable Palm Oil (RSPO), essential for access to European markets. If successful, it will establish a new model for the palm oil industry, and if replicated, result in protection of substantial amounts of at-risk high￾conservation value (HCV) and culturally significant forests for communities in and around concessions. This is a high-risk/high-reward initiative for LESTARI. The partnership is still under negotiation and is not finalized. As many forest and oil palm concessions in Papua are not yet operational, the timing is propitious for a successful model to be scaled up to wider adoption. DATA SOURCES Focus group discussions and key informant interviews with communities, government agency personnel, LESTARI partners and representatives of the private sector, combined with LESTARI reports, supplied the data for this finding. 23 LESTARI MID-TERM EVALUATION USAID.GOV Q7: TO WHAT EXTENT ARE SMART PATROLS PREVENTING WILDLIFE TRAFFICKING AND POACHING? Evaluation Questions Leuser Katingan Cyclops Lorentz Mappi Technical Component 2: Improved Management of Conservation Areas 7 To what extent are SMART patrols preventing wildlife trafficking and poaching? N/A RESPONSE This question is poorly formed, and does not reflect the activities identified in the approved workplan and Activity Monitoring and Evaluation Plan (AMEP). For example, the AMEP does not mention trafficking, an activity is outside the scope of LESTARI. If rephrased as “to what extent are SMART patrols improving protected area management,” Leuser and Cyclops would be rated green, and Katingan and Cyclops would be rated yellow. No conservation area or management authority provided evaluators with evidence sufficient to attribute changes in wildlife crime directly to the use of Spatial Monitoring and Reporting Tool (SMART). Even with adequate data, attribution of change in wildlife crime is notoriously difficult, as a high number of variables shape the crime environment. The application of SMART under LESTARI is restricted to protected areas, and cannot therefore detect wildlife crimes outside protected area boundaries, including "leakage" - where poachers are displaced due to increased law enforcement presence, to areas where enforcement is weak. SMART is contributing to management effectiveness of conservation areas, and may indirectly contribute to success in combating wildlife crime. It allows conservation area and wildlife managers to find patterns in the data about nature and in human behavior with predictive power, as illustrated by Wildlife Conservation Society success elsewhere in the use of game-theoretical algorithms for allocation of resources to combat wildlife crime. SMART data also have other benefits for conservation area management such as wildlife observations and threat observations. FINDINGS SMART, a handheld device used to record location and observation, is designed to streamline data recording and analysis for better management. Some conservation areas have a well-organized reporting system based on SMART documentation of illegal activities (agricultural encroachment, illegal logging, poaching), and wildlife sightings and GPS locations. SMART data can be used with predictive algorithms to optimize patrol deployment. Buy-in to the SMART patrol approach on the part of conservation area managers are strong, but there are some exceptions. SMART patrols are systematically used in Leuser National Park and Rawa Singkil Wildlife Reserve. Trumon Wildlife Corridor patrols do not use the SMART technology, as it does not yet have protected status (though LESTARI is working toward that end). Data is compiled with the National Park Office and analyzed for illegal activity. Actions on the part of the conservation area managers based on this information ranges from working with the police for apprehension, to visiting the community to deliver warnings, and sometimes, in this socially conservative region, religious discussion. The Rawa Singkil Wildlife Reserve has noted areas of increased illegal activities nearest to the adjacent provincial roads, and decreased illegal activities in other areas. However, at this stage, it is hard to establish the exact 24 LESTARI MID-TERM EVALUATION USAID.GOV correlation between SMART Patrols and decreased illegal activities. The project is aware of this and is taking steps to collect additional data. In Sebangau National Park in the Katingan-Kahayan landscape, buy-in to SMART is a different story. National Park managers report that they lack equipment that is compatible with the SMART software. Managers also complain of being unable to conduct SMART patrols as often as agreed, due to inadequate financial support. In Cyclops, SMART patrol data has been used to plan patrol tracks, identify conservation activities by location, and develop land use zoning. Lorentz National Park is using the SMART patrol approach, but they do not use the data primarily to support law enforcement. Managers have found SMART to be most useful as a tool with which to engage local communities and to collect data for zoning. Despite reports of widespread trafficking in wildlife products originating in Papua, including in Lorentz6, key informants report that SMART patrols have never found a violation. This is not supported by the Year 2 Annual Report. The evaluation team cannot reconcile the differences between what managers in the field report and what is reported by the Implementing Partner. However, the report from the field that SMART patrols have not been effective suggests perceptions of key informants of the relevance of the approach as implemented in the Lorentz landscape. DATA SOURCES Focus group discussions and key informant interviews with conservation area personnel, and literature review comprised the data for this conclusion 6 See the 2015 World Heritage 38th UNESCO World Heritage Committee proceedings, which includes the 2014 Mission Report for Lorentz from the agenda item entitled State of conservation of properties inscribed on the World Heritage List and/or on the List of World Heritage in Danger. Document MIS955apr2014, retrieved from https://whc.unesco.org/en/documents/136459/. While the State Party denies widespread poaching in Lorentz, LESTARI states that in Lorentz “animal poaching still frequently occurs.” Protected the Sleeping Papuan Giant https://www.lestari-indonesia.org/wp-content/uploads/2016/12/USAID_LESTARI￾Story_from_the_Field-Protecting_the_Sleeping_Papuan_Giant.pdf. The extent of poaching has not been measured but informants to the evaluation indicate that it is substantial. 25 LESTARI MID-TERM EVALUATION USAID.GOV Q8: TO WHAT EXTENT ARE LESTARI PUBLIC-PRIVATE PARTNERSHIPS SUCCESSFUL IN INCREASING ENVIRONMENTALLY SUSTAINABLE COMMODITY PRODUCTION? Evaluation Questions Leuser Katingan Cyclops Lorentz Mappi Technical Component 2: Improved Management of Conservation Areas 8 To what extent are LESTARI public-private partnerships successful in increasing environmentally sustainable commodity production? RESPONSE The evaluation team found some evidence of effective public/private partnerships, but widespread success eludes the project. Overall, public/private partnerships under LESTARI are not sufficiently mature to determine their effectiveness over the long term, with the exception of Katingan-Kahayan, which has made good progress in working with rubber producers. The team determined that where LESTARI contributes (primarily through grants and partnerships), it does not do an adequate job of market research before engaging with stakeholders in commodity promotion. Opportunities exist for partnerships with the private sector that were not addressed, particularly in taking advantage of new RSPO rules requiring conservation investment for past damage. However, USAID and LESTARI had determined to avoid engagement related to sustainability in the palm oil sector given the number of other actors. This closed off access to some opportunities to engage the private sector. FINDINGS The capacity of farmers to assess market opportunities is poor. This places the onus on LESTARI when identifying opportunities, and this support is lacking. As a result, a LESTARI grant supports community ecotourism where the prospects for business are poor, and there is uncertainty about the markets for a range of commodities. LESTARI’s main contribution has been to increase a community’s capacity and skill in livelihood improvement through partnerships. Table 3 below shows commodities for which LESTARI has made a contribution. At a policy level, LESTARI helps communities to expedite the process of obtaining social forestry permits at the national level7. This is necessary to secure land rights and access to land, in order to take advantage of PPPs. Businesses involved in profit-oriented producer-buyer partnerships with communities for commodity production confirmed LESTARI support to be valuable. The focus of PPP engagement in Katingan-Kahayan so far has been on rubber in Pulang Pisau. Rubber as a commodity does not require many chemicals, can grow in peatland, and has strong potential as a 7 At the time of the evaluation, the government had not yet approved any such permits. 26 LESTARI MID-TERM EVALUATION USAID.GOV carbon sink. Eighty percent of the communities here have traditionally worked in rubber production. The rubber tapping communities received replanting support immediately after the big fire in 2015, which has provided hope and hence prevented them from selling land to palm oil plantations. In 2015, replanting rubber through a PPP was restoring communities’ hope after the large fire incident emergency. LESTARI facilitated a new and more profitable business model for rubber production, which allows for direct access to the processing companies.8 The new business model for rubber producers is helping the rubber tapping communities to better integrate into the rubber processing value chain, improving their economic outlook. However, the market price is currently low and farmers are shifting to other commodities, especially sengon, a softwood grown in plantations for plywood manufacture. Rubber communities currently plan to replant rubber and sengon, but if sengon is profitable, there is a possibility that that it could perversely incentivize plantations at the expense of natural forest restoration. This is an understudied area. In the Leuser landscape, the “group approach” to certified organic farmland in Pepelah (Pining), certifies a community rather than an individual production unit. It provides an effective tool for peer￾enforcement, but it comes with a risk. Certification requires time (e.g., no chemical applications for 5-7 years). Group certification addresses barriers to entry by smallholder farmers, but group certification for recognition by European and Japanese markets also requires an additional set of steps in the form of the development of an auditable internal control system to govern the practices of the group. 8 The model may be difficult to replicate for other commodities because it builds on the geographical proximity of buyers and processors, which will not always be the case. Commodity Landscape Rubber Katingan Cacao Leuser Coffee Leuser Nutmeg Leuser Mangrove fruits Lorentz Crab Lorentz Sambon agroforestry Katingan Vanilla Cyclops Water as an ecosystem service (in design phase) Cyclops, Lorentz Table 3: Commodities in LESTARI documented by the evaluation team 27 LESTARI MID-TERM EVALUATION USAID.GOV Memoranda of understanding have been signed for buyers to purchase commodities from community groups as soon as they receive the “lahan organik” group certification. Obtaining this certification is quite rigid and demanding. The group-based organic cacao certification, for example, requires internal controls include peer-enforcement on those farmers who do any farming activities in the covered geographic area (including non-supported crops). This group approach can be undermined by non-participating community members if insufficient attention is paid to building consensus within the community sufficient to ensure collective compliance with the organic certification standards, Meanwhile the farmers need to earn money. Partnerships for other “branding9” methods may be required that can be accomplished in a shorter amount of time, as an intermediate step. The impact of sustainable livelihood activities on biodiversity and forest conservation is not being measured. Essential oil production from nutmeg, lemongrass or patchouli requires a high amount of fuel wood (pine trees) in/around both Gayo Lues and Tapaktuan, and fuel consumption may cancel out landscape benefits. Though LESTARI does not support these activities, they may be a threat to area forests. Clean-energy investment could be helpful in these cases, possibly through public/private partnerships and technical exchange with the Indonesia Clean Energy Development II program. In Cyclops, LESTARI is working with partner the National Cooperative Business Association (NCBA) on a vanilla cultivation initiative. LESTARI staff have no specific experience with vanilla. The evaluation team visiting the site determined that there is no dedicated NCBA staff with vanilla cultivation expertise based in Cyclops, and that NCBA remotely supports the initiative from Genyem. LESTARI management reports that NCBA has an on-site expert in Sentani. This initiative is in the early stages of development and its chances of success cannot yet be assessed. LESTARI has provided trainings on mangrove fruit processing in Asmat and Mimika. There is as yet no plan for organization of production and supply chains. With the exception of Katingan-Kahayan’s rubber communities in Pulang Pisau, the interviewed communities in Leuser and Katingan-Kahayan have shown dependence on local agents, especially regarding price and market information. DATA SOURCES Focus group discussions and key informant interviews with producers, buyers, and LESTARI staff, supported by literature review. 9 An example of such an approach would be “songbird friendly” coffee, which is verifiable in a shorter amount of time. 28 LESTARI MID-TERM EVALUATION USAID.GOV Q9: TO WHAT EXTENT ARE THESE PARTNERSHIPS REDUCING DEFORESTATION AND FOREST DEGRADATION? Evaluation Questions Leuser Katingan Cyclops Lorentz Mappi Technical Component 2: Improved Management of Conservation Areas 9 To what extent are these partnerships reducing deforestation and forest degradation? RESPONSE While LESTARI is having success at the community level, public/private partnerships under LESTARI are not sufficiently mature overall to gauge their effectiveness in reducing deforestation and forest degrada￾tion over the long term, with the exception of Katingan-Kahayan. This may change dramatically, for example if the Korindo initiative gains traction, as it may be scaleable to other oil palm production concessions. FINDINGS There are no PPPs that have been operational for long enough to provide evidence of reductions in deforestation and forest degradation. In Mappi-Bouven Digoel, the Korean-Indonesian forest products firm Korindo has signaled openness to adopt some sustainability practices (HCV, FPIC) including a potential set-aside of as much as half of the concession in HCV and community lands in cooperation with LESTARI, but the engagement with the company is still in exploration stage. This is a potentially significant development for LESTARI because of the high potential for scaling the approach, particularly where RSPO certification requires conservation offsets to account for past biodiversity loss. LESTARI does not yet monitor greenhouse gas emissions, including environmentally sustainable commodity production10. The processing of commodities can have high energy demands that are being met by fuelwood. For example, in Aceh, essential oil production (nutmeg, lemon grass, patchouli), depends upon fuelwood cut from the adjacent forests to provide the energy needed for processing. While not a LESTARI initiative, this illustrates the potential risk that solutions could work at cross purposes to overall project goals. DATA SOURCES Key informant interviews with partners and staff, review of literature, including Chain Reaction Research11 reports, and review of remote sensing data (via WRI’s Global Forest Watch). 10 USAID technical responses have noted here that, “In the Activity Monitoring and Evaluation Plan (AMEP) v. 2, "wall-to-wall" land use/land cover data will be used to assess GHG emissions for the entire landscape. "Leakage" (such as fuelwood energy demand not addressed by LESTARI activities) within the landscape will still figure in to the GHG emissions assessment.” 11 Chain Reaction Research is a reporting service providing sustainability risk analysis to investors in the oil palm industry, through a partnership between Dutch-based Aidenvironment and Profundo consultancies and US-based Climate Advisors. https://chainreactionresearch.com/ 29 LESTARI MID-TERM EVALUATION USAID.GOV Q10: TO WHAT EXTENT HAS LESTARI INCREASED ACCESS TO AND USE OF LONG TERM FINANCING FOR CONSERVATION AND RESTORATION? Evaluation Questions Leuser Katingan Cyclops Lorentz Mappi Technical Component 3: Improved Private Sector Practices 10 To what extent has LESTARI increased access to and use of long￾term financing for conservation and restoration? RESPONSE LESTARI has done little to increase access and use of long term financing for conservation and restoration. However, government budget allocation in annual work plans have increased for conservation activities through program and activities in developed RPJMDes. This is a direct result of the significant progress made by LESTARI in providing technical assistance and partnerships for local development planning and local governance. FINDINGS In the Leuser landscape significant progress was observed in community planning through the RPJMDes, resulting in increased government budget allocations where the RPJMDes was employed. LESTARI has leveraged project financing through small grants for local organizations (Javlec, Inprosular, Nutmeg Forum, Atjeh International Development, and Aceh Green Community). At least one grantee (OIC) has leveraged LESTARI funds to generate more attention and successfully attract another donor (in this case, the Singapore Zoological Gardens). Using this strategy may enable some grantees to transition into longer-term financial support. LESTARI has leveraged project budgets through match￾making grants for three grantee Foundations/NGOs; Karsa, BOSF, Yayasan Tambuhak Sinta (YTS). YTS is a development foundation established by the mineral exploration company PT Kalimantan Surya Kencana. The three Papuan landscapes are coded yellow because the private sector engagement there remains at an early stage. One initiative shows promise in the Lorentz landscape. With LESTARI support, a number of stakeholders have agreed to establish Friends of Lorentz Foundation to support a conservation program in the Lorentz Lowlands landscape. One of the partners, Freeport, has agreed to provide initial financing. This is in early stages of development. Likewise, an initiative with forest products and palm oil producer Korindo is under development, but is not yet being implemented in the Mappi landscape. DATA SOURCES Interviews with provincial and district personnel, community leaders, grantees and partners of LESTARI, combined with LESTARI reporting provide the basis for this finding. 30 LESTARI MID-TERM EVALUATION USAID.GOV Q11: HOW IS THE IMPLEMENTATION STRUCTURE AND MANAGEMENT OF LESTARI CONTRIBUTING TO ITS SUCCESSES AND ARE THERE AREAS FOR IMPROVEMENT? Evaluation Questions Leuser Katingan Cyclops Lorentz Mappi Technical Component 3: Improved Private Sector Practices 11 How is the implementation structure and management of LESTARI contributing to its successes and are there areas for improvement? RESPONSE The evaluation team found that the structure and management of LESTARI is a significant factor in its success. LESTARI management and staff demonstrate strong commitment to conservation, partners and communities. The LESTARI senior management provides strong support for the field program, and empowers regional leadership to use initiative and make management decisions as necessary. FINDINGS LESTARI’s internal mid-term evaluation, called the Pause and Reflect, has made a positive contribution to project success through collaborative learning and adaptive management, and is an excellent example of a strategy for flexible and adaptive project management. LESTARI has undertaken a political economy analysis for each of its landscapes and used the findings for a mid-term stocktaking, showing a strong commitment to adaptation informed by a sound understanding of the drivers of forest loss. The decentralized organization of LESTARI places technical support and leadership in close proximity to stakeholders. This permits continuous engagement with stakeholders, and promotes stronger local ownership of LESTARI processes and products. LESTARI technical staff and counterparts for the most part enjoyed good working relationships with communities, local government, NGOs, partner organizations, and other stakeholders. LESTARI creates local champions from communities, which will help to ensure that their program will still be implemented when the project ends. The Papuan landscapes, Cyclops, Lorentz, and Mappi, are marked as yellow because start-up was much slower in Papua; accordingly, work is less advanced and there as yet is insufficient success that can be attributed to LESTARI management. This is not to suggest however that there is a problem. It is only a function of the timing of the evaluation. Several LESTARI partners complained about an excessive workload for administering and reporting on LESTARI funds. This is not surprising as USAID reporting requirements are rigorous, and is a common refrain, especially from non-governmental organizations. Complaints about response times from LESTARI were also raised. Our inquiry showed that slow response times were linked to inadequate information for decision-making in requests from partners, indicating that their personnel may require additional training in USAID procurement policies and LESTARI procedures. DATA SOURCES Data for this question was collected in key informant interviews with LESTARI staff, government partners, and stakeholders. 31 LESTARI MID-TERM EVALUATION USAID.GOV Q12: TO WHAT EXTENT ARE LESTARI ACTIVITIES UNDERTAKEN WITH A FOCUS ON SUSTAINABILITY OF RESULTS AFTER THE PROJECT IS CLOSED? Evaluation Questions Leuser Katingan Cyclops Lorentz Mappi Technical Component 3: Improved Private Sector Practices 12 To what extent are LESTARI activities undertaken with a focus on sustainability of results after the project is closed? RESPONSE LESTARI has integrated long term sustainability of activities into its project design through concerted efforts to build the capacity of counterpart institutions and communities to continue the work without LESTARI support in the long-term12. However, based upon key informant interviews, some of LESTARI’s counterparts appear not to have given sufficient credence to the possibility that LESTARI would not be extended. There is therefore a risk that some initiatives may not be sustained without external support because that eventuality was not adequately considered. Evaluators observed that questions about long￾term operations were often deflected and not addressed directly. FINDINGS LESTARI has designed many of its activities so that they can continue after the termination of the project. This is evident in the case of community co-management and social forestry, where the uptake of the Village Level Medium Term Development Planning process (RPJMDes, discussed in Questions 1 and 10) shows strong indications of being institutionalized. It is also evident in the use of METT, and to some extent SMART. It is less so in the case of SST, SIMTARU, and the SEA spatial planning process, which have potentially high upkeep costs. Most LESTARI-supported agricultural communities are not yet sufficiently equipped to cost, develop and innovate their business models by themselves and would be dependent on further assistance by a service provider or the buyer company. Discussions by LESTARI with partners of strategies for continuation post-project have not always resonated. A common refrain in key informant interviews was a plea that LESTARI continue, and there is a general lack of recognition that work will need to continue without external support at some point. There is, in short, a pervasive sense of denial that USAID support for forests and biodiversity will not continue in this form, and an unwillingness to recognize the need to put in place measures to ensure continuation without external support. DATA SOURCES Findings for this question are a synthesis of overall responses to the evaluation questionnaire, supported by reporting. 12 For example, see “LESTARI Model for Amplification and Sustainability”, 2016-17 Annual Report, pp 71-72. 32 LESTARI MID-TERM EVALUATION USAID.GOV Q13: TO WHAT EXTENT IS THE THEORY OF CHANGE STILL VALID? Evaluation Questions Leuser Katingan Cyclops Lorentz Mappi Program Design and Performance 13 To what extent is the theory of change still valid? RESPONSE The theory of change is valid, but has hidden assumptions around private sector motivation to engage, enabling leadership and policy environment, alignment of interests, and access to information. A further assumption underlying the theory of change is that the institutions with which LESTARI are working will be effective for implementing LESTARI-promoted innovations such as co-management. Some of the results chains are incomplete or poorly articulated. FINDINGS LESTARI’s theory of change is that: …if there is increased awareness and advocacy of local stakeholders (government officials, communities, CSOs and private sector) for conservation and natural resource management in ways which contribute to their long term interests; if Conservation Co-Management is actually enhanced to the benefit of key stakeholders, and if the private sector is engaged, there will be improved governance and natural resource management in biologically significant and critical areas, leading to a 41% reduction in GHG emissions by 2020. A graphic showing a simplified results chain for LESTARI is presented in Figure 1 below. The theory of change is incomplete because, while the motivation of local stakeholders is taken into account, the complementary understanding of the motivation of the private sector is not. This produces a lack of clarity on why the private sector would be engaged, how to engage the private sector, and how that engagement would lead to improved governance and natural resource management. The theory assumes a number of conditions, including leadership, a favorable policy environment, alignment of interests, access to information, and coordination across the wide range of decision-makers involved in forests, water, and land use. It also assumes effective institutions to support the policies and innovative approaches that LESTARI has promoted such as co-management. Although the theory of change is valid, the results chains suffer from omissions and assumptions discussed in response to the following evaluation question, also dealing with the theory of change. DATA SOURCES The theory of change comes from LESTARI’s Activity Monitoring and Evaluation Plan. Evaluation of the theory of change involved key informant interviews with partners and other experts in the private sector and among partners in the course of fieldwork. 33 LESTARI MID-TERM EVALUATION USAID.GOV Q14: TO WHAT EXTENT ARE LESTARI ACTIVITIES ALIGNED WITH THE THEORY OF CHANGE? Evaluation Questions Leuser Katingan Cyclops Lorentz Mappi Program Design and Performance 14 To what extent are LESTARI activities aligned with the theory of change? RESPONSE Although LESTARI falls under Development Objective Three of USAID’s Country Development Cooperation Strategy 2014-18, (Global Development Priorities of Mutual Interest Advanced), it also tracks with Development Objective One (Democratic Governance Strengthened) through its work in citizen participation, and with Development Objective Two (Essential Human Services for the Poorest and Most Vulnerable Improved) through its work in public/private partnerships. This theory of change articulates a demand-driven approach to conservation, based upon awareness of linkages between goals and long-term interests of stakeholders, participation in management of resources, and private sector engagement. This is consistent with USAID good practice and in response to top-down approaches to natural resource governance. The simplified results framework is shown in Figure 1. FINDINGS The strategic approaches are clustered in three thematic areas. Landscape-level situational analysis provides guidance on the strategic approaches most suitable for the specific threats to the landscape. The major question is the extent to which these thematic areas accurately reflect and address the threats identified in the project situation analysis. Those threats are identified as: Large Scale Deforestation • Industrial scale palm oil production • Industrial scale timber plantations • Infrastructure development in critical sites • Large scale mining in critical sites Small Scale Deforestation in Critical Areas • Small scale palm oil production • Small scale mining • Small scale logging • Small scale agriculture Overharvesting of Key Commercial Species Illegal Commercial Trade in Wildlife Retaliation for Property Loss Contributing factors identified include: 34 LESTARI MID-TERM EVALUATION USAID.GOV • Unclear land rights • Limited livelihood options • Macro-economic forces • Poor governance at levels in allocating land It is not easy to map how LESTARI’s theory of change corresponds to threats and drivers in the situational model, as described in the results chains under the Strategic Approach. To LESTARI’s credit, as part of its internal collaborative learning and adaptation approach, at mid-term it undertook a political economy analysis (PEA) of each landscape. This was used to refine the situation model based upon more intimate knowledge of the landscape, and strategic approaches have been tailored to issues in the landscape for a more demand-driven approach. The results of this reassessment are not available for consideration in the mid-term evaluation, but it is fair to say that LESTARI’s programing is moving away from “off the shelf” approaches in a very good example of adaptive programming. The PEA identifies opportunities to engage to address industrial oil palm production, a noticeable lacuna in the first two years of the project. Under Forests and Land Use Governance and Advocacy there is Strategic Approach 1, Awareness and Advocacy, 2, Operationalization of SEAs and LCPs, 3, Environmental Governance, also referred to as Sustainable Landscape Governance. • The results chain for Strategic Approach 1 (Awareness and Advocacy) is well formulated, based upon social and behavior communication change practice. Specific tasks and actions address the level of awareness, producing intermediate results addressing specific behaviors leading to desired outcomes. This approach addresses motivation for change. • The results chain for Strategic Approach 2 (Operationalize SEAs/LCPs to Improve Land Use Governance) is more abstract, drawing heavily on activities from other strategic approaches to build towards the adoption of analysis from SEAs and planning from LCPs into budgets and policies. The inputs from awareness and advocacy (Strategic Approach 1) do not clearly translate into buy-in from government, which may explain the uneven and occasionally tepid uptake of this work. In other words, the demand driven approach outlines in Strategic Approach 1 is transformed into a supply driven approach in Strategic Approach 2, a legacy of IFACS. In particular, the rubric of land tenure and property rights is largely missing from land use planning in LESTARI. • Strategic Approach 3 (Sustainable Landscape Governance) also loses sight of the specific problems it is intended to address and marginalizes the governance aspects in favor of a technological approach to system development and operationalization; there is no intermediate result that addresses need or demand, although it is very much implied. The problem in a results chain lies in hidden assumptions, in this case that the client needs are understood, made explicit, and endorsed. Under Conservation Co-Management there is Strategic Approach 4, Co-Management, and 5, Protected Area Management. • Strategic Approach 4 (Improved Forest Management through Co-Management and FMU Strengthening) does a good job of capturing the process of buy-in and the motivations behind stakeholder involvement in co-management of forest resources. Aside from promoting stakeholder involvement in the FMUs, there is, under the AMEP from November 2017, improved articulation of the steps LESTARI must take to strengthen FMUs. More work needs to be done to articulate how 35 LESTARI MID-TERM EVALUATION USAID.GOV FMUs will support co-management in the context of overall institutional strengthening, as specific steps are not articulated in the AMEP. Additionally, the multi-use/multi-stakeholder approach (following a US Forest Service model) requires further analysis to clarify assumptions about how a multi-stakeholder approach works in the Indonesian context. • Strategic Approach 5 (Protected Area Management) presents a well-reasoned and complete set of tasks leading to intermediate results, although 5.12 (advocacy for adoption of METT, SMART, and other initiatives) stresses promotion of the use of these tools, but under-emphasizes the need for sound design and testing procedures to adapt the tools to the Indonesian context. The remaining 3 Strategic Approaches are clustered under Private Sector Engagement. • Strategic Approach 6 (Green Enterprises) glosses over the central requirement for a business plan, apparently assuming that it will emerge from the value chain analysis and partner identification. • Strategic Approach 7 (Private Sector Best Management Practices) fails to clearly articulate the specific steps that would be required to demonstrate the value added to the private sector from the adoption of best management practices. • Strategic Approach 8 (Innovative Financing for Sustainable Land and Forest Management) is predicated upon ecotourism and payment for ecosystem services schemes, but lacks or glosses over critical steps in the assessment of economic opportunities. The rationale for the selection of ecotourism as a principle intermediate result is not explicit. The private sector engagement is the weakest component of the LESTARI program; as written it does not present a strategy that will substantially contribute to LESTARI’s goals. Its activities appear to be somewhat ad hoc. This may be a function of lack of clarity in or understanding of the motivations for private sector engagement in natural resources governance and management, and of risk management from the perspective of businesses, including those businesses targeted by LESTARI. Sixteen indicators were identified to measure progress towards these goals: 1. Percentage reduction of GHG emissions as a result of USG assistance 2. Number of hectares of biological significance and/or natural resources under improved natural resource management as a result of USG assistance 3. Percentage reduction in poaching in focus area 4. Number of public policies addressing climate change and/or biodiversity conservation introduced, changed, or adopted consistent with citizen input 5. Number of sub-national government with improved licensing and permitting mechanisms 6. Number of sub-national government incorporating high quality SEA-LEDS & LCPS into draft spatial plans zonation regulation or draft of mid-term development plan 7. Number of multi-stakeholder forums operational as citizen-based mechanisms for public input on land use 8. Number of community champions engaged in advocacy interventions 9. Number of people reached by LESTARI communication programs to improve awareness and understanding of LEDS and biodiversity conservation 10. Number of conservation areas with at least 70 points in METT scores across LESTARI landscapes 36 LESTARI MID-TERM EVALUATION USAID.GOV 11. Number of Co-management Agreements signed that secure community rights and benefits 12. Number of people recurving USG supported training in natural resources management and/or biodiversity conservation 13. Amount of investment mobilize for climate change as supported by USG assistance 14. Number of people receiving livelihood co-benefits (monetary or non-monetary) 15. Number of private sector firms that have improved management as a result of USG assistance 16. Number of new USG supported public-private partnerships formed. Again, noticeably absent is any measurement of land tenure and resource or property rights. Indicator 3, percentage reduction in poaching in focus area, would produce a figure that is not meaningful in terms of understanding management effectiveness or conservation success, because it doesn’t address attribution. It is difficult to see how indictors 2-16 secure the delivery of indicator 1. Processes, policies, mechanisms, plans, agreements, and training do not necessarily add up to implementation. LESTARI’s strengths are in those areas where it employs demand-driven approaches. These approaches are not without flaws: there are risks of inconsistency, absence of effective technical inputs in design, and challenges in integration within a larger system. In the case of forest and biodiversity conservation, there is also a risk of fragmentation of an ecosystem into units too small to capture ecological processes at scale, and too unwieldy to coordinate for coherent management. LESTARI’s design and implementation have taken many of these risks into account. For example, LESTARI’s approach to policy scales from the local to the regional and back. A parallel project, BIJAK, supports national policy with LESTARI inputs, with the exception of policies related to Reduced Impact Logging, which LESTARI addresses directly. The evaluation team was unable to meet with BIJAK. LESTARI works closely with BAPPENAS to ensure consistency with national strategies and polices. Spatial planning and the use of the landscape approach concentrate the focus to prevent fragmentation. LESTARI has had to adjust to a changing policy environment that has threatened to undermine some of the key assumptions. This has led to some program adjustments in emphasis in policy between provincial and district levels. LESTARI also still struggles with institutional limitations, particularly with regard to the FMUs in LESTARI landscapes, which have limited human capacity and no infrastructure. If it can be embraced by business, LESTARI’s innovative strategy of using the FPIC approach as a framework for planning, as opposed to a framework for regulation, could provide the breakthrough LESTARI needs to get traction with the business community. DATA SOURCES Assessment of relevance of the theory of change summarizes the focus group discussions, key informant interviews, literature reviews, and direct observation on the part of the team. It also includes a review of the Second Edition of the Activity Monitoring and Evaluation Plan, which provides detailed charts of the situation analysis and results chains that inform the theory of change. 37 LESTARI MID-TERM EVALUATION USAID.GOV Key Result 1: Reduce GHG Emissions from Land Use Sector Key Result 2: Key Species Protected Carbon Rich Forest, Peatland, Mangrove Ecosystems and the Habitat of Key Species Maintained Technical Theme 2: Conservation Co￾Management Technical Theme 3: Private Sector Engagement Intermediate Result 1: Improved Forest Management Reduced Threats on Key Species Technical Theme 1: Forest and Land Use Governance and Advocacy Intermediate Result 2: Improved Land Use Governance Reduced Rate of Deforestation and Degradation Figure 1: Simplified LESTARI Results Chain 38 LESTARI MID-TERM EVALUATION USAID.GOV Q15: TO WHAT EXTENT IS THERE BUY IN FOR LESTARI INTERVENTIONS ACROSS THE GOI (NATIONAL, PROVINCIAL, LOCAL, COMMUNITY)? Evaluation Questions Leuser Katingan Cyclops Lorentz Mappi Management Effectiveness and Sustainability 15 To what extent is there buy-in for LESTARI interventions across the GOI (national, provincial, local, community)? RESPONSE There is strong support for LESTARI in government at all levels. The challenge for LESTARI is that there may be too much buy-in, and over-reliance upon LESTARI (see Q 12). FINDINGS All government officials interviewed (at national, provincial, Kabupaten, and community levels) expressed appreciation of, and support for LESTARI’s work, without exception. Specific mention was made of the positive contributions through the RPJMDes process (Q1, Q10), protected area support (Q6), policy harmonization (Q3), and mobilization of stakeholders through the MSF process (Q4). DATA SOURCES Key informant interviews with government officials. Q16: TO WHAT EXTENT IS USAID MANAGEMENT OF LESTARI CONTRIBUTING TO ITS SUCCESSES AND ARE THERE AREAS FOR IMPROVEMENT? Evaluation Questions Leuser Katingan Cyclops Lorentz Mappi Management Effectiveness and Sustainability 16 To what extent is USAID management of LESTARI contributing to its successes and are there areas for improvement? RESPONSE LESTARI is a manifestation of the long-term Comprehensive Partnership between the USA and the Republic of Indonesia. The Joint Statement issued on October 26, 2015, on the occasion of the state visit of President Joko Widodo to Washington, affirmed that the preservation of peatlands and other high-carbon landscapes, as a priority area of cooperation between the two countries. USAID management of LESTARI reflects the high profile of this activity as the implementation of the commitment between the two countries. USAID has been very proactive and supportive of LESTARI in terms of securing high-level government buy-in and agency cooperation. LESTARI and its government partners have also benefited from technical assistance secured by USAID/Indonesia, including by the Forest and Biodiversity office of USAID, the Department of the Interior and the Department of Agriculture. Participation in the “Pause and Reflect” mid-term learning and adaptive management process by USAID was strong and constructive. The evaluation team found evidence of enhanced levels of involvement in direct project management, sometimes beyond what is considered appropriate for contract implementation; this issue was substantially addressed before the evaluation was completed. USAID has taken a decision that may have hindered LESTARI’s ability to engage the private sector, in de-emphasizing engagement in the promotion of sustainable palm oil, because recent decisions under 39 LESTARI MID-TERM EVALUATION USAID.GOV RSPO have created new openings for private sector engagement in conservation. LESTARI is well positioned to provide technical assistance to private sector entities in investments in conservation, discussed under question 6 above. FINDINGS LESTARI uses a comprehensive approach, with strong emphasis on policy engagement across local, district, provincial and national levels; private sector engagement; and demand-driven interventions. It also benefits from effective and engaged government counterparts. This landscape approach differs significantly from the landscape approach used by other USAID projects, including the USAID Central Africa Regional Program for the Environment (CARPE III), USAID/Cambodia’s Supporting Forests and Biodiversity Program (SFB), and USAID/West Africa’s Sustainable and Thriving Environments for West Africa Regional Development (STEWARD IV). CARPE and SFB focused primarily on conservation areas and their buffer zones, whereas STEWARD and LESTARI focused on broader landscapes, with relatively limited activities within the conservation areas. The comprehensive approach being employed by LESTARI provides a useful example more broadly on landscape level approaches to sustainable development, and may, by the end of the project, provide valuable lessons, which warrant an impact evaluation and a comparative study of landscape approaches throughout USAID for adaptive management, including the identification of good practice. The evaluation team’s conceptual framework for the assessment of USAID’s management recognized the high profile of the LESTARI project and the consequent desire for USAID to ensure that this project was appropriately monitored. We looked at technical assistance for the project, and supervisory functions. The team directly observed interference with the direct management functions of a performance contract by the contractor. This included: • Inappropriate communication by USAID personnel with rank and file staff and with communities. This interferes with the direct management functions of a performance contract by the contractor. • Requirements to amend the annual workplan affecting the scope of the contract. Specifically, there is over￾emphasis on FMU support at the expense of activities specifically requested by landscape partners. Sound technical arguments for a balanced program were rejected. • Interference with the ability of the contractor to be responsive to partners and to have effective working relationships with them. This is reflected in the response to Question 11 above. This has the potential to undermine the ability of the project to meet the contract deliverables. LESTARI is at risk of losing senior staff if USAID management issues are not addressed. DATA SOURCES Data comes from confidential interviews with key informants, and reviews of documents such as annotated drafts from the work-planning process. 40 LESTARI MID-TERM EVALUATION USAID.GOV 5. CONCLUSIONS LESTARI is a well-managed and effective program. It is making significant progress in outreach and awareness, as well as in conservation management. However, it has yet to achieve a breakthrough success in private-sector engagement or investment in sustainable forest management. GOVERNANCE AND ADVOCACY LESTARI’s work on governance, particularly at the community level, is well regarded and is building an enduring constituency for forest conservation. Reining in the forces of forest destruction is a complex, long-term process that LESTARI can set in motion but cannot by itself complete. LESTARI has made greatest headway at the community level, where technical support for social forestry is in high demand. The project has launched well-designed efforts in: • Peatland restoration • Land policies to promote forest conservation, particularly for major commodities including palm oil, cacao and coffee. Spatial Plans produced with SEA may in some cases be developed in too top-down a fashion, limiting both community input and buy-in. They do not adequately address land tenure and property rights issues. In Papua, for instance, clan heads customarily govern land use. They do not participate in spatial planning, but instead, an Adat Council, with no authority or mandate in land governance participates on behalf of Adat communities. This produces a disconnect. The evaluation could find no evidence of buy-in or understanding of spatial planning at the clan or community level. The sustainability of spatial planning beyond LESTARI is questionable, even in cases where it is well used now. IMPROVED MANAGEMENT OF CONSERVATION AREAS Through a parallel activity to LESTARI, the U.S. Forest Service (USFS) is working (with USAID support) to support the MoEF and to provide technical assistance for FMU management. (The USFS partnership with USAID is not part of this evaluation’s scope of work). As the world’s largest forest management authority, the USFS can offer considerable technical and managerial capacity to Indonesia. The approach being taken is to fill gaps in the existing management planning framework through the adaptation of the planning tools used in US National Forests. As reviewed in draft form it presents an off the shelf solution not adapted to Indonesia’s context and specific needs.13 In terms of combating wildlife crime, LESTARI lacks a robust, measurable strategy. SMART has recently been adopted, and most jurisdictions using it have not yet put in place a system for data analysis. It is unclear if this will happen before the conclusion of LESTARI. 13 The USFS has produced the draft management toolkit, which appears to be copied verbatim from USFS planning guidelines, including “designated areas” specific to US law, such as Wild and Scenic Rivers, and National Scenic and Historic Trails. 41 LESTARI MID-TERM EVALUATION USAID.GOV We were unable to find evidence that the SMART patrols are successfully preventing wildlife poaching or trafficking. But this does not mean that SMART patrols are ineffectual—the question implies unrealistic expectations of what SMART patrols can do. Moreover, national data on wildlife trafficking and poaching is not available, so there is no baseline against which to measure the impacts of SMART patrols, even where project data are collected. Even with adequate data, attribution of change in wildlife crime is notoriously difficult, as a high number of variables shape the crime environment. While SMART does not eliminate wildlife crime, it is however a potentially important management tool. It allows conservation area and wildlife managers to find patterns in nature and in human behavior with predictive power, as illustrated by Wildlife Conservation Society success elsewhere in the use of game-theoretical algorithms for allocation of resources to combat wildlife crime. Patrols have used SMART to document agricultural encroachment, illegal logging and human/wildlife conflict to good effect. A strategy to promote sengon (A. chinensis, of the mimosa family) plantations by communities on degraded lands enjoys support at the highest levels in Indonesia. Sengon is grown for plywood (barecore), and for biomass energy. It generated nearly USD 500 million in foreign exchange earnings in 2016. Sengon plantations are promoted on degraded lands, to reduce pressure on natural forests and agricultural lands. As a substitute to industrial logging of natural forests, this may be a positive trend in Indonesian forestry. Among proponents are the Secretary General of Ministry of Environment and Forestry Dr. Hadi Daryanto, and prominent members of President Widodo14’s inner circle; including his brother-in-law Hari Mulyono, who chairs the Indonesia Barecore Association. As a strategy for sustainable forestry, sengon in peatlands may require additional study, to determine if it is compatible with peatland restoration. There is potential for weak rubber markets to drive producers toward the planting sengon in peatlands, as a more profitable alternative, and the implications for peatland restoration must be clarified. PRIVATE SECTOR ENGAGEMENT Indonesia currently sees its economic future, including its ability to meet the Sustainable Development Goals, in palm oil production. Therefore, any strategy that rejects oil palm plantations outright is unlikely to succeed. A way forward that includes such plantations while also protecting high conservation value resources and community assets is more politically feasible—and still vastly preferable to wholesale forest destruction. LESTARI is promoting an innovative, FPIC-inspired approach to concession management in the Mappi-Bouven Digoel landscape, involving substantial forest set-asides for cultural 14 Indonesia President Joko Widodo, a furniture manufacturer prior to becoming a politician, has a long record of sengon promotion. In a 2013 speech at Gadjah Mada University while Governor of Jakarta, he called on regional governments to mobilize to plant production forests of sengon. (https://metro.tempo.co/read/1075746/pergerakan-tanah-di-jalur-puncak￾dipantau-dipasangi-bore-pile). 42 LESTARI MID-TERM EVALUATION USAID.GOV and conservation purposes. Through the FPIC approach, communities would work with concession managers to develop and implement a land-use plan. If successful, the approach could be a game-changer in terms of harnessing the power of the private sector for sustainable land management and forest conservation. Success in engaging the private sector on this approach is far from guaranteed, however. Even if successful, it is liable to encounter resistance from conservation pressure groups. Beyond this, however, private sector engagement specifically in mobilizing financing to prevent forest loss is weak. ADDITIONAL OBSERVATIONS BY SPECIFIC LANDSCAPE CAN BE FOUND IN ANNEX III. 43 LESTARI MID-TERM EVALUATION USAID.GOV 6. RECOMMENDATIONS A) TECHNICAL AREA: GOVERNANCE AND ADVOCACY To maximize the sustainability and impact of LESTARI, as well as the innovation capacity of the communities (for instance to adapt to changing market dynamics), we recommend the following: 1. Continue to build, as reflected in the current workplan, upon the successful, well-accepted tool of village planning, the RPJMDes process by communicating success, scaling to additional communities, and building the capacity of government and civil society organizations to replicate the approach and the use of FPIC. Participating in this process has increased the confidence of villagers to actively shape the future of their lives. 2. Where the communities have prioritized sustainable livelihoods through the governance process, LESTARI can help communities to follow through on their decisions by giving them a better understanding of the markets in which they operate, including access to market information. Planning at both the village and livelihood level (e.g. among farmers groups/BUMDes) could incorporate analysis and costing of various economic options. Community members would benefit from greater awareness of basic market mechanisms, how locally important value chains function, and how to calculate the cost of initiatives (e.g., understanding break-even points for revenue creation). These are all things that were broadly absent in the livelihood activities encountered during the evaluation. 3. Prioritize grants on the basis of the ability and interest of grant applicants to share what they learn and mentor communities that are not grant recipients, in the interests of helping LESTARI successes grow to scale. Successfully implemented projects using LESTARI grants could serve as demonstration or learning sites, to help disseminate successful approaches to social forestry and community development. 4. Taking success to scale is a major challenge for LESTARI. In several landscapes, interventions are still limited to a handful of pilot projects. While generating important lessons and providing proof of concept, by themselves they will have negligible impact against the drivers of deforestation and biodiversity loss. LESTARI can achieve scale through a clearly articulated strategy and timetable for taking pilot programs to scale that integrate these strands. Strands that cannot be integrated should be reviewed with a view to their discontinuation 5. Long term impact, which cannot be measured before the termination of LESTARI, requires communication, replication, and mainstreaming through building the capacity of government and civil society organizations. What can be measured is the degree of support for LESTARI innovations among influential leaders. One element of a strategy for sustainability may be to promote a network of the Bupatis with whom LESTARI has enjoyed a strong working relationship. Such a network is an asset to the project, and could help to guide the consolidation and institutionalization of LESTARI innovations, allowing them to ultimately scale beyond the original focal landscapes. B) TECHNICAL AREA: IMPROVED CONSERVATION MANAGEMENT 1. While input from the USFS could ultimately prove useful in this context, at this point, the FMU system and individual units need to prioritize plans for financial self-sufficiency that take into consideration the tenets of the USAID Biodiversity Code and Sustainable Landscapes targets. This is 44 LESTARI MID-TERM EVALUATION USAID.GOV not to say that significant support to build the capacity of the system is not required. A useful approach for LESTARI in the time remaining would be to apply their institutional analysis of the FMUs within the landscapes, help to match policy goals and capacities, prioritize capacity building, with the goal of having a business plan for each FMU at the conclusion of LESTARI. The diagnostic should assess operational and process constraints faced by the FMU system and build consensus on institutional-development priorities. This process should measure the overall institutional health of the individual FMU, including • Organizational structure and mandate, operational processes, and human resource capacity; • The capacity of the organization to conduct evidence-based analysis, solicit feedback from key stakeholders, and design strong policies and plans; • The planning, resources, and capacity available to facilitate implementation of projects and activities; and • The degree to which there is a feedback mechanism for assessing implementation progress. Recommendations from this process would provide a strategy for institutional development and training consistent with USAID’s Human and Institutional Capacity Development policy. Once the basics are addressed, and the components are in place to take advantage of the tools available through the USFS, a comprehensive toolkit can be rolled out. 1. Independent, third-party assessment using the METT scorecard may help to provide a more accurate picture of management effectiveness in conservation areas, especially if coaching on how to interpret the results and recommendations on next steps were included. If BAPPENAS, in cooperation with relevant line ministries, were to lead the METT process, substantial internal learning could take place within the Government of Indonesia, and a similar scorecard approach could be adapted elsewhere. 2. LESTARI should refocus its investment in SMART to accentuate its value as a management tool for evidence-based decision-making. This will entail further work to help users to develop plans based upon an analysis of the data. 3. LESTARI should investigate the use of situational crime prevention techniques for wildlife crime, which are highly congruent with LESTARI’s decentralized, demand-driven principles and the landscape approach. Situational crime prevention takes into account the circumstances in which a crime is commissioned, including motivations and rationale, opportunity, and risk of punishment, and builds support from the bottom up with communities, for support of the rule of law by removing socio￾economic barriers to voluntary compliance, providing a compelling counterweight to coercion. The use of situational crime prevention to combat wildlife crime is a fairly recent innovation, and there are experts working in Southeast Asia who could share their experience with LESTARI and help the project integrate the approach if desired. C) TECHNICAL AREA: PRIVATE SECTOR ENGAGEMENT 1. To increase the production of commodities for sustainable livelihoods at levels sufficient to reverse deforestation trends requires that LESTARI thoroughly assess local economic development opportunities and provide corresponding technical advice including: • The economic feasibility of commodities (long-term trends); 45 LESTARI MID-TERM EVALUATION USAID.GOV • Containment of risk, e.g., through diversification; • The opportunity cost of giving up illegal practices using forest resources; and • The costs and benefits of different types of certification, and the likelihood of sustainability of approaches being promoted, and to determine if alternative methods of “conservation branding” may be used that can be achieved within the time available. LESTARI will need to evaluate the possibility of accomplishing this within available time and resources. If they are insufficient to address this issue thoroughly, USAID may wish to consider redeploying resources to areas where there are stronger chances of success. 1. If LESTARI is to sustainably achieve its economic, social, and ecological goals, it will need to encourage both policy-makers and producers to develop appropriate agroforestry approaches in the peatlands. Consultation with the Center for International Forestry Research (CIFOR) and the World Agroforestry Center (ICRAF) may be useful. 2. Greenhouse gas emissions and energy requirements of commodity-oriented partnerships will need to be monitored, as planned, to ensure that the costs do not outweigh the benefits in terms of LESTARI’s goals. An additional benefit of monitoring may be the possible role that LESTARI could play in creating enabling environments for other projects and investors. Additionally, LESTARI may be able to make clean energy options more accessible to its local partners. One way would be to sharing lessons from USAID’s own Indonesia Clean Energy Development project. Another option may be to factor support for GHG emissions reductions into LESTARI partnerships with the private sector. Buyers, especially larger brands, are increasingly asked to disclose carbon emissions, and carbon emission impacts on environmental and socio-economic conditions. If LESTARI can support GHG monitoring with verifiable data, this may benefit potential private sector partners. 3. In order to increase access to and use of long term financing, LESTARI should increase efforts to engage with the private sector and with not-for-profit entities working to promote environmental and social governance and green investment as discussed in findings under questions 6, 8, and 9. It is clear that the palm oil industry involves some sensitivities that make it difficult to work with. While not advocating that LESTARI promote sustainable palm oil production per se, it is noteworthy that the segment of the private sector catering to global markets seeks Round Table on Sustainable Palm Oil (RSPO) certification. RSPO principles and criteria embrace FPIC principles and now require investment in restitution for past harm. This creates an opening for LESTARI to enter into strategic partnerships with organizations that do directly engage the industry, such as Forest Carbon and its subsidiary LESTARI Capital (no relationship to the LESTARI project). Linking set-asides in existing concessions for community and high conservation value land may be linked to active restitution, thus helping to advance LESTARI goals for mobilizing private sector investment. 4. LESTARI should also be willing to abandon non-productive initiatives to mobilize conservation finance. The lack of progress on payment for ecosystem services schemes suggest that they should be re-examined and dropped if there is not strong support. 5. Important progress may be underway in the plans for a fund for the Lorentz Lowlands. LESTARI should pay particular attention to the governance dimensions of this fund, to ensure that it does promote community buy-in, and does not facilitate “policy capture” or undue influence of private sector partners in conservation decisions. 46 LESTARI MID-TERM EVALUATION USAID.GOV D) MANAGEMENT EFFECTIVENESS AND SUSTAINABILITY 1. Many partners complained about an excessive workload for administering and reporting; this is not surprising as USAID reporting requirements are rigorous. LESTARI may help by improving communications with partners, and by ensuring a satisfactory response time on requests and decisions. Specifically, it should have standards for responding to partners in a timely manner, including for decisions, and give additional guidance to subcontractors and grantees about requirements when making requests, as inadequate attention to such requirements is a leading cause of delays. 2. LESTARI should review sustainability plans and curtail these activities if they do not enjoy buy-in or the likelihood of continued investment post-project. For example the spatial plans discussed under Conclusions above should be reviewed for sustainability. 3. It may benefit USAID to have “cross-pollination” through interdisciplinary program teams or collaborative learning and evaluation activities with corresponding experts in the Democracy and Governance sector. Additionally, under the America First policy, it is reasonable to expect a renewed focus on economic growth issues. Should this transpire, it would be beneficial to link green development with private investment. 47 LESTARI MID-TERM EVALUATION USAID.GOV ANNEX I: EVALUATION SCOPE OF WORK I. PURPOSE OF THE EVALUATION The purpose of the Midterm Performance Evaluation (henceforth called "the evaluation") of LESTARI is to provide USAID/ Indonesia and the Ministry of Environment and Forestry (MoEF) with an independent review of progress toward agreed targets for reduced terrestrial emissions of greenhouse gases and improved conservation of forests and biodiversity. Recommendations from the evaluation will feed into adaptive management for remainder of the activity. The evaluation is expected to highlight progress towards expected results as well as challenges in implementation. USAID/Indonesia, Government of Indonesia counterparts at national and subnational levels, implementing partner(s), and other key stakeholders are expected to incorporate agreed recommendations into annual work plans, monitoring and reporting, and other aspects of ongoing LESTARI implementation. It will also inform Agency-level biodiversity and climate change stakeholders. If changes to overall LESTARI metrics and/or targets are indicated, those changes will be discussed with the implementing partner and may involve modifications to the contract. Recommendations may also lead to requested changes in implementation approaches, resource allocations, personnel structure, or other adaptive actions to lead to programmatic success across components. The evaluation also will assess progress toward intermediate objectives agreed in the contract and/or work plans. All six landscapes will be included (Aceh, Central Kalimantan, and Papua). II. SUMMARY INFORMATION Strategy/Project/Activity Name LESTARI Implementer Tetra Tech ARD, Inc. Cooperative Agreement/Contract# AID-497-T0-15-00005 Total Estimated Ceiling of the Evaluated Project/ Activity(TEC) $46,831,909.00 Life of Strategy, Project, or Activity July 2015 - July 2020 Active Geographic Regions Six landscapes (detailed below) in the provinces of Aceh, Papua, and Central Kalimantan Development Objective(s) (DOs) DO 3: Global Development Priorities of Mutual Interest Advanced USAID Office: USAID/Indonesia III. BACKGROUND LESTARI aims to reduce land-based greenhouse gas (GHG) em1ss1ons and conserve biodiversity in carbon rich and biologically significant forest and mangrove ecosystems in priority landscapes in Indonesia. LESTARI builds on achievements of the Indonesia Forests and Subcontract Agreement No. 610900 .01.17.155 I page 5 of 46 Climate Support (IFACS) project to address drivers of deforestation and degradation and to conserve important ecosystems and species. LESTARI will reduce emissions and conserve biodiversity in selected landscapes through improving (a) district land-use governance, (b) management of conservation areas and protection of key species, (c) private sector and industry practices, and (d) constituencies for conservation among relevant government and nongovernmental stakeholders. LESTARI is implemented by Tetra Tech with a consortium of partners that includes WWF-lndonesia, Wildlife Conservation Society, Winrock International, Blue Forests, Yayasan Sahabat 48 LESTARI MID-TERM EVALUATION USAID.GOV Cipta, PT Hydro South Pole Carbon, Sustainable Travel International, Michigan State University, and the FIELD Foundation. LESTARI runs from July 2015 to July 2020. A. Description of the Problem, Development Hypothesis(es), and Theory of Change Indonesia is considered a megadiverse country, a result of unique biogeography within the humid tropics: the dynamic, volcanic archipelago falls at the intersection of the Oriental and Australian biogeographic regions. The country is composed of 17,493 islands, leading to unique ecosystems and species and a high rate of endemism. Globally valued species such as the orangutan, tiger, rhino, elephants and birds of paradise represent just a fraction of Indonesia's unique biodiversity. Important terrestrial ecosystems include mangroves, lowland forests, peat/swamp forests and montane forests. Indonesia has Asia's largest continuous tract of tropical rain forests and 10% of the world's forests. Much of the remaining intact forests are in the remote mountainous regions of the country. It has important carbon rich swamp/peat forests and soils, the protection of which is critical for preventing or reducing land-based carbon emissions. Healthy intact forest ecosystems are also an important safety net for local communities living nearby, presenting economic opportunities through forest dependent livelihoods that prevent their slide into extreme poverty and improving their resilience to climate and other external shocks and stresses. While at least 30 million people directly depend on Indonesia's forests and on the ecosystem services they provide, these ecosystems have been and continue to be under extreme threat. Indonesia's deforestation rate is among the highest in the world, despite improvements in the past decade. Deforestation is most critical in the peat and swamp forests of Sumatra, Kalimantan and Papua. Currently, the conversion of forests to oil palm plantations is the most obvious and direct threat to Indonesia's remaining forests. In some cases unsustainable industrial scale logging precedes conversion to palm oil or other commodities. Forests additionally are threatened by encroachment by smallholders and shifting cultivators, who often are squeezed out of available land and resources by larger private sector actors and inequitable and opaque land use decisions. Forest fires, particularly those occurring on peatlands, are a significant contributor to Indonesia's emissions and another threat to forests \]. High temperatures and drought associated with the 2015 El Nino phenomenon exacerbated annual fires and are estimated to have resulted in 1.5 billion tons of carbon being released into the atmosphere. Deforestation and forest degradation in Indonesia result from a complex dynamic of political, economic and institutional drivers. Decision-making on land use involves a large array of laws and actors and the process is rarely transparent or consistent. More importantly, land use decisions are driven by economic incentives that prioritize extractive industries and agriculture over low emissions and conservation￾oriented business models. Numerous actors participate in decision-making at the national, provincial and district levels, including central government officials and legislators, district and provincial officials and legislators, private companies and communities. The differential levels of influence among these actors, however, often means that the voices, capacity, and opportunities for participation are limited by rigid government processes, lack of valid avenues for engagement, and perverse incentive and power structures that marginalize some groups. To address these challenges, USAID/Indonesia developed a situation model that identifies ecosystem and biodiversity focal interests and associated threats, pressures and drivers impact these focal interests. The overall development hypothesis is that better forest and land use governance, increased conservation co-management, and stronger private sector engagement will result in improved forest 49 LESTARI MID-TERM EVALUATION USAID.GOV management and land-use governance that reduces the rate of deforestation and degradation while avoiding the over-exploitation of key species. The approach can be summarized as follows: increased awareness and advocacy of local stakeholders for conservation and natural resource management in ways that contribute to their long-term interests, along with enhanced conservation co-management to the benefit of key stakeholders , and engagement by the private sector, will lead to improved governance and natural resource management in biologically significant and critical areas, leading to a 41% reduction in greenhouse gas (GHG) emissions by 2020. The overall goal of LESTARI is that, at the end of five years, USAID assistance will have reduced land based GHG emissions, and conserved valuable biodiversity, in carbon rich and biologically significant forest and mangrove ecosystems within selected landscapes. This goal is expected to be achieved as the outcome of two intermediate results: improved forest management and improved land use governance. Key results to be achieved by the completion of the project at the goal and intermediate level are: GOAL • At least 41% of total CO2-equivalent emissions reduced from land use, land use change and deforestation averaged across all landscapes within the project scope; and • At least 8.42 million hectares of primary or secondary forest (and peatland), including orangutan habitat, under improved management. EXPECTED OUTCOMES Improved Forest Management • Management of at least six conservation areas improved, resulting in the conservation of valuable orangutan and other key species habitat, and the reduction in poaching of threatened and endemic species; • At least ten public-private partnerships (PPPs) promoting low-emissions conservation oriented development established; and • Funding leveraged from public and private sources, representing co-investment in project outcomes. Improved Land Use Governance • Increased commitment of key private sector, government, and community stakeholders regarding the positive benefits of conservation and sustainable use of forests and the species they encompass; • Policies, laws, regulations, and procedures in support of low emission development and forest conservation and management increased, promulgated, and enforced at all levels; and • Model(s) for successful integration of district, provincial, and national low emissions development and forest conservation strategies developed and shared at all levels of government and with other key stakeholders. LESTARI developed eight strategic approaches or results chains (RCs) that indicate steps in the adaptive management process towards achieving the goal and outcomes. These are organized into three complementary technical areas: 50 LESTARI MID-TERM EVALUATION USAID.GOV Forest and land-use governance and advocacy 1. Awareness and advocacy a) Increase government awareness, appreciation and advocacy for governance; b) Improve local level awareness, appreciation and advocacy for governance; and c) Empower media coverage of environmental issues 2. Operationalize strategic environmental assessments and landscape conservation plans a) Operationalize strategic environmental assessments and landscape conservation plans - many of which were developed under IFACS 3. Environmental governance a) Improve licensing and permitting processes; b) Monitor and enforce land use; c) Strengthen citizen-based mechanisms for public input on land use; and d) Increase district governments' willingness to adopt input from citizen-based mechanisms Conservation co-management 4. Co-management a) Improve co-management adjacent to conservation areas; b) Pilot innovative financing for critical areas; and c) Improve co-management adjacent to critical areas 5. Protected areas management a) Improve conservation area management capacity; b) Pilot innovative financing for conservation areas; and c) Enhance national-level policy coordination Private sector engagement 6. Green enterprises a) Develop public-private partnerships for green enterprises; and b) Improve sustainable livelihoods and alternative livelihoods for communities adjacent to critical areas 7. Private sector best management practices a) Develop and disseminate industry certification, best management practices, and conservation management and mitigation plans 8. Payment for ecosystem services and REDD 1 innovative finance a) Enhance district readiness to access and effectively utilize financing 51 LESTARI MID-TERM EVALUATION USAID.GOV Field implementation follows a landscape approach, where the eight strategic approaches are adapted to the biophysical, socioeconomic, political, and cultural context within each landscape. LESTARI interventions are targeted in six strategic landscapes on three of Indonesia's largest islands, where primary forest cover remains most intact and carbon stocks are greatest. These are summarized in the table below. Figure 1 depicts the location of each LESTARI operational landscape within Indonesia. REDD+ is reducing emissions from deforestation and forest degradation plus the role of conservation, sustainable management of forests and enhancement of forest carbon stocks. It is a mechanism developed by Parties to the U.N. Framework Convention on Climate Change that creates a financial value for the carbon stored in forests by offering incentives for developing countries to reduce emissions from forested lands and invest in low-carbon paths to sustainable development through payments for results. 52 LESTARI MID-TERM EVALUATION USAID.GOV B. Summary of the Project/Activity Monitoring, Evaluation, and Learning (MEL) Plan The LESTARI Activity Monitoring and Evaluation Plan (AMEP) is framed by the Situation Model and Theories of Change/Results Chains that emerged from the model (Annexes A and B). An overarching Results Framework (RF) was developed (Figure 2) to explain how LESTARI Strategic Approaches will address the drivers of deforestation and forest degradation and reduce emissions and threats to biodiversity presented in the Situation Model. The RF demonstrates causality and the logical linkages between shorter-term, intermediate, and longer-term results. The Results Framework is founded upon the overarching theory that if there is increased awareness and advocacy of local stakeholders (government officials, communities, CSOs and private sector) for conservation and natural resource management in ways which contribute to their tong term interests; if Conservation Co￾Management is actually enhanced to the benefit of key stakeholders, and if the private sector is engaged, there will be improved governance and natural resource management in biologically significant and critical areas 2 , leading to a 41% reduction in GHG emissions by 2020. The Results Framework was then translated into a matrix showing how LESTARI plans to integrate LESTARl's Technical Components into the eight strategic approaches that can be nuanced according to the context of each targeted landscape. This matrix also demonstrates our theory of change for each Strategic Approach, in accordance with the Results Chains provided in Annex B: Results Chains. 53 LESTARI MID-TERM EVALUATION USAID.GOV IV. EVALUATION QUESTIONS, DESIGN AND MANAGEMENT Because the final results and outcomes expected from LESTARI are the reduction of terrestrial emissions of GHGs and the protection of key species within the six landscapes, the evaluation will center on addressing the set of questions below related to the extent of progress toward overall LESTARI goals and objectives as well as the soundness of the strategic approaches and additional questions related to operational efficiency and effectiveness of LESTARI and its implementing partners. Guiding Question: Is LESTARI on track to achieve its goal to bring at least 8.42 million hectares of primary or secondary forest (and peatland), including orangutan habitat, under improved management, ultimately reducing deforestation? The team must consider LESTAR l's theory of change and results chains and the following factors under this question. TECHNICAL COMPONENT 1: Forest and Land Use Governance & Advocacy Are the activities under Technical Component 1, Improved Land Use Governance, sufficient to address oblique land use decision-making resulting in forest loss in the landscapes? At this point in LESTARI project implementation, the following questions related to LESTARl’s 's theory of change under this component could be considered: 1) Are LESTARI outreach activities, including social media campaigns, story pitching, interactive dialogues, and journalist trainings, sufficient to build constituencies for conservation and influence the forest and land use policies? (Awareness and Advocacy - LESTARI 1) 2) Are LESTARI awareness and advocacy activities effective in identifying target audiences, desired behaviors, and the appropriate media? How has LESTARl's target audience retained, adopted, and implemented the desired knowledge they have gained through LESTARI? (Awareness and Advocacy- LESTARI 1) 3) Is LESTARI support on operationalization of SEAs and LCPs preventing conversion of critical areas into the other purpose area (APL)? (Operationalize SEAs/LCP to improve Land Use Governance - LESTARI 3) 4) Are LESTARI SENLCP recommendations being incorporated into other governance documents or processes such as spatial plan revisions, Provincial/District development plans for forest management and land use, draft regulations, or District/Provincial Development Plans (RPJM-D and RPKD)? (Operationalize SEAs/LCP to improve Land Use Governance - LESTARI 3) 5) Are LESTARI investments to strengthen citizen-based Multi-Stakeholder Forums (MSF), sufficient to influence land use decision-making, monitoring and planning? (Sustainable Landscape and Forest Governance - LESTARI 2) 6) Are LESTARI activities to increase citizen-based influence on forest management and land use decision, effectively channeled to BIJAK to influence national level policymakers? (Sustainable Landscape and Forest Governance - LESTARI 2) 7) Are LESTARI activities to developing systematic and transparent monitoring of land use, sufficient to encourage provincial/district governments to monitor land use and licensing? (Sustainable Landscape and Forest Governance - LESTARI 2) 8) Are LESTARI activities to improve forest management through co-management and FMU capacity building sufficient to increase forest stewardship, ultimately contributing to 8.4 Million of biological significance and/or natural resources under improved natural resource management? (Improve Forest Management Through Co-Management LESTARI 4) 54 LESTARI MID-TERM EVALUATION USAID.GOV TECHNICAL COMPONENT 2: Improved Management of Conservation Areas Are the activities under Technical Component 2, Improved Management of Concession Areas & Protection of Key Species, sufficient to improve the management of Conservation Areas (CAs) and other critical areas of high biodiversity within the LESTARI landscapes to conserve valuable habitat for species and preserve ecological function, the goods and services they provide, as well as stem the loss of key wildlife and endemic species that are under high threat from poaching? At this point in LESTARI project implementation, the following questions related to LESTARl's theory of change under this component could be considered: 1) Are the LESTARI activities to improve METT scores sufficient to improve management of Conservation Areas (CAs)? 2) Are the LESTARI activities under LESTARI 1 - Awareness and Advocacy and LESTARI 2 - Sustainable landscape governance, sufficient to establish constituencies for conservation that support CA management? 3) Is LESTARI support for SMART patrol sufficient to combat wildlife trafficking and reduce poaching in the landscapes? TECHNICAL COMPONENT 3: Improved Private Sector Practices Are the activities under Technical Component 3, Improved Private Sector and Industry Practices, sufficient to engage and leverage private sector entities within USAID focal areas to embrace low emissions conservation oriented business models and practices that reduce deforestation and degradation and maintain important species habitat? At this point in LEST ARI project implementation, the following questions related to LESTARl's theory of change under this component could be considered: 1. Are LESTARI activities to promoting public private partnerships (PPP) for green enterprise in the landscape sufficient to mobilize green investment and improve sustainable commodity production? Is this leading to a reduction in deforestation or forest degradation? 2. Is LESTARI support on innovative financing for sustainable land and forest management sufficient to secure long term financing mechanism for conservation and restoration in the landscapes? 3. How have LESTARI activities under this component changed government or private sector thinking about how forest conservation can positively impact local economic development? Evaluation questions related to Management Effectiveness and Sustainability: o How is the implementation structure and management of LEST ARI contributing to its successes and are there areas for improvement? o Is LESTARI engaging and empowering the right set of stakeholders-in terms of their impact on and influence over the resource -to reduce deforestation and threats to biodiversity in their landscapes. o Will the key institutions that LEST ARI is strengthening continue the work after USAID funding is phased out? The evaluation team will be expected to develop a detailed design and methodology based on their expertise, understanding of the evaluation requirements and available documentation. LESTARI is a wide-ranging activity designed to tackle multiple aspects of a complex set of challenges within highly nuanced contexts in six landscapes. USAID recognizes that limitations on data quality and availability; 55 LESTARI MID-TERM EVALUATION USAID.GOV multiple sets of public, private and community stakeholders, often with differing objectives; travel difficulties to field sites; and other factors all affect the ability to conduct a comprehensive performance evaluation within the time and budget available. In this section, some general guidelines are provided on methodological aspects of the evaluation. The evaluation team will present its recommendations on how to balance these factors in its Evaluation Design deliverable (see below) The overall design will rely on mixed methods for data gathering and analysis. Quantitative data from LESTARI reports and other sources will be combined with qualitative data, again from LESTARI and other sources. Each evaluation question is likely to be best addressed through a different mixture of internal (USAID and LESTARI implementing partners) and external (all others) sources, of quantitative or qualitative information, and of methods for gathering and analyzing that information. Tools used to gather information will also vary, depending on the evaluation question. Collecting primary quantitative data will be impractical, but otherwise the evaluation team may recommend a combination of literature review (including reports and relevant scientific literature), review of monitoring data and data from secondary sources, information gleaned from key informant and focus group interviews, or other efficient data gathering methodologies. Effort should be made to obtain diverse views and triangulate key findings across different stakeholder groups. The team will propose data collection and analysis methods that will be reviewed and approved as part of the design deliverable. The team will be expected to analyze and synthesize the information rather than merely summarize it. That is, USAID/Indonesia will be less interested in recitation of interview responses than in an analysis of what those responses mean in the context of addressing the evaluation questions. USAID/Indonesia also expects to receive an indication of the quality of evidence associated with each finding and recommendation. To the greatest extent possible within available time and budget, the evaluation team will make field site visits to each of the six LESTARI landscapes. The Evaluation Design will include a recommended set of sites to visit; indicative timing of visits; personnel to be involved from the evaluation team, LESTARI team, USAID/Indonesia, and other stakeholders; and other relevant details on field site data gathering (e.g., roundtables, key informant interviews, focus group discussions, etc.). It may not be necessary, efficient or productive to conduct site visits in all 14 Districts where LESTARI operates. When feasible and practical, the evaluation team will assess gender equity and women's empowerment aspects of each evaluation question. Some of the questions have a clearer and more relevant gender dimension than others. If possible, the evaluation and discussion of results could include ethnolinguistic, age, socioeconomic status, or other disaggregation relevant to the evaluation questions. V. DELIVERABLES AND REPORTING REQUIREMENTS The following deliverables will be prepared and delivered by the evaluation team at different stages of conducting the LESTARI evaluation. Changes to the deliverables - including timing, content, approvals, etc. - must be agreed in advance by USAID/Indonesia before they are implemented. 1. Evaluation Design: Within 20 days of the award for the LESTARI evaluation, the evaluation team must submit to the COR an evaluation design (which will become an annex to the Evaluation report) of no more than 35 pages. The evaluation design will include: (1) a detailed evaluation design matrix (attached to this SOW in Annex) that links the evaluation questions in the SOW to data sources, methods, and 56 LESTARI MID-TERM EVALUATION USAID.GOV the data analysis plan; (2) draft data collection instruments or their main features; (3) a list of potential interviewees and sites to be visited and proposed selection criteria and/or sampling plan (must include calculations and a justification of sample size); (4) known limitations to the evaluation design; and (5) a dissemination plan. The evaluation design will also include a brief discussion of how the evaluation team plans to mitigate potential conflicts of interest between the evaluation contractor and the LESTARI prime contractor, given their shared corporate ownership. 2. Workplan: The draft work plan will be 10 pages or less and include: (1) the anticipated schedule and logistical arrangements necessary to conduct the evaluation; (2) a list of the members of the evaluation team, delineated by roles and responsibilities; and (3) a preliminary indicative timeline for each deliverable, factoring in review periods. 3. The COR will provide comments, request changes, or approve the Evaluation Design and Workplan - as appropriate -within ten business days of receipt. 4. In-briefing inception report: Within three business days of arrival in Indonesia, the evaluation team will have an in-briefing with the USAID/Indonesia Environment Office, Front Office/Program Office, BAPPENAS, and Ministry of Environment and Forests for introductions and to discuss the team's understanding of the assignment, initial assumptions, evaluation questions, methodology, and workplan, and/or to adjust the Statement of Work (SOW), if necessary. The evaluation team's meeting notes will serve as an Inception Report deliverable. 5. Interim Briefings: The team will provide the evaluation COR/manager with periodic briefings and feedback on the team's findings, as agreed upon during the in-briefing. If desired or necessary, weekly briefings by phone can be arranged. 6. Exit Briefing I Presentation: The evaluation team will participate in an exit briefing prior to leaving Indonesia to discuss the status of data collection, analysis, and preliminary summary of findings. It is understood that recommendations would be premature until analyses are completed; however, if any "red flag" issues have emerged during the evaluation, those should be discussed at the exit briefing so that USAID/Indonesia may take more immediate corrective action as appropriate. The presentation portion of the exit briefing will consist of an electronic deliverable in PowerPoint or other agreed format, and given in a level of detail appropriate for the audience. Details for the exit briefing and presentation will be agreed upon during the in-briefing, consisting of approximate schedule, location, expected maximum length, preliminary list of attendees (e.g., whether or not to invite LESTARI staff, Ministry officials, or other stakeholders), and other details as necessary. These details may need to be adjusted if specific sensitivities arise during the evaluation, and will be finalized no later than five business days before the exit briefing and presentation is delivered. 7. Draft Evaluation Report: The draft evaluation report should be consistent with the guidance provided in Section IX: Final Report Format. The report will address each of the questions identified in the SOW and any other issues the team considers to have a bearing on the objectives of the evaluation. Any such issues can be included in the report only after consultation with USAID. The submission date for the draft evaluation report will be determined in the evaluation work plan. [A good practice is for the evaluation team to share an early draft or detailed outline that includes main findings and bullets before completing the draft evaluation report.] Once the initial draft evaluation report is submitted, USAID/Indonesia will have 20 business days in which to review and comment on the initial draft, after 57 LESTARI MID-TERM EVALUATION USAID.GOV which point the COR will submit consolidated comments to the evaluation team. The evaluation team will then submit a revised final draft report not more than 1o business days after receiving USAID comments, and USAID/Indonesia again will review and send comments within 15 business days of its submission. 8. Socialization I Dissemination of Recommendations: After USAID review of the draft evaluation report, the evaluation team will organize and convene one or more roundtable discussion with stakeholders at USAID/Washington and possibly with contractor home office staff. 9. Final Evaluation Report: Again, the evaluation team will take no more than 10 business days to respond to and incorporate final comments from USAID/Indonesia. The evaluation team leader will then submit the final report to the COR. All project data and records will be submitted in full and should be in electronic form in easily readable format, organized and documented for use by those not fully familiar with the intervention or evaluation, and owned by USAID. VI. EVALUATION TEAM COMPOSITION The team should comprise not more than five individuals including as many as three local evaluators. All should be proficient in English and have excellent analytic and communication skills. The team may be supplemented by one technical staff from a GOI counterpart institution. • Team Leader, evaluation specialist. At least ten years' professional experience with significant experience evaluating forest and landscape conservation programs. Masters level, PhD preferred in social or environmental science with strong proficiency in evaluation methodologies. • Deputy Team Leader, institutional and human capacity development specialist. Masters level, at least ten years' professional experience, ideally in the forestry sector. Will also play major role in organizing, editing and producing the evaluation report. Must have excellent writing skills. • Biodiversity specialist. At least five years' professional experience in research, evaluation or program management of landscape scale biodiversity projects. Practical experience in project implementation or evaluation, knowledge of landscape and protected areas management. Masters level, PhD preferred in conservation biology, ecology or related field. Experience with conservation financing preferred. • Land use and land governance specialist. At least five years' professional experience. Focus on approaches for climate change mitigation and low emissions development. Practical experience in project implementation or evaluation. Masters level, PhD preferred in climate science, forest ecology or forestry. • Private sector specialist in forest and green enterprises. At least ten years of experience, knowledge of Indonesian business environment. Focus on assessment of management practices. Background in business studies, economics or related field. Experience with conservation and climate change projects preferred. Experience with alternative forest based livelihoods preferred. All team members will be required to provide a signed statement attesting to a lack of conflict of interest or describing any existing conflict of interest: that they are not currently staff or a beneficiary of LESTARI, its prime and subcontractors. The evaluation team shall demonstrate familiarity with USAID's evaluation policies and guidance included in the USAID Automated Directive System (ADS) in Chapter 200. One staff member from the Ministry 58 LESTARI MID-TERM EVALUATION USAID.GOV of Environment and Forestry will participate on the evaluation team in to participate in one site visit and provide overall input on GOI perspectives. VII. FINAL REPORT FORMAT The evaluation final report should include an abstract; executive summary; background of the local context and the strategies/projects/activities being evaluated; the evaluation purpose and main evaluation questions; the methodology or methodologies; the limitations to the evaluation; findings, conclusions, and recommendations. For more detail, see "How-To Note: Preparing Evaluation Reports" and ADS 201mah, USAID Evaluation Report Requirements. An optional evaluation report template is available in the Evaluation Toolkit. An abstract of not more than 250 words will briefly describe what was evaluated, evaluation questions, methods, and key findings or conclusions. The abstract should appear on its own page immediately after the evaluation report cover. The executive summary should be 2-5 pages in length and summarize the purpose, background of the project being evaluated, main evaluation questions, methods, findings, conclusions, and recommendations and lessons learned (if applicable). The executive summary will also be translated into Indonesian. The evaluation methodology shall be explained in the report in detail or summarized in the report and detailed in an Annex. Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (e.g., selection bias, recall bias, unobservable differences between comparator groups, etc.). The annexes to the report shall include: • The Evaluation SOW - attaching a revised version if applicable and based on COR agreement; • Any statements of difference regarding significant unresolved differences of opinion by funders, implementers, and/or members of the evaluation team; • All data collection and analysis tools used in conducting the evaluation, such as questionnaires, checklists, and discussion guides; • All sources of information, properly identified 3 and listed; • Signed disclosure of conflict of interest forms for all evaluation team members, either attesting to a lack of conflicts of interest or describing existing conflicts of interest; and • Summary information about evaluation team members, including qualifications, experience, and role on the team. In accordance with ADS 201, the contractor will make the final evaluation reports publicly available through the Development Experience Clearinghouse within three months of the evaluation's conclusion. VIII. CRITERIA TO ENSURE THE QUALITY OF THE EVALUATION REPORT Per ADS 201maa, Criteria to Ensure the Quality of the Evaluation Report, draft and final evaluation reports will be evaluated against the following criteria to ensure the quality of the evaluation report. APA format preferred but not required; other acceptable formats are Council of Science Editors or Chicago Manual of Style 59 LESTARI MID-TERM EVALUATION USAID.GOV Evaluation reports should represent a thoughtful, well-researched, and well-organized effort to objectively evaluate the strategy, project, or activity. • Evaluation reports should be readily understood and should identify key points clearly, distinctly, and succinctly. • The Executive Summary of an evaluation report should present a concise and accurate statement of the most critical elements of the report. • Evaluation reports should adequately address all evaluation questions included in the SOW, or the evaluation questions subsequently revised and documented in consultation and agreement with USAID. • Evaluation methodology should be explained in detail and sources of information properly identified. • Limitations to the evaluation should be adequately disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). • Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or simply the compilation of people's opinions. • Findings and conclusions should be specific, concise, and supported by strong quantitative or qualitative evidence. • If evaluation findings assess person-level outcomes or impact, they should also be separately assessed for both males and females. • If recommendations are included, they should be supported by a specific set of findings and should be action-oriented, practical, and specific. IX. OTHER REQUIREMENTS All quantitative data collected by the evaluation team must be provided in machine-readable, non-proprietary formats as required by USAID's Open Data policy (see ADS 579). The data should be organized and fully documented for use by those not fully familiar with the project or the evaluation. USAID will retain ownership of the survey and all datasets developed. All modifications to the required elements of the SOW of the contract/agreement, whether in technical requirements, evaluation questions, evaluation team composition, methodology, or timeline, need to be agreed upon in writing by the COR. Any revisions should be updated in the SOW that is included as an annex to the Evaluation Report. 60 LESTARI MID-TERM EVALUATION USAID.GOV ANNEX II: LESTARI MTE FINAL WORKPLAN AND APPROVED ADDENDUM LESTARI Midterm Evaluation Workplan & Methodology Introduction Integra is pleased to present the LESTARI Midterm Evaluation (MTE) work plan to USAID/Indonesia. This document is based on the Statement of Work (SOW) and discussions with Keri Culver, Quality Control Advisor for USAID/Indonesia’s MESP implementer. The following pages outline the methodology the team intends to utilize and the tasks, roles and responsibilities of the team necessary for completion of the assignment. The purpose of the LESTARI task order is to conduct a midterm performance evaluation of the USAID funded LESTARI program. The evaluation is being conducted to provide USAID/Indonesia and the Government of Indonesia’s Ministry of Environment and Forests (MoEF) with an independent review of progress toward the program’s targets for reductions in greenhouse gas emissions and conservation of forests and biodiversity. The evaluation is intended to help USAID/Indonesia, Government of Indonesia counterparts at national and subnational levels, implementing partners and key stakeholders to assess the status of the program, and incorporate recommendations into the strategy, implementation and monitoring of LESTARI. Work Plan Based on past experience in Indonesia, the Integra team understands that this work plan, inevitably, will need to be flexible and responsive to changing logistics in order to achieve the MTE objectives. A graphic depiction of the work plan details is provided below. The following narrative highlights high level issues and processes. Start Up Tasks associated with start-up focus on preparing the Integra team for setting the groundwork and designing an evaluation approach consistent with USAID Evaluation Policy and best practice. Upon contract execution, the Integra team initiated logistics, administrative and technical work due to the rapid start up and fielding of the team. The Team Lead and Senior Manager are conducting relevant interviews in the Washington, DC area with various LESTARI stakeholders to inform and refine the evaluation methodology in mid-October. The approval of this Deliverable 1 (Work plan and evaluation design approval) sets in motion the fieldwork phase of the evaluation. The evaluation team is expected to mobilize on schedule and Integra expects to meet the requirements for Deliverable 2 (In-brief with USAID/Indonesia) in the first full week of November. 61 LESTARI MID-TERM EVALUATION USAID.GOV Fieldwork Upon arrival in Jakarta, the Integra team proposes that a directed and purposeful in-brief take place with the LESTARI COR, MESP COR, Mission M&E officer and other key Mission staff on Monday, November 6. The purpose of this in-brief are twofold: (1) to conduct an in-person review of the SOW and (2) to confirm the evaluation methodology. To this second purpose, the in-brief will provide an opportunity to discuss any logistical challenges which may require an adjustment to the work plan, such as issuance of Papua travel permits and air travel logistics. The Integra approach to fieldwork will require the evaluation team break into sub-teams to provide adequate time and coverage at each landscape. This approach will leverage the skills, knowledge, and capacity of each team member to the evaluation’s advantage. Following the in-brief, the team will reconvene to workshop the completion of data collection tools and confirm methodological approach. The workshop will reinforce consistency across the sub-teams, help with workload distribution and promote sharing of technical knowledge and evaluation best practices for each evaluation question. The evaluation team expects to allocate one week for each landscape, thus ensuring adequate time to deeply develop each evaluation question by landscape. The team will complete their time in Jakarta by conducting interviews with key stakeholders before commencing travel to the landscapes. The team anticipates travel to Cyclops and Katingan-Kahayan Landscapes no later than Nov. 11. At the conclusion of fieldwork, the full team will reconvene to compile results and discuss findings. On the basis of the consensus view of the team, a PowerPoint-based briefing will be prepared for the Mission. The Integra team will return to Jakarta to present Deliverable 3, our exit briefing and highlights of the work accomplished in the field, including a synthesis of preliminary findings during the first week of December. After the Mission debriefing, the team will return to their respective homes. Finalization Immediately upon completion of the fieldwork, the team will transcribe, organize, and analyze all field data. The Team Lead and Senior Manager will coordinate the writing process, resulting in submission of a draft evaluation report (Deliverable 4) by the start of the second week of January, providing USAID/Indonesia with two weeks to provide comments. The Integra Senior Manager will coordinate a briefing with USAID/Washington, during the last week of January, on the evaluation findings and recommendations. The Team Lead and Senior Manager will synthesize and revise the report, under the expectation that a final report will be due to USAID/Indonesia by February 16. 62 LESTARI MID-TERM EVALUATION USAID.GOV Expected deliverable due dates November 2 Deliverable 1: Work plan and evaluation design approval November 6 Deliverable 2: In-brief with USAID/Indonesia December 5 Deliverable 3: Exit briefing and presentation with USAID/Indonesia January 8 Deliverable 4: Draft evaluation report submitted to USAID/Indonesia January 30 Deliverable 5: USAID/W briefing on evaluation findings and recommendations February 16 Deliverable 6: Final evaluation report submitted to USAID/Indonesia 63 LESTARI MID-TERM EVALUATION USAID.GOV Months October November December January February Calendar Week 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 Contract Week 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 - - Startup Contract Award Team Planning Meeting Kick off Conference Call with COR Conduct briefing meetings with key informants in Washington, DC Review Agency, Mission and LESTARI documents D1: Work plan and evaluation design approval * Fieldwork Team travels to Indonesia D2: In-brief with USAID/Indonesia * Conduct evaluation launch workshop with full LESTARI MTE team Finalize evaluation data collection tools Conduct KIIs with Jakarta based key informants Evaluation team travels to landscapes Cyclops Katingan-Kahayan Leuser Lorentz Mappi-Bouven Digoel D3: Exit briefing and presentation with USAID/Indonesia * Bi-weekly updates with COR on progress (update work plan at end of first week in field, if necessary) Finalizing D4: Draft evaluation report submitted to USAID/Indonesia * USAID Indonesia review of draft evaluation report Feedback synthesis to draft document Socialization and dissemination of recommendations D5: USAID/Washington briefing on evaluation findings and recommendations * D6: Final evaluation report submitted to USAID/Indonesia * 64 LESTARI MID-TERM EVALUATION USAID.GOV Evaluation Methodology This evaluation will focus on LESTARI performance with respect to expected results and objectives by technical component; program design and management; the prospect of long-term sustainability; and practical recommendations for performance improvement and strategic planning. The specific objectives of the mid-term evaluation are: • To assess the progress toward meeting LESTARI objectives; • To assess the validity of program strategies, approaches and assumptions; • To assess program performance management by LESTARI implementing partners; and • To identify lessons learned and recommend actions for improving performance. Under each technical component, a series of questions identify aspects of the program’s performance are considered. The LESTARI Midterm Evaluation scope of work detailed the following list of evaluation questions: TECHNICAL COMPONENT 1: Forest and Land Use Governance and Advocacy 1. Are LESTARI outreach activities, including social media campaigns, story pitching, interactive dialogues, and journalist trainings, sufficient to build constituencies for conservation and influence the forest and land use policies? (Awareness and Advocacy – LESTARI 1). 2. Are LESTARI awareness and advocacy activities effective in identifying target audiences, desired behaviors, and the appropriate media? How has LESTARI’s target audience retained, adopted, and implemented the desired knowledge they have gained through LESTARI? (Awareness and Advocacy – LESTARI 1). 3. Is LESTARI support on operationalization of SEAs and LCPs preventing conversion of critical areas into the other purpose area (APL)? (Operationalize SEAs/LCP to improve Land Use Governance – LESTARI 3). 4. Are LESTARI SEA/LCP recommendations being incorporated into other governance documents or processes such as spatial plan revisions, Provincial/District development plans for forest management and land use, draft regulations, or District/Provincial Development Plans (RPJM-D and RPKD)? (Operationalize SEAs/LCP to improve Land Use Governance – LESTARI 3). 5. Are LESTARI investments to strengthen citizen-based Multi-Stakeholder Forums (MSF), sufficient to influence land use decision-making, monitoring and planning? (Sustainable Landscape and Forest Governance – LESTARI 2) 6. Are LESTARI activities to increase citizen-based influence on forest management and land use decision, effectively channeled to BIJAK to influence national level policymakers? (Sustainable Landscape and Forest Governance – LESTARI 2). 7. Are LESTARI activities to developing systematic and transparent monitoring of land use, sufficient to encourage provincial/district governments to monitor land use and licensing? (Sustainable Landscape and Forest Governance – LESTARI 2). 8. Are LESTARI activities to improve forest management through co-management and FMU capacity building sufficient to increase forest stewardship, ultimately contributing to 8.4 Million of biological significance and/or natural resources under improved natural resource management? (Improve Forest Management Through Co-Management - LESTARI 4). 65 LESTARI MID-TERM EVALUATION USAID.GOV TECHNICAL COMPONENT 2: Improved Management of Conservation Areas 1. Are the LESTARI activities to improve METT scores sufficient to improve management of Conservation Areas (CAs)? 2. Are the LESTARI activities under LESTARI 1 - Awareness and Advocacy and LESTARI 2 - Sustainable landscape governance, sufficient to establish constituencies for conservation that support CA management? 3. Is LESTARI support for SMART patrol sufficient to combat wildlife trafficking and reduce poaching in the landscapes? TECHNICAL COMPONENT 3: Improved Private Sector Practices 1. Are LESTARI activities to promoting public private partnerships (PPP) for green enterprise in the landscape sufficient to mobilize green investment and improve sustainable commodity production? Is this leading to a reduction in deforestation or forest degradation? 2. Is LESTARI support on innovative financing for sustainable land and forest management sufficient to secure long term financing mechanism for conservation and restoration in the landscapes? 3. How have LESTARI activities under this component changed government or private sector thinking about how forest conservation can positively impact local economic development? Management Effectiveness and Sustainability: 1. How is the implementation structure and management of LESTARI contributing to its successes and are there areas for improvement? 2. Is LESTARI engaging and empowering the right set of stakeholders—in terms of their impact on and influence over the resource —to reduce deforestation and threats to biodiversity in their landscapes. 3. Will the key institutions that LESTARI is strengthening continue the work after USAID funding is phased out? Program Design and Performance Questions 1. What evidence exists to support and validate the results framework and theory of change for LESTARI? 2. Do the strategic approaches align with the theory of change and address the direct and indirect drivers, key actors and stakeholders in all landscapes? To begin developing the midterm evaluation methodology, the evaluation team conducted a detailed review of each evaluation question in order to address each question adequately through the data collection process. This detailed review found the originally proposed evaluation question list totals 22 unique questions. This number is well in excess of USAID’s own Evaluation Policy, which suggests a total of no more than 12 questions be structured into an evaluation. Additionally, the evaluators raise concerns regarding the expected quality in 22 question evaluation. The review of the originally evaluation questions led to the following trends: • Questions were specifically geared towards an activity, rather than an objective or result • Certain questions made additional assumptions which restricted the lens through which evaluator would address the desired objective through • Between technical components, there is duplicity of the same questions While developing the evaluation question matrix, the evaluation team developed a Crosswalk Table (Table 2, page 10) which outlines each individual scope of work question, any concerns related to the phrasing of the question, and a proposed refinement of the question. Through this exercise, the 66 LESTARI MID-TERM EVALUATION USAID.GOV list of questions is refined to 15 total evaluation questions. The evaluation team feels strongly that this refinement of the questions adequately addresses USAID Indonesia’s desired information set, while framing the evaluation questions in line with best practices. The evaluation team proposes the following refinement questions for evaluation methodology framing and final reporting purposes: TECHNICAL COMPONENT 1: Forest and Land Use Governance and Advocacy 1. Is LESTARI effective in identifying and engaging target audiences in order to achieve the activity goal? 2. Is LESTARI building constituencies in support of biodiversity and conservation? 3. Is LESTARI preventing conversion of critical areas? 4. Are Multi-Stakeholder Forums influencing land use decision-making, monitoring and planning? 5. Is government uptake of LESTARI land use monitoring data effective for land registration? 6. Is LESTARI support being incorporated into governance documents at the district, province and national levels? TECHNICAL COMPONENT 2: Improved Management of Conservation Areas 7. Is LESTARI encouraging improved management of conservation areas? 8. Is LESTARI support of the co-management structures and FMUs capable of increasing forest stewardship? 9. Is SMART use by patrols effective in combatting wildlife trafficking and poaching in the landscapes? TECHNICAL COMPONENT 3: Improved Private Sector Practices 10. Is LESTARI promoting green investment which reduces deforestation/degradation? 11. Is LESTARI securing long term financing mechanisms for conservation and restoration? 12. Is LESTARI encouraging private sector participation to increase local economic development? Management Effectiveness and Sustainability: 13. How is the implementation structure and management of LESTARI contributing to its successes and are there areas for improvement? 14. What mechanisms are in place to insure LESTARI sustainability? Program Design and Performance Questions 15. What evidence exists to support and validate the results framework and theory of change for LESTARI? To ensure that the LESTARI MTE report captures USAID Indonesia’s original evaluation questions’ intentions, the Evaluation Question (EQ) Matrix (Table 3, page 15) includes specifications around which activities, where applicable, will be included in the analysis of each refined question. This process, beginning with the Crosswalk Table and ending at the EQ matrix demonstrates the entire thought process in refining the evaluation scope consistent with a non-experimental, mixed methods evaluation approach. The EQ matrix links each evaluation question with data sources, including required documents for review, targeted key informants and stakeholder groups for interviews and focus groups. 67 LESTARI MID-TERM EVALUATION USAID.GOV Methodological Approach The evaluation will use a non-experimental, observational design appropriate for a mid-term performance evaluation, involving a mixed-methods approach, combining qualitative and quantitative research methods and analysis. This design approach allows USAID and its stakeholders to understand the effectiveness of LESTARI programming and build a knowledge base to inform decision making, enabling USAID to test the key hypotheses behind the overarching results framework’s theory of change. The evaluation will be conducted in three phases: preparation, field work, and analysis. Phase 1: Preparation The evaluation team will conduct a comprehensive desk review of relevant documents and data prior to arrival in Jakarta. To begin informing the interview protocols and data collection tools, the Team Lead and Senior Project Manager will hold preliminary discussions with USAID Washington, US Forest Service, and Tetra Tech backstops in Washington, D.C. Rough drafts of the interview protocols will be “workshopped” with the entire evaluation team upon arrival in Jakarta, ensuring each team member’s technical lens has contributed to the questionnaires and that each team member understands the intention of each question. The workshopping exercise will also provide an opportunity to ensure that gender, indigenous community and socioeconomically disadvantaged group measurements are integrated into the field data collection effort. Phase 2: Field Data Collection Beginning the first week of November, the LESTARI MTE team will convene in Jakarta to officially begin the field data collection process, which is expected to last 4 weeks (see detailed calendar in Annex A). The evaluation team will meet with USAID Indonesia, LESTARI management and implementing partner teams, national government representatives and parks authorities, civil society representatives working in the area of biodiversity and conservation, community members in the targeted landscapes, other donor representatives and academic institutions, where applicable, to gather additional relevant information that will be used for triangulation purposes to obtain confirmation of findings and results. Given the geographic breath of LESTARI activities, the evaluation team will split into two groups, ensuring adequate time at each landscape to deeply develop each evaluation question. Each team was carefully considered to address the threats and unique constraints facing each landscape. The following Table 4 illustrates the break out of team members by landscape, expected localities, team technical skills coverage and expected travel dates15: 15 The proposed team breakout is conditional on Ms. Makuch receiving a Papua permit. In the event that the team is unable to acquire the appropriate permits, a revision to this breakout will be needed. 68 LESTARI MID-TERM EVALUATION USAID.GOV To insure the highest level of data quality, each evaluation team member will be provided with a digital recorder for all interviews and focus groups. Each landscape team will meet at the end of the day to begin data entry. Data entry will be done by all evaluation team members, which will allow the evaluation team to be aware of any data collection problems and correct them in real time. Field reports will be provided at each landscape from all team members to the evaluation team lead weekly, ensuring consistent data quality is met. Data collection tools The evaluation team will develop a robust set of tools to ensure that all relevant stakeholders contribute to the evaluation effort. Data collection involves multiple activities and methods, but will largely focus on qualitative information gathered through semi-structured interviews of key informants from government, private sector companies operating concessions or managing forest land in each landscape, participants in green enterprise development activities, landscape managers and multi-stakeholder forum members. At the community level, group discussions will be conducted with communities on LESTARI activities on awareness raising, engagement in land use planning, small enterprise development and other aspects of the program. In every site visit, the evaluation team will ensure that indigenous groups, women and other disadvantaged minorities are interviewed to gather independent feedback for gender and social inclusion. The evaluation will produce qualitative information and quantitative data from the following proposed sources: • Interviews with Key Informants at various levels of interlocutors (to include local government representatives) • Document Review including LESTARI and GoI reports and assessments, academic literature, international environmental NGOs and expert sources in Indonesia • Focus Group Discussions with direct project beneficiary members and community members, providing an opportunity to assess constituencies, local economic development and women’s engagement in LESTARI • Institutional Analysis (supported by information gleaned through a variety of data sources including direct observation and secondary data) The table below details the technical areas with their expected data collection protocols. It is worth noting that the range of actors engaged through each technical area requires that all data collection protocols feature in some form. Data collection tools, by evaluation area Evaluation Areas Document Review Key Informant Interviews Focus Group Discussions Direct Observation Forest and Land Use Governance and Advocacy ♦ ♦ ♦ ♦ Improved Management of Conservation Areas ♦ ♦ ♦ ♦ Improved Private Sector Practices ♦ ♦ ♦ ♦ Management Effectiveness and Sustainability ♦ ♦ ♦ 69 LESTARI MID-TERM EVALUATION USAID.GOV All data provided by the various tools will be triangulated with contextual and institutional information, with the quantitative data supporting the qualitative contextual piece under which LESTARI is implemented. Phase 3: Data Analysis & Reporting Analysis will look to capture quantitative changes in the objective outcomes across all three technical evaluation components, the qualitative changes in perceptions around conservation, land use and economic opportunities, and the processes that have shaped the implementation of the activity. Data will be analyzed and synthesized through a structured integration of quantitative indicators/data with qualitative data and direct observations supported by data from LESTARI’s baseline data, monitoring plan, secondary documentation and expert inputs. For evaluation questions that address the sufficiency of activities or the effectiveness of various interventions, quantitative targets established in LESTARI annual work plans and monitoring plans will be assessed by the team’s technical specialists and external expert opinion to determine the progress in achieving desired outcomes, and the sufficiency of activities. To the extent possible, the evaluation team will strive to use quantitative data and clearly show the methodology for interpretation of opinions and perceptions from interviews and group discussions. 70 LESTARI MID-TERM EVALUATION USAID.GOV Crosswalk Table LESTARI SOW Evaluation Question Clarified Question Technical Evaluation Discussion Proposed Refinement TECHNICAL COMPONENT 1: Forest and Land Use Governance & Advocacy 1. Are LESTARI outreach activities, including social media campaigns, story pitching, interactive dialogues, and journalist trainings, sufficient to build constituencies for conservation and influence the forest and land use policies? (Awareness and Advocacy – LESTARI 1). 1. Are LESTARI outreach activities sufficient in building constituencies for forest and land use policies? TC1.1 & TC1.2 are in fact three separate questions which look to measure LESTARI’s performance around constituency outcomes. The logical breakout of the “constituency” questions, which are appropriate for a midterm evaluation, is to look at 1) has LESTARI identified the correct audience to influence any desired outcomes and 2) has LESTARI supported these constituencies to influence forest and land use policies and conservation (link to question TC2.2). The last question in this sequence hints at impact, which is outside the scope of a performance evaluation. The evaluation team proposes refining the question to focus on the questions which are appropriate to a performance evaluation. 1. Is LESTARI effective in identifying and engaging target audiences in order to achieve the activity goal? 2. Are LESTARI awareness and advocacy activities effective in identifying target audiences, desired behaviors, and the appropriate media? How has LESTARI’s target audience retained, adopted, and implemented the desired knowledge they have gained through LESTARI? (Awareness and Advocacy – LESTARI 1). 2. Are LESTARI activities effective in identifying target audiences? 3. How has LESTARI’s target audience retained, adopted, and implemented the desired knowledge they have gained through LESTARI? 2. Is LESTARI building constituencies in support of biodiversity and conservation? 3. Is LESTARI support on operationalization of SEAs and LCPs preventing conversion of critical areas into the other purpose area (APL)? (Operationalize SEAs/LCP to improve Land Use Governance – LESTARI 3). 4. Is LESTARI preventing conversion of critical areas? The clarified question removes the focus on the activity in the original question and refocuses the question on the outcome, which is prevention of conversion. 3. Is LESTARI preventing conversion of critical areas? 4. Are LESTARI SEA/LCP 5. Are LESTARI Questions TC1.4 and TC1.5 are sequentially out of 6. Is LESTARI support being 71 LESTARI MID-TERM EVALUATION USAID.GOV recommendations being incorporated into other governance documents or processes such as spatial plan revisions, Provincial/District development plans for forest management and land use, draft regulations, or District/Provincial Development Plans (RPJM-D and RPKD)? (Operationalize SEAs/LCP to improve Land Use Governance – LESTARI 3). recommendations being incorporated into national and provincial governance documents? order given the LESTARI theory of change. MSF influence in land use activities should feed into governance documents. The evaluation team proposes switching the order here to generate a narrative consistent with the LESTAR theory of change. incorporated into governance documents at the district, province and national levels? 5. Are LESTARI investments to strengthen citizen-based Multi-Stakeholder Forums (MSF), sufficient to influence land use decision-making, monitoring and planning? (Sustainable Landscape and Forest Governance – LESTARI 2) 6. Are Multi-Stakeholder Forums influencing land use decision-making, monitoring and planning? The clarified question removes the focus on the activity in the original question and refocuses the question on the outcome, which is changes to land use. 4. Are Multi-Stakeholder Forums influencing land use decision-making, monitoring and planning? 6. Are LESTARI activities to increase citizen-based influence on forest management and land use decision, effectively channeled to BIJAK to influence national level policymakers? (Sustainable Landscape and Forest Governance – LESTARI 2). 7. Are Multi-Stakeholder Forums effectively channeled to BIJAK to influence national level policymakers? This question is a subset question related to USAID question TC1.4. The objective of LESTARI activities, such as SEAs/LCP, MSF, and others, is in part intended to influence national policymakers and their drafting of national governance documents. We propose dropping this question as it is already covered in the refinement of question TC1.4 and TC1.5. 7. Are LESTARI activities to developing systematic and transparent monitoring of land use, sufficient to encourage provincial/district governments to monitor land use and licensing? (Sustainable Landscape and Forest Governance – LESTARI 2). 8. Is LESTARI land use monitoring encouraging evidence-based decision making by provincial/ district governments? 9. Is LESTARI developing systematic and transparent The primary question is looking at LESTARI outcomes in influencing government (be it district/provincial/national, all of which are simple disaggregates.) This question is thematically covered in questions TC1.4, TC1.6, and Management 2. 5. Is government uptake of LESTARI land use monitoring data effective for land registration? 72 LESTARI MID-TERM EVALUATION USAID.GOV monitoring of land use? 10. Is this being used by any stakeholders of influence? The sub-questions are not covered elsewhere in the evaluation questions are justified as a unique question. The team propose dropping the broader question (#8 in the second column) and synthesizing question #9&10 into one cohesive question. 8. Are LESTARI activities to improve forest management through co￾management and FMU capacity building sufficient to increase forest stewardship, ultimately contributing to 8.4 Million of biological significance and/or natural resources under improved natural resource management? (Improve Forest Management Through Co-Management - LESTARI 4). 11. Are co-management and FMU capacity building activities sufficient to increase forest stewardship? This question looks to evaluate activities around forest stewardship, which fundamentally is not an advocacy function but an improved management function. The evaluation team proposed this question be moved to TC 2. TECHNICAL COMPONENT 2: Improved Management of Conservation Areas 1. Are the LESTARI activities to improve METT scores sufficient to improve management of Conservation Areas (CAs)? 12. Are LESTARI activities encouraging improved management of conservation areas? METT scores are a function of an activity, and we would assume that the scores are not the highest￾level outcome of interest to USAID. The question has been clarified to focus on the desired outcome of improved management, which will be evaluated against all LESTARI activities, including but not limited to METT scores. 7. Is LESTARI encouraging improved management of conservation areas? Relocate Question TC1.8 to TC2 with refinement 8. Is LESTARI support of the co-management structures and FMUs capable of increasing forest stewardship? 73 LESTARI MID-TERM EVALUATION USAID.GOV 2. Are the LESTARI activities under LESTARI 1 - Awareness and Advocacy and LESTARI 2 - Sustainable landscape governance, sufficient to establish constituencies for conservation that support CA management? 13. Are LESTARI activities sufficient in building constituencies for conservation? This question is duplicative to question TC 1.1 as this is the same implementation vehicle with a different disaggregate in the outcome. 3. Is LESTARI support for SMART patrol sufficient to combat wildlife trafficking and reduce poaching in the landscapes? 14. Are SMART patrol combatting wildlife trafficking and poaching in the landscapes? The word “sufficient” is problematic as it is completely subjective. The clarification question realigns to expected outcome of SMART patrols, namely combatting wildlife trafficking. 9. Is SMART use by patrols effective in combatting wildlife trafficking and poaching in the landscapes? TECHNICAL COMPONENT 3: Improved Private Sector Practices 1. Are LESTARI activities to promoting public private partnerships (PPP) for green enterprise in the landscape sufficient to mobilize green investment and improve sustainable commodity production? Is this leading to a reduction in deforestation or forest degradation? 15. Are LESTARI activities encouraging green investment in sustainable commodity production? 16. Is green investment influencing a reduction in deforestation or forest degradation? Question TC3.1 is a compound question which we have broken out into its two separate thematic questions. The refinement question assumes that LESTARI’s green investment activities have the intended outcome of reducing deforestation/ degradation. The analysis component will address both LESTARI’s activities around green investment and their outcomes. 10. Is LESTARI promoting green investment which reduces deforestation/degradation? 2. Is LESTARI support on innovative financing for sustainable land and forest management sufficient to secure long term financing mechanism for conservation and restoration in the landscapes? 17. Are LESTARI activities sufficient to secure long term financing mechanisms for conservation and restoration? Again, the word “sufficient” is problematic; the clarified question realigns the focus to the relationship between long term financing and conservation. 11. Is LESTARI securing long term financing mechanisms for conservation and restoration? 74 LESTARI MID-TERM EVALUATION USAID.GOV 3. How have LESTARI activities under this component changed government or private sector thinking about how forest conservation can positively impact local economic development? 18. Have LESTARI activities influenced the private sector to increase local economic development? It is impossible for an evaluation to measure the “thinking” of government or private sector actors. A viable proxy for this type of measurement is the expected outcome, which is increased local economic development in forest conservation areas. The evaluation team proposes reworking the question to reflect the measurement realities of such a question. 12. Is LESTARI encouraging private sector participation to increase local economic development? Management Effectiveness and Sustainability 1. How is the implementation structure and management of LESTARI contributing to its successes and are there areas for improvement? 19. How is the implementation structure and management of LESTARI contributing to its successes and are there areas for improvement? 13. How is the implementation structure and management of LESTARI contributing to its successes and are there areas for improvement? 2. Is LESTARI engaging and empowering the right set of stakeholders—in terms of their impact on and influence over the resource —to reduce deforestation and threats to biodiversity in their landscapes This question is duplicative and is answered in questions TC1.1, TC1.2, TC2.2. The evaluation team proposes dropping this question. 3. Will the key institutions that LESTARI is strengthening continue the work after USAID funding is phased out? 20. What mechanisms are in place to insure LESTARI sustainability? It is outside the scope of a performance evaluation to evaluate the impact and behavior change of indirect LESTARI partners, and there is virtually no means by which an evaluation could determine the likelihood of a non-project actor working on project specific outcomes. 14. What mechanisms are in place to insure LESTARI sustainability? 75 LESTARI MID-TERM EVALUATION USAID.GOV At the midterm, it is premature to ask what evidence there is that LESTARI work has been “mainstreamed” and to measure if this is sufficient for sustainability. The evaluation team proposes widening this question to look at what mechanisms could potentially act as vehicles of sustainability in a post-LESTARI world. Program Design and Performance Questions 1. What evidence exists to support and validate the results framework and theory of change for LESTARI? 21. What evidence exists to support and validate the results framework and theory of change for LESTARI? The true scope of this question is more appropriate for a final or impact evaluation; however, under a restricted scope, the evaluation team feels evidence should be available to validate the theory of change. 15. What evidence exists to support and validate the results framework and theory of change for LESTARI? 2. Do the strategic approaches align with the theory of change and address the direct and indirect drivers, key actors and stakeholders in all landscapes? This question is the culmination of question TC1.2, TC1.3, TC1.5, TC1.7 and Program Design and Performance Question #1; this question appears redundant. The evaluation team proposes dropping this question. 76 LESTARI MID-TERM EVALUATION USAID.GOV ANNEX III: LESTARI MTE SUMMARY OF FINDINS BY LANDSCAPE # EVALUATION QUESTIONS LEUSER LANDSCAPE DISCUSSION NOTES 1 To what extent are LESTARI outreach activities building constituencies in support of biodiversity and forest conservation? • “water” is the entry point for “biodiversity and forest conservation” outreach activities, which is good since this gives a holistic approach to existing and proposed conservation actions • There was journalist training for 40 local journalists in 5 districts in Aceh. These 40 local has the capacity in reporting in the sense of “environmentalist”. However, they didn’t put the mass media as the partner anymore due to the editorial issues. Articles should be filtered by editorial that is most likely influenced by editorial’s interest • In addition, youth and women groups are involved in awareness raising through media social campaign. The sense of “Leuser and Environment” are introduced to these groups to influence the wider constituencies. There was 21 women to be trained in social media journalism. The participants come from 5 districts of Leuser Landscape areas. • Leuser’s outreach, through RPJMDes planning, has also increased understanding of greater environmental issues to disasters which they face. For example, communities can now see the connection between flooding and environment conservation issues. • The scheme of social forestry increased the community’s understanding on the conservation activities in conservation areas. For example in Pantan Cuaca, the community is aware that cultivating coffee well under social forestry scheme gives more benefits for ecology, Private Sector Engagement: ● To not to lose momentum: support PPP with less stringent pre-certification label to support the participating farmers already while continuously improving towards complying with the certification requirements ● Consider the opportunity cost for alternative livelihoods when mining and/or growing palm oil. ● For corporates: develop basic tool-kit showing cost-benefit of more sustainable business practices in both the mining industry and the palm oil industry. ● Set up data-base of possible service providers for referral → for palm oil, work through GAPKI and selected engaged companies 77 LESTARI MID-TERM EVALUATION USAID.GOV economy, and social. While in Pining, community aware that implementing organic Cacao gives other benefits such as health and ecology, instead of higher prices. Private Sector Engagement: • Agricultural communities follow premium price arrangement for organic non-forest commodity production (nutmeg, cocoa, coffee); Opportunity: the group approach of paying premium price for cacao produced on certified organic farmland in Pepelah (Pining) provides an effective tool for peer-enforcement; Risk: preparing the farmland for certification requires time (e.g., min. 13 commodities needed for multicropping, no chemicals for 5-7 years) and meanwhile the farmers need to earn money. Currently they do that selling fermented produce and other vegetable like chili. • The impact by sustainable livelihood activities on biodiversity and forest conservation are currently not measured. Essential oil production from nutmeg/lemongrass/ patchouli requires a high amount of firewood (pine) in/around both Gayo Lues and Tapaktuan • Some communities mine and grow palm oil. Their activities as well as activities by mining and palm oil concessionaires are acknowledged, yet unsupported because “We [at LESTARI] do not have any tools for sustainable mining or palm oil. 2 To what extent has LESTARI been able to prevent conversion of critical areas to other purpose areas? • KLHS (SEA) for reviewing the RPJMA in Kalimantan Tengah gives lots understanding in sustainable developments. This RPJMA will be the reference for districts in implementing the same process of reviewing RPJMA. SEA is a process that should be done local government. KLHS is in line with the vision and mission of Private Sector Engagement: ● To keep the momentum and trust by the community, support PPP with less stringent pre-certification label to support the participating farmers already while continuously improving towards complying with the certification requirements (see above Q1) 78 LESTARI MID -TERM EVALUATION USAID.GOV RPJMA, Aceh Green. • Social Forestry scheme between KPH 5 and 6 (FMU) and community group in some village Aceh reduce the opening land in protected area. LESTARI speeds up the process of permit in MoEF/national level. Now, the social forestry permits for Pantan Cuaca District is in the MoEF. The community is waiting to do some following actions for coffee plantation in the conservation areas. • Bappeda Spatial Planner in Aceh Barat Daya has noted that timber has become scarce and attributes this to greater protection from several agreements reached through Lestari interventions. Private Sector Engagement • LESTARI activities with agricultural communities have increased their knowledge on non -chemical crop planting and with this, prevented some of the farmers from converting their land into palm oil plantation 3 To what extent are LESTARI policy recommendations incorporated into governance documents at the district, province, and national levels? • LESTARI has facilitated in harmonizing national forest resource use laws (UU or Law No.23/2014 and Law No. 11/2006 with Aceh Qanum No.7) with stakeholders (concerned government offices) for the suggested revisions to be incorporated in Qanum no.7 under the provincial government, which is now set to integrate these recommendations. At the field level this is necessary, as we have interviewed farmers utterly confused with contradictory laws governing their resource use. • LESTARI has also assisted in shaping provincial, district and village levels development plans where conservation measures from LESTARI were adapted into the plans, and successfully 79 LESTARI MID-TERM EVALUATION USAID.GOV acquiring necessary budgetary allocations for the workplan. • The Trumon Corridor (linking two protected areas) and composed of productive and APL forests is being declared as a TAHURA (Taman Hutan Raya) a public forest park--with much support from the Bupati, and district level officials. The declaration has fingerprint support of LESTARI and is being touted by the BUPATI as a pet project, now pending its legislative declaration. - For example, Bappeda Aceh Barat Daya has adopted Lestari mechanisms for spatial planning for urban planning as well. 4 To what extent are Multi￾Stakeholder Forums influencing land use decision-making, monitoring and planning? • The MSF-FMUL representatives come from former IFAC implementers or current grantee applicants and other partners. The group does not seem to function as a united group influencing overall conservation management decisions for the whole landscape area, but instead are individually concerned with their own project proposals for implementation and therefore funding. One option is to re-orient the nature of MSFs as an advisory board that can assist LESTARI in conservation decisions, putting this body on top of project implementers (an independent overseer, with no ties to a project ending, more sustainable). 5 To what extent have LESTARI activities led to provincial and district governments to effectively monitor land use and licensing? • Bappedas Blankejeran has reported that land clearances within the district has gone down based on their internal monitoring system (need to check answer as to what this is). WCS has indicated that before L, data for land use monitoring are all varied, and LESTARI has worked towards unifying land use monitoring. • SST at the provincial level is being implemented, but key informant was not available during MTE’s field work. 80 LESTARI MID -TERM EVALUATION USAID.GOV 6 To what extent is LESTARI effectively encouraging management of conservation areas (e.g. co -management, FMUs, METT, etc.)? • KPH5 and KPH6 are forest lands being managed and regulated in various stages of proposal, agreement, and implementation. Several co - management agreements are reached (6), and we sensed communities’ enthusiasm and support for social forestry, village forestry, with the aim of balancing conservation and resource access. L’s work in this is very appreciated across villages and government agencies (please see our data set list description). We also witnessed the close collaborative work of government agencies, communities, NGOs or implementers as facilitated by Lestari to reach co -management agreements. There is an understanding for some villagers that from the agreement, it will be their first time to do social forestry, and they are willing to learn from the experience. METT Scores • National Parks representatives found it useful to have a baseline data on their efficiency rating, which they acknowledge as a service from the LESTARI project. They now have an aim to increase their scores annually. • TNGL head: “It is a good measurement for improving management skills in conservation, but what is really important and cannot be measured is what is in the heart.” 7 To what extent are SMART patrols preventing wildlife trafficking and poaching? • SMART patrols are done in a systematic order in all conservation areas of Gayo Lues, Sinkgil Wildlife Reserve, and also the Trumon National Park (non -SMART patrols). Data is compiled with the Taman National Offices and recorded to see illegal activity sightings. We did not have enough time to see trends being established as to determine SMART patrols are responsible for any decrease in wildlife trafficking. Also 81 LESTARI MID-TERM EVALUATION USAID.GOV establishing direct correlation between 1.5 years of regular SMART patrol and reduction of wildlife trafficking and poaching, is probably premature. However, SMART Patrols is expressed as a positive service added to the Taman National Parks system. 8 To what extent are LESTARI public-private partnerships successful in increasing environmentally sustainable commodity production? • Focus of PPP engagement so far has been on agricultural farming and agroforestry; the business models include no chemicals with the aim to obtain certification of organic farmland (for which the farmers have been learning how to grow coffee, cacao, chili etc. without chemicals and via multi-cropping) • “...environmentally sustainable commodity production” - should be changed to “...environmentally sustainable livelihoods” to allow for alternative revenue creation to be supported by LESTARI which includes adding value along the value chain and services sectors (e.g., small-scale renewable energy provision, packaging, marketing, information center, etc.). 9 To what extent are these partnerships reducing deforestation and forest degradation? • In principle, e.g. the group-based organic cacao certification includes peer-enforcement on those farmers who do any farming activities in the forest (including non-supported crops). • It is unclear, however, how many “kebun jauh” exist, and if additional livelihood activities which are lucrative (e.g. marihuana plantations) can be totally avoided with the help of LESTARI. • For processing raw material into essential oil, many pine trees are needed, which actually increases deforestation. So far, the project does not measure the unintended carbon emissions from sustainable livelihood, its processing energy or transport. • Consider the opportunity cost for alternative livelihoods when mining, growing palm oil or marihuana. 10 To what extent has LESTARI increased access and use of long term financing for conservation and restoration? • Via match-making grants LESTARI has leveraged project budget from six grantee organizations (Javlec, Inprosular, Nutmeg forum, Atjeh International Development, Aceh Green Community), e.g., the founder of the nutmeg forum has created a company to distribute nutmeg-based products (essential oil, dried fruit) 82 LESTARI MID -TERM EVALUATION USAID.GOV • Other donors: grantees (e.g. OIC) leveraged having USAID Lestari funds to generate more attention and other donors successfully. Some can survive without Lestari support, but Lestari added -value is well -articulated especially in policy, increased capacity, etc. • Public institutional financing: Government budget allocation in annual work plans increased for conservation activities through program and activities in developed RPJMDes for 6 years in village level • In district and provincial level, KLHS (SEA) that is assisted to review RPJMD, results in the conservation and restoration program for 6 years. The planned program and activities accesses national or local budget. • PES study completed; No PES yet in implementation • Corporate finance: MoUs with buyer companies to support more sustainable production of agricultural raw materials (organic farmland) • If “long -term financing” originally meant attracting institutional investors, e.g. via the issuance of municipal bonds and/or corporate bonds, this has not been implemented 11 How is the implementation structure and management of LESTARI contributing to its successes and are there areas for improvement? • We find LESTARI management and staff committed and dedicated to conservation, partners and communities. Their passion can be seen through. There were areas for improvement, but we lack time to investigate: what is evident is that LESTARI management in this landscape is proactive in going for efficient human resource. • Not many villages selected under the different grants/sub -contracts. actually overlap, hence the 83 LESTARI MID-TERM EVALUATION USAID.GOV activities (village planning, livelihood, private sector, etc.) are less integrated than in KK. 12 To what extent are LESTARI activities undertaken with a focus on sustainability of results after the project is closed? • Economic sustainability: Most LESTARI￾supported agricultural communities are not yet sufficiently equipped to cost, develop and innovate their business models by themselves and would be dependent on further assistance by a service provider or the buyer company. • Environmental sustainability: the LESTARI sustainable livelihood activities do not measure the (unintended) impact on the environment yet. • We did not notice exit strategies with partners being discussed, perhaps because the mid-year still feel like planning stages for some of the interventions • Lestari creates local champion from community level to ensure that their program will still be implemented when the project ends. Training on Trainer (ToT) for cacao and coffee farmers identify the potential local champions. • Understandably, landscape approach conservation goals need to be undertaken for far longer periods of time than 4 years. During assessment at mid-term, we find that while there were many significant achievements, there were also a feeling of just finishing the planning stages and therefore implementation might not be reached during the project duration • Landscape approach work needs to be beyond 4 years. LESTARI got off to a very good start, but the ballooning needs of huge landscapes being protected means the momentum might be hard to contain within the limited project duration without having created disappointments among many partners. 84 LESTARI MID-TERM EVALUATION USAID.GOV 13 To what extent is the theory of change still valid? • Most of the elements of theory of change remain valid. What would make it more valid as expressed by respondents would be greater harmonization of laws and greater coordination among regulation agencies to achieve Lestari’s aim. Poverty reduction is the overwhelming factor echoed by respondents (government agencies, NGO partners, conservation agents as necessary to conserve the forest and biodiversity. 14 To what extent are LESTARI activities aligned with the Theory of Change? Co-management, Advocacy for increased constituency, and private sector engagement are being targeted equally by LESTARI activities. We therefore find them aligned. Private Sector Engagement: • Implementation: main focus on agricultural farming and agroforestry; pipeline: eco-tourism and non-timber forest produce (NTFP) • LESTARI incorporates sustainable livelihoods approach as a tool for conservation • Match-making buyer-supplier relationships accelerates the building of supply chains Private Sector Engagement (same as in KK): • Couple the existing demand orientation with a critical assessment of local economic development opportunities and corresponding technical advice (incl. early TA on the economic feasibility of supported commodities (long-term trends), de-risking via diversification, consideration of opportunity cost of giving up illegal practices, and measurement of related carbon emission (incl. from energy and transport). Buyers - especially larger brands - are increasingly asked to disclose on environmental and socio-economic impact created/SDG reporting along the value chain, to which this could contribute. 15 To what extent is there buy in for LESTARI interventions across the GOI (national, provincial, local, community)? At the national level: national parks are invested in LESTARI support and have absorbed LESTARI interventions (SMART Patrols); more evident at the district and village levels where the mechanism installed by LESTARI of participatory and inter￾village consultations to produce RPJMD are adopted as a “tool” or “mechanism” to achieve planning, and acquire Dana Desa (villa funds). These LESTARI instruments have a buy-in. Respondents indicate that they will continue on with these instruments and capacities even beyond LESTARI project timeline. For example, Bappeda Aceh Barat Daya has adopted LESTARI mechanisms for spatial 85 LESTARI MID-TERM EVALUATION USAID.GOV planning for urban planning as well. The buy in for spatial planning strategy was very positive. 16 To what extent is USAID management of LESTARI contributing to its successes and are there areas for improvement? • Intrusiveness (micro-management) • Imposing changing strategies midway arbitrarily (from landscape approach to leaning heavily on forest protection disregarding provision of economic options) • Absence of connections between forest and wildlife conservation with renewable energy options in project design. 86 LESTARI MID-TERM EVALUATION USAID.GOV # EVALUATION QUESTIONS KATINGAN LANDSCAPE DISCUSSION Recommendations 1 To what extent are LESTARI outreach activities building constituencies in support of biodiversity and forest conservation? • “fire prevention” is the entry point for “biodiversity and forest conservation” activities and in building awareness; for example, for orangutan conservation, a video of fire and its effects on orangutan sent a powerful msg. • Since fire is very visible and has become politicized nationally and internationally (2015 haze prompted ASEAN and EU to discuss the issue) - this was an opportunity that was put to good use by LESTARI in building constituencies for policy support, and wider support for habitat/peatland restoration • Education and awareness through articles and advertisement in mass media, radio, as well as social media spread the awareness raising for conservation activities. It has not yet surveyed whether the information is well informed and received by the audiences, however the team has already conducting well managed communication and education to public. They had regular talk show about FPIC (PADIATAPA) and fire prevention in Radio. Now, they have public service announcements on fire prevention in two radio stations: Kalaweit and RRI. Regular articles also publish in Kalteng Pos, a local mass media. PADIATAPA and improving livelihood through rubber is themes for articles. PADIATAPA as the process to prevent fire by channel blocking. it is good practices for local community to map their potential channel blocking as well as livelihood improvement through rubber plantation. • KK’s outreach, through RPJMDes planning, increased understanding of environmental issues to disasters. In Buntoi and Garung Villages, they understand well that fire Private sector engagement: • apply the conservation tool(s) for farmland - e.g., SOP on “fire prevention on farmland” - to other commodities; engage the rubber tapping community to transfer their fire prevention knowledge/experience to other farmers of the same village. 87 LESTARI MID-TERM EVALUATION USAID.GOV prevention can be in line with environmental conservation and economic improvement through Rubber and Sengon. In other area, village forest is the activities to be addressed in RPJMDes. Private sector engagement: • Rubber communities in Pulang Pisau (KUBK) support via an SOP on fire prevention on their farmland (dams, maintenance, rewetting); new business model helps to be in control and have a better price, however market price is currently low and farmers shift to other commodities (sengon) • The interviewed timber concessionaire(Dwima) was happy with the RIL-C training (reduced impact logging) because it helped achieving economic savings → good prospects for ownership and continuation “We are one group - we will all apply the same standard.” 2 To what extent has LESTARI been able to prevent conversion of critical areas to other purpose areas? • There are many examples of avoided conversion of forests into palm oil plantation: For example: Accelerating village forest permit in Tangkehan Villages is the result of LESTARI in prevent the conversion to palm oil which the villagers vehemently oppose. • KLHS (SEA) for reviewing the RPJMD in Kalimantan Tengah gives lots understanding in sustainable developments. This RPJMD will be the reference for districts in implementing the same process of reviewing RPJMD. SEA is a process that being done with local government and therefore positive. • Social Forestry scheme between KPH (FMU) and community group in some village reduce the opening land in protected area. LESTARI speeds up the process of permit in MoEF/national level. • LESTARI should assist and ensure the forest village planning and implementation development in Tangkahen through well managed ecotourism plan • LESTARI should urgently assist the two national parks with a conservation-focused and restauration focused eco-tourism concept and implementation, prior to other actors entering the area with a non￾conservation approach (waste, pollution, visitor numbers, etc.). 88 LESTARI MID-TERM EVALUATION USAID.GOV 3 To what extent are LESTARI policy recommendations incorporated into governance documents at the district, province, and national levels? • Harmonizing various fire regulations has been a huge policy contribution of THE LESTARI project in the last two years, which they did through outreach and engaging stakeholders (government agencies) to reach greater understanding, coordination, cooperation. Recommendations for repealing various 3-4 target laws and regulations are underway to get the necessary legislative stamp. Lestari’s support for Policy work is very much appreciated and is done in tandem with existing technical support like UNOPS. • FPIC through multi-stakeholder consultations in building dam blockage as part of fire prevention and habitat restoration is one of the success stories of KK. This process has been unanimously praised as useful by all level of respondents: governments, village representatives, disaster management units, government planning representatives. This process was done in a very participatory way, and in voluntary basis where approximately 23 villages were consulted, resulting in 11 villages participating. • The Integrated Fire Management plan is being institutionalized through cooperation with several departments led by the Disaster Management department. Here LESTARI support is on policy, and in facilitating the coordination. Village: • RPJMDes District: • Spatial planning Province: • One stop shop activities (collaboration with OSS as a National: • LESTARI’s TA mostly at landscape level; for national level intervention: coordinate with MinEnFor and ESDM. Possibly determine scope of collaboration with other USAID programs (e.g. BIJAK) with mandate at national level to support LESTARI implementation. 89 LESTARI MID-TERM EVALUATION USAID.GOV user of the SST for coordinating/ monitoring licensing decision making) were stopped, yet our interview showed that the SST tool would be “exactly what we need here” (and what the OSS is supposed to create). OSS is does not get sufficient information by BAPPEDA (who is coordinating among the agencies) - but OSS is also aware that (1) sometimes not the appropriate staff attends the meetings, (2) internally (OSS) people in charge of licensing might not be supportive to change “The ones who are still in the room are more progressive” (after some people, including the licensing department, have left the room). National: • One map policy (is a national policy and not a LESTARI activity, but something that LESTARI is supporting through pilots at provincial level in Aceh and in Papua through SST. Initial activities were done in KK the first year and were stopped due to lack of political support, according to LESTARI respondents) • Current legal mandates for land use decision-making at national level (e.g. concessions for mining by ESDM, concessions for forestry by DINAS EnFor Province) can be conflicting with the legal mandates at local levels (spatial planning, RPJMDes) 4 To what extent are Multi￾Stakeholder Forums influencing land use decision-making, monitoring and planning? • Team 9 (core group of the MSF in Pulang Pisau) shows strong ownership and an elaborate level of comprehension of the positive impact of fire prevention as well as respective budget management • Team 9 through FPICs process encourages BRG (Badan Restorasi Gambut - Peatland Restoration Board) to allocates activities and programs in fire prevention by 90 LESTARI MID-TERM EVALUATION USAID.GOV channel blocking. This team leads for fire prevention planning that actually for controlling land use as well. 5 To what extent have LESTARI activities led to provincial and district governments to effectively monitor land use and licensing? • LESTARI has Michigan State University as a sub￾contractor in cooperation with 2 universities: Palangka Raya and Muhammdiyah Universities who are doing land use monitoring and MRV among other topics through an e-learning platform that will be launched in public next year. • Unlike Aceh, it is hard to make a conclusion for this in KK. Our informants at Environment and Forestry were new; and Bappedas was led by an upbeat and positive person who appreciates LESTARI support, and who also works with other donors like GGGI (Global Green Growth institute). It is hard to determine whether LESTARI activities have directly led to effective monitoring of land use. 6 To what extent is LESTARI effectively encouraging management of conservation areas (e.g. co-management, FMUs, METT, etc.)? • • L support is evident at establishing village forest at Garung village(co-management) and situation same in other villages • Self-claimed by Dwima: RIL-C training provided by a LESTARI subcontractor (Tropical Forest Foundation) decreases threat to wildlife (minimized damage to the plantation, pre-felling monitoring and evacuation of wildlife); in collaboration with Dwima (FSC-certified), Dayak tribes living in the concession area shall be encouraged to co-manage the HCV (high conservation value) areas (future). • KPH/FMUs 16 and 31 were casually mentioned, and it seems not related to LESTARI activities. 91 LESTARI MID-TERM EVALUATION USAID.GOV • METT Scores • National Parks representatives found it useful to have a baseline data on their efficiency rating, which they acknowledge as a service from the LESTARI project. They now have an aim to increase their scores annually. • Good practice to have a third-party validation, which was experienced by Sebangou NP, which lowered their self￾assessment but also showed an increased rate from the baseline. • 7 To what extent are SMART patrols preventing wildlife trafficking and poaching? • National Parks (B3RNP and SNP) both do not hold equipment which is compatible to the software • Conservation managers also complain of being unable to conduct SMART patrols as often as agreed. 8 To what extent are LESTARI public-private partnerships successful in increasing environmentally sustainable commodity production? • “PPPs” in LESTARI = (1) profit-orientated producer￾buyer relations as well as (2) partnership between LESTARI and a private partner • Focus of PPP engagement so far has been on rubber in Pulang Pisau (production forest which does not need many chemicals, can grow in peatland, and has shown to have a large potential as a carbon sink); 80% of the communities have traditionally worked on rubber ● “...environmentally sustainable commodity production” - should be changed to “...environmentally sustainable livelihoods” to allow for alternative revenue creation to be supported by LESTARI which includes adding value along the value chain and services sectors (e.g., small-scale renewable energy provision, packaging, marketing, information center, etc.). 9 To what extent are these partnerships reducing deforestation and forest degradation? • Rubber tapping communities have received replanting support immediately after the big fire in 2015, which has provided hope and hence prevented from selling the land to palm oil plantation • Rubber community now plans to replant rubber and sengon. Opportunity: The SOP on fire prevention can be 92 LESTARI MID-TERM EVALUATION USAID.GOV applied to other commodities. Risk: While for replanting, sengon is suitable, for expansion it actually needs land￾clearing. 10 To what extent has LESTARI increased access and use of long￾term financing for conservation and restoration? • Via match-making grants LESTARI has leveraged project budget from three grantee Foundations/NGOs (Karsa, BOSF, YTS), e.g. Yayasan Tambuhak Sinta (YTS) is a development foundation established by the mineral exploration company PT Kalimantan Surya Kencana; BOSF has many different donors) • Other donors: grantees (e.g. OIC) leveraged having USAID Lestari funds to generate more attention and other donors successfully. Some can survive without LESTARI support, but LESTARI added-value are well￾articulated especially in policy, increased capacity, etc. • Public institutional financing: Government budget allocation in annual work plans increased for conservation activities through program and activities in developed RPJMDes for 6 years in village level • In district and provincial level, KLHS (SEA) that is assisted to review RPJMD, results in the conservation and restoration program for 6 years. The planned program and activities accesses national or local budget. • PES study completed; No PES yet in implementation • • Corporate finance: Rubber business model in Pulang Pisau has integrated the rubber tapping community into the rubber processing value chain; MoUs with buyer companies to support more sustainable production of Project should ensure the involvement of the certain agencies of local government to put implemented programs in community level in financing their annual work plans, for co-funding 93 LESTARI MID-TERM EVALUATION USAID.GOV agricultural raw materials (organic farmland) • If “long-term financing” originally meant attracting institutional investors, e.g. via the issuance of municipal bonds and/or corporate bonds, this has not been implemented. 11 How is the implementation structure and management of LESTARI contributing to its successes and are there areas for improvement? • We find LESTARI management and staff committed and dedicated to conservation, partners and communities. Their passion can be seen through. There are generally many areas for improvement, and in our rapid assessment, it is that internal communication amongst partner implementers should be done in a timely and energetic manner, with follow ups needed to ensure communication, and avoid misunderstanding amongst partners. 12 To what extent are LESTARI activities undertaken with a focus on sustainability of results after the project is closed? L creates local champion from community level to ensure that their program will still be implemented when the project ends. Understandably, landscape approach conservation goals need to be undertaken for far longer periods of time than 4 years. During assessment at mid-term, we find that while there were many significant achievements, there were also a feeling of just finishing the planning stages and therefore implementation might not be reached during the project duration. Exit strategies do not seem to be clearly explained or communicated to various partners. Private Sector Engagement: • Rubber tapping community in Buntoi (Pulang Pisau) (previous support by IFACS) confirmed the value of the fire prevention SOP for their farmland operation. For rubber, in control of analyzing price and market information (via mobile access to internet), understanding the value chain and in control of bargaining Landscape approach work needs to be beyond 4 years. LESTARI got off to a very good start, but the ballooning needs of huge landscapes being protected means the momentum might be hard to contain within the limited project duration without having created disappointments among many partners. Private Sector Engagement: • For sustainable livelihood, build on achievements with RPJMDes to support the villagers (beyond the rubber community) to be more in control of their future livelihood (innovation capacity). This includes understanding the market in which they are operating via user-friendly tools and their application like financial 94 LESTARI MID-TERM EVALUATION USAID.GOV directly with the processing company (no agent). Note: difficult to replicate this business model in other areas and other commodities where the processing companies are not geographically nearby (e.g. South of the KK landscape or Java). • The communities where LESTARI supported with rubber now think about growing sengon (see above under Q9). plan/break even, analysis of market information, understanding of simplified examples for business model innovation 13 To what extent is the theory of change still valid? • Most of the elements of theory of change remain valid. What would make it more valid as expressed by respondents would be greater harmonization of laws and greater coordination among regulation agencies to achieve LESTARI’S aim. Poverty reduction is the overwhelming factor echoed by respondents (government agencies, NGO partners, conservation agents as) necessary to conserve the forest and biodiversity. 14 To what extent are LESTARI activities aligned with the Theory of Change? Co-management, Advocacy for increased constituency, and private sector engagement are equally being targeted by LESTARI activities. We therefore find them aligned. Private Sector Engagement: ● Implementation: main focus on rubber tapping communities (KUBK) in Pulang Pisau and timber concessions in Katingan; pipeline: expansion of rubber farming in other areas of the landscape, eco-tourism and non-timber forest produce (NTFP) ● LESTARI incorporates sustainable livelihoods approach as a tool for conservation ● Match-making buyer-supplier relationships accelerates the building of supply chains ● Learning with RIL-C (return on investment below three years) supports the long-term benefit of the timber concessionaires (economic savings, image). Private Sector Engagement: • Couple the existing demand orientation with a critical assessment of local economic development opportunities and corresponding technical advice (incl. early TA on the economic feasibility of supported commodities (long-term trends), de￾risking via diversification, consideration of opportunity cost of giving up illegal practices, and measurement of related carbon emission (incl. from energy and transport). Buyers - especially larger brands - are increasingly asked to disclose on environmental and socio- 95 LESTARI MID-TERM EVALUATION USAID.GOV economic impact created/SDG reporting along the value chain, to which this could contribute. 15 To what extent is there buy in for LESTARI interventions across the GOI (national, provincial, local, community)? • Regulation and Regent Decree for developing RPJMDes, in term of budget allocation for conservation management is being processed to be the reference for village • There is a huge buy in for the FPIC method; the multi￾stakeholder plans for canal blocking • There is overwhelming and organized support for the Integrated Fire Management that is absorbed by the Disaster Management Unit, Bappenas, MOEF. (Team 9 MSF) 16 To what extent is USAID management of LESTARI contributing to its successes and are there areas for improvement? • A major delay in the ecotourism plans promised to 2 NPs are creating discord and disappointment amongst partners, and damaging USAID’s reputation. 96 LESTARI MID-TERM EVALUATION USAID.GOV # EVALUATION QUESTIONS CYCLOPS LANDSCAPE DISCUSSION NOTES 1 To what extent are LESTARI outreach activities building constituencies in support of biodiversity and forest conservation? • There has been a coalition between province government (DPMK), Jayapura district government (DPMK) and BKSDA on supporting conservation RPJMK, conservation village regulation, community mapping, community rangers, and management in the buffer zone • Outreach to migrant inhabitants seems still minimum: only through Smart Patrol and the vanilla initiative It seems there has been no (effective) outreach targeting: (a) the buyers of commodities illegally produced in Cyclops: timber and wildlife; (b) Beneficiaries of Cyclops environment service: water users • More (effective) outreach activities targeting migrant inhabitants, buyers of illegal commodities produced in Cyclops, and water users in cities/towns surrounding Cyclops (Jayapura, Sentani, etc. 2 To what extent has LESTARI been able to prevent conversion of critical areas to other purpose areas? • The support package for pilot/priority villages looks promising to prevent conversion in Cyclops: RPJMK, conservation village regulation, and community rangers. The villagers of Nechebe are committed to protect the area which according to their custom need to be protected • The full package has been applied only in 2 pilot villages so far which both are all villages with only local people (no migrant settlement). How much the full support package can anticipate • the need for more land as consequence of permanent crop cultivation is still questionable Intervention on migrant settlements conducted through the vanilla initiative and smart patrol activity • The vanilla initiative is still in the early stage, hard to assess its impact on preventing conversion. However, one of the farmer groups has agreed with the draft of the • Need to investigate more on how to anticipate the impact of permanent crop cultivation on the need for more land in the Nature Reserve for the future Need to carefully review the technical/cultural feasibility and economic viability of vanilla cultivation in Cyclops 97 LESTARI MID-TERM EVALUATION USAID.GOV conservation agreement where they agree not to clear land inside the Nature Reservation. 3 To what extent are LESTARI policy recommendations incorporated into governance documents at the district, province, and national levels? • The community maps and Smart Patrol data are used by BKSDA to define land use zoning of Cyclops NR • Local Regulation of Jayapura District 9/2015 on protection and management of Cyclops buffer zone • Local Regulation of Jayapura District 10/2016 on Adat village • Consider advising BKSDA/local government to produce guideline for conservation village regulation 4 To what extent are Multi￾Stakeholder Forums influencing land use decision-making, monitoring and planning? • MSF has not been formed for Cyclops. They plan to form it on the 6th of Dec as part of MSF Papua Province; • Consider learning from Mimika on how to make MSF effective in mobilizing political support and financial resources for conservation initiatives 5 To what extent have LESTARI activities led to provincial and district governments to effectively monitor land use and licensing? • Province government has significant role in licensing decision at district. LESTARI efforts on supporting SIMTARU and SST in provincial government is strategic. The SIMTARU has been installed and operational, used by Bappeda of the province to monitor land use and licensing. • The one stop licensing service at province government (PTSP) does not fully use SIMTARU and still has no clear understanding about SST While Bupati depends on the input from the Environment Impact Assessment team (which organized by the Environment Agency of Papua Province) for making decision on environment license, the agency does not use SIMTARU and SST. • There is no case where SIMTARU (and SST) have been used to prevent license application that conflicted with spatial plan in the two districts. • LESTARI needs to work with other strategic agencies at province government on monitoring land use and licensing: Environment agency, Plantation agency, Forestry agency, and Mining agency. • Need more intensive work with Province PTSP and BAPPEDDA to ensure that SST development serve their need without overlapping function with SIMTARU • Need to increase the capacity of district government on utilization of SIMTARU and SST to support decision making on location permit, environment license, building permit. 98 LESTARI MID-TERM EVALUATION USAID.GOV 6 To what extent is LESTARI effectively encouraging management of conservation areas (e.g. co-management, FMUs, METT, etc.)? • The METT score for Cyclops has been increased from 43 (2015) to 61 (2017) • Co-management has been piloted in 2 villages plus some start-up activities in vanilla villages • Need to find the most efficient way to scale-up the full-package support in the 2 pilot villages to all villages in Cyclops • Need to carefully review the technical/cultural feasibility and economic viability of vanilla cultivation in Cyclops 7 To what extent are SMART patrols preventing wildlife trafficking and poaching? • Smart Patrol has been practiced though not supported by Resort Based Management • The data have been used to plan patrol tracks, identify defend conservation activities in different location, and defining land use zoning • Continue working with BKSDA to establish necessary capacity to make smart patrol fully functioning 99 LESTARI MID-TERM EVALUATION USAID.GOV 8 To what extent are LESTARI public-private partnerships successful in increasing environmentally sustainable commodity production? • The support on vanilla cultivation targets villages with high threats to Cyclops where local and migrant people are living. But the activities are still in the early stage in 4 villages covering 180 members (formation of production group, organizing). Conservation agreement has been drafted and one of the groups--led by Pak Edison Fale--has agreed with the draft. The agreement includes a clause where the local community agree not to clear land inside Cyclops NR • Need to carefully review the technical/cultural feasibility and economic viability of vanilla cultivation in Cyclops • Need to prepare a clear production and value chain plan 9 To what extent are these partnerships reducing deforestation and forest degradation? • The farmers just start planting the vanilla crop. It is too early to assess how much this initiative can really prevent encroachment • Investigate the possibility to replicate the Edison Fale model to other groups 10 To what extent has LESTARI increased access and use of long term financing for conservation and restoration? • Leveraging local government and village fund to support conservation program through mid-term village development plan. The province government (DPMK) and the village (Nechebe) confirm that they agree to use village fund to support conservation activities such as community ranger. • There is idea to develop PES scheme with the water utility (PDAM) in Jayapura. The head of BKSDA also has the idea to mobilize financing resources from businesses in Jayapura (such as banks). Both are still in discussion. • More (effective) outreach activities targeting migrant inhabitants, buyers of illegal commodities produced in Cyclops, and water users in cities/towns surrounding Cyclops (Jayapura, Sentani, etc) 100 LESTARI MID-TERM EVALUATION USAID.GOV 11 How is the implementation structure and management of LESTARI contributing to its successes and are there areas for improvement? • Appointing WWF as the leading implementing partner in Cyclops is a strategic decision. WWF's long-term experience in Cyclops is a great asset for LESTARI to accelerate the process. • LESTARI staff working on vanilla initiative has no specific experience with vanilla. There is no NCBI staff who dedicated supporting and based in Cyclops (remotely support from Genyem) • Need to re-arrange the cooperating with NCBI to have dedicated staff with strong knowledge and skill on vanilla production 12 To what extent are LESTARI activities undertaken with a focus on sustainability of results after the project is closed? • Financing sustainability: leveraging village and local government fund through village mid-term development plan • Technical sustainability: Support organizational and human capacity of BKSDA/FMU and BKSDA/FMU staff: establishment of resort-based management (RBM), conservation area management through METT, and provide trainings on participatory mapping, GIS, multidisciplinary land use assessment (MLA), and smart patrol • Political sustainability: Some of the advices have been formalized into regulations • Carefully plan institutional development of FMU Cyclops who will take over the management of Cyclops NR • More (effective) outreach activities targeting water users in cities/towns surrounding Cyclops (Jayapura, Sentani, etc.) 101 LESTARI MID-TERM EVALUATION USAID.GOV 13 To what extent is the theory of change still valid? • The main threat to Cyclops is encroachment by migrant and the development of urban area in Jayapura. • Land-use governance can prevent government issue license in Cyclops and together with co-management can increase acceptability of Natural Reserve by the local people while increase the capacity of conservation area management • Private sector engagement can change the practice of producer group as well as buyer. There is a potential to leverage sustainable financing through PES scheme • Continue with the ToC with emphasize to have more intervention on migrant settlements and outreach activities targeting water users 14 To what extent are LESTARI activities aligned with the Theory of Change? • LESTARI activities fully aligned with ToC. We do not find any activities implemented by LESTARI that are irrelevant with the ToC. • Lack of activities with the aim to practice of consumers/buyers of commodities produced in Cyclops: timber and wildlife 15 To what extent is there buy in for LESTARI interventions across the GOI (national, provincial, local, community)? • All respondents from GoI appreciates LESTARI supports of which some of them expect LESTARI to be prolonged after 2020 16 To what extent is USAID management of LESTARI contributing to its successes and are there areas for improvement? • We did not hear any comment about USAID management of LESTARI 102 LESTARI MID-TERM EVALUATION USAID.GOV # EVALUATION QUESTIONS LORENTZ LANDSCAPE DISCUSSION NOTES 1 To what extent are LESTARI outreach activities building constituencies in support of biodiversity and forest conservation? • There is an active coalition between civil society group (Papua Animal Care), government (BKSDA), and private sector (Freeport) in combating wildlife trafficking and public awareness raising on wildlife conservation in Mimika • Number of stakeholders has agreed to establish Friends of Lorentz Foundation to support conservation program in Lorentz area of which one of them (Freeport) agree to provide initial financing • Village community and district government agree to support mangrove forest conservation at least in 2 villages in Asmat • Need outreach activity targeting buyers of commodities produced in Lorentz/Mimika/Asmat: timber and oil palm 2 To what extent has LESTARI been able to prevent conversion of critical areas to other purpose areas? • The district secretary of Asmat agree to change the classification of Rawa Baki from limited production forest to protected forest • FMU VI Mimika agree to protect sacred and important areas for adapt people, to be integrated into long-term forest management plan (RPHJP) • LESTARI does not work to improve the practice of timber and oil palm concessionaires in Mimika. • Need to better consult the long-term forest management plan of the FMU VI to the stakeholders: concessionaries, local timber business association, and local communities 3 To what extent are LESTARI policy recommendations incorporated into governance documents at the district, province, and national levels? • Bappeda of Mimika will incorporate recommendation from SEA into Spatial Plan Review of Mimika • The Lorentz National Park will adopt community maps into the revision of national park zoning • Adat community land use plan data are incorporated into detail spatial plan of POMAKO industrial estate • The district secretary of Asmat agree to change the classification of Rawa Baki from limited production forest 103 LESTARI MID-TERM EVALUATION USAID.GOV to protected forest 4 To what extent are Multi￾Stakeholder Forums influencing land use decision-making, monitoring and planning? • The MSF in Mimika is very active for facilitation multistakeholder discussion on spatial plan review and land use monitoring 5 To what extent have LESTARI activities led to provincial and district governments to effectively monitor land use and licensing? • Province government has significant role in licensing decision at district. LESTARI efforts on supporting SIMTARU and SST in province government is strategic. The SIMTARU has been installed and operational, used by Bappeda of the province to monitor land use and licensing. • SIMTARU has been used in province and Mimika: There is a case where governor of Papua declined the proposal from Bupati Mimika to convert protected forest for smelter plant. And another case where the spatial plan coordination board (BKPRD) of Mimika rejected a license for sago processing plant because it is in protected forest • Bupati (head of district) has the role to issue location permit and environment license. But none of the district governments have been using SIMTARU to support the Bupati for decision making on location permit and environment license. • While Bupati depends on the input from the • LESTARI needs to work with other strategic agencies at province government on monitoring land use and licensing: Environment agency, Plantation agency, Forestry agency, and Mining agency. • Need more intensive work with Province PTSP and BAPPEDDA to ensure that SST development serve their need without overlapping function with SIMTARU • LESTARI needs to increase the capacity of local government and civil society groups on land use monitoring and supporting law enforcement (may be by operationalizing SIMTARU and SST) 104 LESTARI MID-TERM EVALUATION USAID.GOV Environment Impact Assessment team (which organized by the Environment Agency of Papua Province) for making decision on environment license, the agency does not use SIMTARU and SST. • The one stop licensing service at province government (PTSP) does not fully use SIMTARU and still has no clear understanding about SST 6 To what extent is LESTARI effectively encouraging management of conservation areas (e.g. co-management, FMUs, METT, etc.)? • METT score of Lorentz National Park has been improved from 53 (2015) to 60 (2017) • Co-management agreement between local communities and crab traders in mangrove ecosystem has been signed for two pilot villages in Asmat • Community land use plan has been incorporated into the draft of long-term forest management plan Mimika VI • LESTARI needs to explore alternative fiscal policies to support financing national park and FMU operation 7 To what extent are SMART patrols preventing wildlife trafficking and poaching? • Lorentz National Park is practicing Smart Patrols, but they do not use the data to support law enforcement. Smart Patrol is more useful for engaging with local communities and to collect data for zoning than for preventing wildlife trafficking and poaching • Despite the fact that wildlife trafficking in Papua is rampant, smart patrols never find violations • Need to explore alternative tool/technology for "patrolling" ex￾situ using social media by concerned communities 105 LESTARI MID-TERM EVALUATION USAID.GOV 8 To what extent are LESTARI public-private partnerships successful in increasing environmentally sustainable commodity production? • Co-management agreement between local communities and crab traders in mangrove ecosystem has been signed for two pilot villages in Asmat (61,000 ha) • LESTARI provided trainings on mangrove fruit processing in Asmat and Mimika. There is no plan on production organization and supply chain • Need to produce a solid production and business plan for the alternative source of income initiative such as mangrove fruit processing 9 To what extent are these partnerships reducing deforestation and forest degradation? • The mangrove forest in the two pilot villages in Asmat (Ewer and Saw) looks still intact • Need to provide the capacity to the patterning parties and district government to monitor the quality of the mangrove forests regulated in the agreement 10 To what extent has LESTARI increased access and use of long term financing for conservation and restoration? • Number of stakeholders has agreed to establish Friends of Lorentz Foundation to support conservation program in Lorentz area of which one of them (Freeport) agree to provide initial financing There are 9 villages with conservation village mid-term development plan to leverage local government and village fund • Carefully design the organization and the business plan of the Friends of Lorentz Foundation • Carefully assess the impact of the ending special autonomy (Otsus) fund by 2021 to the financial capacity of the local government and villages 11 How is the implementation structure and management of LESTARI contributing to its successes and are there areas for improvement? • LESTARI implementing partners appreciate the planning cycle (planning, monitoring and evaluation) tools applied in the project management: clear direction and indicators to measure the progress (AMEP) • DCP and technical team leaders sometimes send different messages/guidance to field officers in the landscape • Continue with the planning cycle tools Improve coordination between DCP and technical team leaders on providing guidance to the staff in the landscapes 106 LESTARI MID-TERM EVALUATION USAID.GOV 12 To what extent are LESTARI activities undertaken with a focus on sustainability of results after the project is closed? • Financial sustainability: village mid-term development plan, Fried of Lorentz Foundation • Political sustainability: Lorentz zoning plan, FMU long￾term management plan, SEA-Spatial plan review to change the classification of Rawa Baki into protected forest/Essential ecosystem zone, champions • Technical sustainability: Provide trainings to landscape managers: Lorentz national park, FMU VI, local government officers • Consider focusing on the Friend of Lorentz Foundation as the prime candidate to take over LESTARI roles 13 To what extent is the theory of change still valid? • Conservation co-management and private sector engagement are required to improve the management of Lorentz National Park, FMU VI as well as mangrove forests by Adat community, landscape managers, and private sector. • Land use governance is essential to rationalize land use plan in Asmat (to have more land for town development in Agats and to convert Rawa Baki into protected area) as well as for land use monitoring and combating wildlife trafficking in Mimika and Lorentz national park 14 To what extent are LESTARI activities aligned with the Theory of Change? • LESTARI activities fully aligned with ToC. We do not find any activities implemented by LESTARI that are irrelevant with the ToC • There have been lack of activities to engage with timber and oil palm concessionaires in Mimika and wildlife trafficking in Asmat 107 LESTARI MID-TERM EVALUATION USAID.GOV 15 To what extent is there buy in for LESTARI interventions across the GOI (national, provincial, local, community)? • All respondents from GoI appreciate LESTARI supports of which some of them expect LESTARI to be prolonged after 2020 16 To what extent is USAID management of LESTARI contributing to its successes and are there areas for improvement? # EVALUATION QUESTIONS MAPPI LANDSCAPE DISCUSSION NOTES 1 To what extent are LESTARI outreach activities building constituencies in support of biodiversity and forest conservation? • LESTARI is mainly working with local governments on HCV assessment and SEA-spatial plan reviews to enlarge protected areas. We received confirmation from the 2 district governments that they appreciate LESTARI support on SEA-spatial plan and will increase the protected areas in the spatial plan • Adat councils in the two districts support LESTARI on the SEA-spatial plan review. But there is no evidence of activities to support SEA-spatial plan and HCV mapping based on their own initiative. They did something only when they were encouraged by LESTARI. • KORINDO signals the openness to adopt some sustainability practices (HCV, FPIC) and to cooperate with LESTARI, but the engagement with the company is still in exploration stage. • Activities on constituency building at community level is very low • Improve the engagement with NGOs working on community based conservation to have better cooperation in supporting SEA-spatial plan review. The key NGOs include Sylva Papua who seems to have solid work with adapt communities in more than 300,000 ha in the same landscape • Continue cooperation with KORINDO and keep it transparent to local and national civil society groups • Need outreach activity targeting buyers of commodities produced in Mappi Boven: timber and oil palm 108 LESTARI MID-TERM EVALUATION USAID.GOV • There is no evidence a coalition between the stakeholders pursuing one policy initiative or one key activity co-financed by more than one actor. 2 To what extent has LESTARI been able to prevent conversion of critical areas to other purpose areas? • There is a potential SEA-spatial plan review will lead to extend the protected areas in both districts • There is an opportunity for KORINDO to protect some of the HCV areas and adapt forests. This is particularly in the oil palm concession since they are considering applying for RSPO certification and has the interest to apply FPIC. But we don't know how much of the HCV areas will be protected. • LESTARI has not started working in priority villages • Beside support on SEA-spatial plan, LESTARI needs to increase the capacity of local government and civil society groups on land use monitoring and supporting law enforcement (may be by operationalizing SIMTARU and SST). 109 LESTARI MID-TERM EVALUATION USAID.GOV 3 To what extent are LESTARI policy recommendations incorporated into governance documents at the district, province, and national levels? • The two district governments express appreciation for LESTARI support on the SEA-spatial plan review, and the willingness to adopt the HCV data to enlarge protected areas. • Spatial plan review is still on progress. 4 To what extent are Multi￾Stakeholder Forums influencing land use decision-making, monitoring and planning? • MSF has been formed in the two districts. The members are involved in the process of SEA-spatial plan review • There is no initiative on land use monitoring. SIMTARU and SST are not used in the two districts. • The MSF members in Boven Digul emphasize MSF organization than the cooperation on key initiatives that serve their common interests • There is a need to change MSF format in Boven Digul, to emphasize cooperation on key initiatives rather than the organization of the MSF itself 5 To what extent have LESTARI activities led to provincial and district governments to effectively monitor land use and licensing? • The provincial government has significant role in licensing decision at district level. LESTARI efforts on supporting SIMTARU and SST in province government is strategic. The SIMTARU has been installed and operational, used by Bappeda of the province to monitor land use and licensing. However, there is no case where SIMTARU (and SST) have been used to prevent license application that conflicted with spatial plan in the two districts. • Bupati (head of district) has the role to issue location permit and environment license. But none of the district governments have been using SIMTARU to support the Bupati for decision making on location permit and environment license. • While the Bupati depends on the input from the Environment Impact Assessment team (which is • LESTARI needs to work with other strategic agencies at province government on monitoring land use and licensing: Environment agency, Plantation agency, Forestry agency, and Mining agency. • Need more intensive work with Province PTSP and BAPPEDDA to ensure that SST development serve their need without overlapping function with SIMTARU • Need to increase the capacity of district government on utilization of SIMTARU and SST to support decision making on location permit 110 LESTARI MID-TERM EVALUATION USAID.GOV organized by the Environment Agency of Papua Province) for making decisions on environment license, the agency does not use SIMTARU and SST. • The one stop licensing service at province government (PTSP) does not fully use SIMTARU and still has no clear understanding about SST and environment license 6 To what extent is LESTARI effectively encouraging management of conservation areas (e.g. co-management, FMUs, METT, etc.)? • There is no conservation area in Mappi Boven • LESTARI needs to increase the capacity of local government and civil society groups on land use monitoring and supporting law enforcement (may be by operationalizing SIMTARU and SST) to ensure that HCV areas are protected. 7 To what extent are SMART patrols preventing wildlife trafficking and poaching? • There is no application of Smart Patrol in Mappi Boven 8 To what extent are LESTARI public-private partnerships successful in increasing environmentally sustainable commodity production? • There is no initiative on sustainable commodity production yet underway. • The cooperation with KORINDO is still in exploration stage • Need to explore potential public￾private-community partnership in sustainable timber and non-timber forest production to replace the role of rubber in local economy. 111 LESTARI MID-TERM EVALUATION USAID.GOV 9 To what extent are these partnerships reducing deforestation and forest degradation? • No evidence was produced 10 To what extent has LESTARI increased access and use of long term financing for conservation and restoration? • If the Spatial plan review successfully increase the protected areas, government fund will be used to protect the protected areas. • Continue with SEA-spatial plan review process by engaging NGOs working on community-based conservation 11 How is the implementation structure and management of LESTARI contributing to its successes and are there areas for improvement? • LESTARI implementing partners appreciate the planning cycle (planning, monitoring and evaluation) tools applied in the project management: clear direction and indicators to measure the progress (AMEP) • The practice where the implementing partner need to pay activities in advance a bit hampering the smooth activities • Unclear timeframe standard on decision making for approving business travel and activities could risk on delaying activities or unnecessary high cost expenses • Continue with the planning cycle tools • Where possible, LESTARI provides advance payment to implementing partner for conducting approved activities • Consider applying SoP for decision making for business travel and activities (SoW) approval 12 To what extent are LESTARI activities undertaken with a focus on sustainability of results after the project is closed? • Political sustainability: there is an opportunity to enlarge protected areas through spatial plan review • Continue with SEA-spatial plan review process by engaging NGOs working on community-based conservation • Increase efforts on capacity development for district government and civil society organization for their functions on licensing and land use monitoring 112 LESTARI MID-TERM EVALUATION USAID.GOV 13 To what extent is the theory of change still valid? • The main threat to Mappi Boven is forest conversion by concessionaires and traditional agricultural practices involving use of fire for crop-field preparation • Land-use governance can enlarge protected areas and improve compliance on spatial laws and regulation. • Co-management and private sector engagement can increase sustainable commodity production by local communities and concessionaires and/or partnership between the two. • Continue with the ToC with emphasize to develop strategic cooperation with NGOs working on community-based conservation 14 To what extent are LESTARI activities aligned with the Theory of Change? • LESTARI activities fully aligned with ToC. We do not find any activities implemented by LESTARI that are irrelevant with the ToC. Co-management activities have not been started in Mappi Boven 15 To what extent is there buy in for LESTARI interventions across the GOI (national, provincial, local, community)? • Local government leaders in the two districts indicated that they are willing to adopt LESTARI advice on SEA￾Spatial plan review 16 To what extent is USAID management of LESTARI contributing to its successes and are there areas for improvement? • No discussion raised by communities 113 LESTARI MID-TERM EVALUATION USAID.GOV ANNEX IV: EVALUATION MATRIX Evaluation Question Matrix Evaluation Question (Proposed Refinement) Data Collection Strategy Origin of the data TECHNICAL COMPONENT 1: Forest and Land Use Governance & Advocacy 1. Is LESTARI effective in identifying and engaging target audiences in order to achieve the activity goal? Document review Key informant interviews Focus group discussions LESTARI activity reports (social media campaigns, story pitching, journalist activities); Media reports LESTARI activity partners; Civil society leaders; Local and national GOI officials; Private sector/ concessionaires; Journalists Multi-stakeholder forum members; Landscape community members 2. Is LESTARI building constituencies in support of biodiversity and conservation? Key informant interview s Focus group discussions Direct observations LESTARI staff; LESTARI activity partners; Civil society leaders; Local and national GOI officials; Private sector/ concessionaires Multi-stakeholder forum members; Landscape community members Integra evaluation team field notes from within landscape of evidence of functional constituencies 3. Is LESTARI preventing conversion of critical areas? Document review Direct observations Synthesis from data collection LESTARI activity reports (SEA/LCPs); LESTARI monitoring data; Data and data visualization (e.g. maps) produced by external sources (i.e. WRI Global Forest Watch). Integra evaluation team field notes and photos of forest integrity Change analysis against multiple data sources 4. Are Multi-Stakeholder Forums influencing land use decision-making, monitoring and planning? Key informant interviews Focus group discussions Synthesis from data collection Landscape managers; LESTARI activity partners; Multi-stakeholder forum management Multi-stakeholder forum members Cross analyze data with evidence of conversion 5. Is government uptake of LESTARI land use monitoring data effective for land Document review LESTARI monitoring data 114 LESTARI MID-TERM EVALUATION USAID.GOV registration? Key informant interviews Focus group discussions Local and national GOI officials; Private sector/ concessionaires; Landscape managers Multi- stakeholder forum members 6. Is LESTARI support being incorporated into governance documents at the district, province and national levels? Document review Key informant interviews LESTARI activity reports; Relevant GOI documents Local and national GOI officials; Private sector/concessionaires TECHNICAL COMPONENT 2: Improved Management of Conservation Areas 7. Is LESTARI encouraging improved management of conservation areas? Document review Key informant interviews Direct observations LESTARI activity reports; Review of METT scores LESTARI activity partners; Local GOI officials; Landscape managers; Private sector/ concessionaires Integra evaluation team field notes and photos of improved management practices 8. Is LESTARI support of the co￾management structures and FMUs capable of increasing forest stewardship? Document review Key informant interviews Focus group discussions Multi-stakeholder forum reports; Key planning documents LESTARI staff; LESTARI activity partners; Multi-stakeholder forum management; Private sector/ concessionaires Landscape community members 9. Is SMART use by patrols effective in combatting wildlife trafficking and poaching in the landscapes? Document review Key informant interviews Direct observations Synthesis from data collection LESTARI activity reports; Landscape management patrol records Landscape managers; SMART users (rangers) Integra evaluation team field notes on number and condition of SMART devises, utilization, etc. LESTARI data cross checked against TRAFFIC, Interpol data TECHNICAL COMPONENT 3: Improved Private Sector Practices 10. Is LESTARI promoting green investment which reduces deforestation/degradation? Document review Key informant interviews LESTARI activity reports LESTARI activity partners; Forest Carbon; Private sector/ concessionaires 11. Is LESTARI securing long term financing Document review LESTARI activity reports 115 LESTARI MID-TERM EVALUATION USAID.GOV mechanisms for conservation and restoration? Key informant interviews Private sector/ concessionaires; GOI economic growth development planners; Green finance institutions 12. Is LESTARI encouraging private sector participation to increase local economic development? Document review Key informant interviews Focus group discussions Policy and planning documents LESTARI activity partners; Multi-stakeholder forum management; Private sector/ concessionaires Landscape community members Management Effectiveness and Sustainability 13. How is the implementation structure and management of LESTARI contributing to its successes and are there areas for improvement? Key informant interviews Synthesis from data collection LESTARI staff; LESTARI activity partners Synthesis of all relevant data collected under Technical Component questions above; analysis of various data points around changing conditions and responsiveness to learning by LESTARI 14. What mechanisms are in place to insure LESTARI sustainability? Key informant interviews Direct observation Civil society leaders; Multi-stakeholder forum management; Local and national GOI officials; Private sector/ concessionaires Integra evaluation team field notes Program Design and Performance Questions 15. What evidence exists to support and validate the results framework and theory of change for LESTARI? Document review Synthesis from data collection LESTARI strategic documents; LESTARI activity reports Synthesis of all relevant data collected under Technical Component questions 116 LESTARI MID-TERM EVALUATION USAID.GOV ANNEX V: TEAM COMPOSITION John Waugh Evaluation Team Leader John Waugh is Integra’s Vice President for Climate and Environment. He has more than 30 years of experience in climate change, biodiversity and natural resource project management, planning, and policy. He is experienced in the management of programs and teams, and skilled in the analysis and replication of factors contributing to ecological and social resilience, participatory conservation strategies and planning, climate adaptation planning, and sustainable finance mechanisms. He has led or supervised numerous program and project evaluations and assessments for USAID, including heading a 12-person international scientific team to evaluate the USAID Central Africa Regional Program for the Environment, involving travel to eight remote and insecure landscapes in three countries. He has also evaluated a program implemented by the Organization of American States with funds from the Global Environment Facility and activities for IUCN. He has first-hand knowledge and work experience in more than 30 countries in Africa, Latin America and the Caribbean, East and South-East Asia, working in progressively more complex roles in NGOs, international organizations, and most recently in the private sector. He is adept at designing solutions, recruiting experts, and managing teams to address challenging client requirements. He has successfully designed and implemented initiatives that have resulted in significant changes in international environmental policy, and led delegations to UN bodies including UNEP, UNCSD, and FAO. He has been a panelist or guest-lectured at numerous venues such as the Smithsonian Conservation Biology Institute and the Woodrow Wilson Center. Mr. Waugh is the author of more than 20 articles or reports on environment and development issues. He is also a longstanding member of the World Commission on Protected Areas and the Species Survival Commission. Imelda “Dada” Bacudo Deputy Team Leader and Land Use and Land Governance Specialist Imelda Bacudo has over 15 years of experience with government agencies and NGOs in developing countries as a project coordinator, strategic planner and evaluator working on the intersections of protected area management, wildlife conservation and sustainable development with several years of specialization on monitoring and evaluation and conservation finance. She holds a Master of Science degree in Environmental Science and Management, from Yale University. Ms. Bacudo is a REDD+ expert and has been working in Indonesia on agriculture and climate change for food security. She excels at partnership and relationship management, engaging with the national and provincial governments and relevant stakeholders to promote and formulate climate policies, and increase capacities to access climate finance. Currently working for the German International Cooperation Agency (GIZ) in partnership with ASEAN, she leads the team on climate smart agriculture value chains, successfully engaging governments from ten ASEAN member states and development partners to contribute to strengthened regional policies and international governance frameworks for greater policy support and wider access to climate finance. Working for UNEP and the Laotian Department of Forestry, she designed monitoring and evaluation tools to be implemented by the government in the areas of climate change, land degradation, and biodiversity. She has conducted evaluation work for DfID, the Uganda Wildlife Authority, the Philippines’ Department of the Environment and the European Union, and is highly skilled in the areas of systems evaluation, climate policy and implementation. With experience in 117 LESTARI MID-TERM EVALUATION USAID.GOV the Philippines, Thailand, Laos, and Uganda, Ms. Bacudo has worked in six countries and has overseen projects that have been implemented in over 10 countries. Ade Cahyat Deputy Team Leader / Institutional and Human Capacity Development Specialist Ade Cahyat is a senior capacity development specialist and team leader with over 19 years of experience leading community capacity building projects on forestry, financial and private sector development, institutional strengthening, and green economic activities. As Team Leader for capacity development under GIZ, Mr. Cahyat’s work focuses on the conceptualization, technology selection, public-private partnership development, and the enabling framework creation for four nationally appropriate climate actions (NAMAs), in line with the country’s development priorities. He has initiated training programs on results-based planning and monitoring, employee information systems, key financial ratios for reviewing budget plans, and transparency and accountability on social funds for multiple GIZ￾led initiatives. Mr. Cahyat led a team supporting the East Kalimantan Province Government on mainstreaming low emission development into its mid-term development plan, Rencana Pembangunan Jangka Menengah Daerah Provinsi Kalimantan Barat Tahun (RPJMD) 2013-2018. The mainstreaming included emission-intensity targets as one of its key performance indicators, supported by capacity building through government programs in relevant sectors (mining and energy, forestry, agriculture), to ensure the achievement of targets. Mr. Cahyat’s experience has taken him all across Indonesia, working in Jakarta and the provinces of East Kalimantan, West Kalimantan, Central Java, South Sumatra, and Papua, Indonesia. Mr. Ade Cahyat, as a result of his extensive work experience, has cultivated relationships with various stakeholders, working closely with and building the capacity of local communities, as well as a number of donors: United States Agency for International Development (USAID), Department for International Development (DFID), Canadian International Development Agency (CIDA), the Danish International Development Agency (DANIDA), and GIZ. He holds a Master of Arts degree in Public Administration from the Australian National University. Ridwan “Duan” Ahmad Biodiversity Specialist Key Qualifications Duan Ahmad is a biodiversity and conservation specialist with over 12 years of experience in conservation and environmental impacts research. He holds a Master of Science degree in Environmental and Natural Resource Management from Institut Pertanian Bogor. He has worked with the Ministry of Environment, Ministry of Energy and Mineral Resources, and the Ministry of Marine Affairs and Fisheries as a junior scientist and has expertise in remote sensing, protected areas management, and primatology and marine mammals’ conservation. Mr. Ahmad has conducted academic research and project implementation work related biodiversity and conservation of Indonesian flora and fauna for the past 10 years designing, conducting, and leading biodiversity and conservation management analysis to include, environmental baseline assessments, environmental impact assessments, environmental sensitivity indices, and environmental monitoring programs. To date, he has been actively involved in over 20 such assessments, to include impact assessments on mangrove forest and wildlife on Lembaat Island, rare and protected species environmental baseline assessment in the Kusan River Ecosystem in South Kalimantan, and impacts from mining on wildlife, vegetation, and vulnerable populations in West Papua. As part of the EBA and EIA work, Mr. Ahmad provides recommendations to clients and to local and/or central government institutions (e.g. the Ministry of Forestry and Environment and Local State Environmental Bodies) on managing biodiversity in general, on how to help maintain relevant rare species, and on human-wildlife conflict management/mitigation connected to the projects undertaken. 118 LESTARI MID-TERM EVALUATION USAID.GOV Amalia Wulansari Gender and Social Inclusion Specialist Amalia Wulansari has over 10 years of experience working on communications, facilitation and training on sustainable development and climate change activities. She holds a Master in Public Administration degree from the University of Diponegoro. She has 5 years of cumulative experience as a gender and social inclusion expert in Indonesia, supporting action research and designing, planning, implementing, evaluation, and up-scaling gender interventions in donor programming. She has conducted policy advocacy on government level in climate change adaptation through community-based adaptation and led training workshops and capacity-building activities at institutional and individual levels to engage watershed stakeholders and community members. Using a systematic approach, for example, to identify risks and local resources, she works with communities to develop strategies for reducing risk in the Garang Watershed of Semarang. Ms. Wulansari excels at developing enabling environments for gender and social inclusion within projects and organizations through the development and design of Social and Gender Integrated Plans. To date, she has helped design six (6) plans and has led on all components of the planning to include developing design surveys, conducting interviews with stakeholders and beneficiaries, facilitation of focus group discussions (in order to get information on gender disaggregated data), and the compiling of the data for recommendation generation and final report writing. Ms. Wulansari works closely and inclusively with communities in rural area and coastal areas; developing community action plans for adaption, focusing on women empowerment for resilience building and providing environmental education. She has expertise in stakeholder’s facilitation for developing climate change adaptation and mitigation strategies, especially for government institution. She has also conducted three (3) gender analyses on the impact of climate change for women in low-income communities. During her time with the BINTARTI Foundation, Ms. Wulansari has been responsible for backstopping M&E interventions for four (4) projects, where she supported the M&E Teams in monitoring project’s outputs achievement. She has also developed indicators for project activities, surveyed beneficiaries, and led needs assessments to explore and prioritize potential adaptive capacity as a Gender and Social Inclusion expert. Karin Merle Private Sector Specialist Karin Merle is a senior project manager and technical adviser in sustainable policy and industrial engagement, focusing on green growth in emerging economies. She holds a Master in International Business and Cultural Studies from the University of Mannheim. She has twelve years of international experience in development finance, sustainable production and consumption, and trade in Indonesia, Thailand, Colombia, South Africa, Malaysia, Spain, and Germany. Her work for GIZ in the ASEAN region included the design of a sustainable consumption and production facility. She co-chairs and international specialist networking and knowledge platform on green finance hosted by the World Bank. Merle has researched market development of “superfoods” in Indonesia for the government of Peru as a component of Peru/Indonesia trade promotion. Merle has extensive experience with trade regulation, including labeling requirements. Her monitoring and evaluation experience includes work in consultancies and research projects, including research to produce a user-friendly handbook on results￾based monitoring. 119 LESTARI MID-TERM EVALUATION USAID.GOV Katrina Makuch Evaluation Specialist Katrina Makuch is a monitoring, evaluation and learning (MEL) professional with 12 years of experience in the design, implementation, and data collection and learning/knowledge management for monitoring and evaluation systems of international development programs. Ms. Makuch’s portfolio of work focuses on monitoring methodologies, design and management of regional and complex evaluations and projects in various sectors, including climate adaptation, biodiversity, agriculture, livelihoods, water, sanitation, and hygiene, gender, rule of law, and infrastructure. She has led the implementation of baseline and endline surveys in culturally diverse and complex environments, managing multiple survey firms, subject matter experts, methodological and sampling experts, statisticians, and enumerators to produce high quality, accurate data and analysis. Ms. Makuch has wide-ranging experience directing development and implementation of M&E strategies and building capacity of technical, non-technical, and local staff to be able to conduct effective, efficient, and accurate evaluations. Ms. Makuch has managed large and regionally diverse teams on over 20 M&E activities, eight (8) evaluations, and has served as an economic growth and livelihoods expert, coordinating results and ensuring the quality of deliverables to project beneficiaries and clients. Her work on the DfiD-funded Climate Resilient Infrastructure Development Facility focused on designing an M&E system to capture performance data at the confluence of conservation, climate adaptation and water infrastructure across 6 SADC countries and their corresponding landscapes. Further, her extensive experience conducting M&E efforts in Democratic Republic of Congo, including USAID’s Central Africa Regional Program for the Environment Evaluation highlights her ability to apply rigorous evaluation design in challenging landscapes to measure project outcomes. To date, she has worked in Afghanistan, Bosnia-Herzegovina, Cambodia, the Democratic Republic of Congo, Djibouti, Haiti, Indonesia, Liberia, Rwanda, Mali, Serbia, Senegal, Somalia, South Africa and Uganda. 120 LESTARI MID-TERM EVALUATION USAID.GOV ANNEX VI: ITINERARY Evaluation Coverage, by Landscape Landscape Locations Team Members Technical Skills Coverage Dates Sumatra Leuser Tapaktuan - Meulaboh - Gayo Lues - Blang Pidie - Banda Aceh Waugh, Bacudo, Merle, Wulansari HIDC, Governance, Private Sector, Green Finance, Evaluation, and Gender Nov 9 – 19 Kalimantan Katingan – Kahayan Palangka Raya - TN Gunung Mas - Pulang Pisau Bacudo, Merle, Wulansari HIDC, Governance, Private Sector, Green Finance, Evaluation, and Gender Nov 20 – Dec 1 Papua16 Cyclops Jayapura – Timika – Agats – Merauke Cahyat, Ahmad, Makuch Evaluation, Private Sector, Land Governance, Gender, Biodiversity Nov 9 – 13 Lorentz Nov 14 – 23 Mappi – Bouven Digoel Nov 24 – Dec 1 16 Per discussion with USAID Indonesia, in the interest of time and value added, the evaluation team will only travel to 3 of the 4 landscapes on Papua 121 LESTARI MID-TERM EVALUATION USAID.GOV