EVALUATION Lewa Wildlife Conservancy – Northern Rangelands Trust Support Program (Phase II): Final Performance Evaluation USAID/Kenya and East Africa Office of Environment and Natural Resources March 2017 This publication was produced at the request of the United States Agency for International Development (USAID). It was prepared independently by Mark Renzi, James Ndung’u, Everlyne Amile and Management Systems International. NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 2 Northern Rangelands Trust: Final Performance Evaluation USAID/Kenya and East Africa Environment Office March 2017 Contracted under Task Order # AID-623-TO-16-00014 Final Performance Evaluation for USAID/NRT: Award Number: AID-623-A-00-09-00011 Cover Photo Lewa Wildlife Conservancy is home to various wildlife species, including the impala. (Photo: Paul Temple/MSI) DISCLAIMER This study is made possible by the support of the American people through USAID. The contents are the sole responsibility of Management Systems International and do not necessarily reflect the views of USAID or the United States Government. NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION ii CONTENTS Acronyms ...........................................................................................................................iii Executive Summary .......................................................................................................... 4 Project Background ............................................................................................................................. 4 Evaluation Purpose and Evaluation Questions .............................................................................. 4 Activity Background.......................................................................................................... 8 Development Problem ....................................................................................................................... 8 USAID Activity Strategy and Tasks ...............................................................................................10 Evaluation Methods and Limitations ............................................................................. 10 Data Collection Methods and Sources .........................................................................................11 Findings and Conclusions ................................................................................................ 14 Evaluation Question 1 ......................................................................................................................14 Evaluation Question 2 ......................................................................................................................22 Evaluation Question 3 ......................................................................................................................27 Recommendations .......................................................................................................... 35 Annexes ............................................................................................................................ 38 Annex I: Evaluation Statement of Work ......................................................................................38 Annex II: Strategic Insights ..............................................................................................................60 Annex III: Data Collection Instruments .......................................................................................64 Annex IV: Sources of Information ................................................................................................. 69 Annex V: List of Interviewees .........................................................................................................71 Annex VI: Disclosure of Any Conflicts of Interest ....................................................................73 NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION iii ACRONYMS AFD French Development Agency AGM Annual General Meeting CBO Community-Based Organization DANIDA Danish International Development Agency ENRM Environment and Natural Resource Management GOK Government of Kenya KFS Kenya Forest Service KPR Kenya Police Reserve / Kenya Police Reservist KWS Kenya Wildlife Service NGO Non-Governmental Organization NRT Northern Rangelands Trust NRT/T NRT Trading SOW Statement of Work USAID/KEA United States Agency for International Development/Kenya and Eastern Africa Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 4 EXECUTIVE SUMMARY This is an ex-post performance evaluation of the Lewa Wildlife Conservancy – Northern Rangelands Trust (NRT) Support Program Phase II Project (AID-623-A-00-09-00011, $6,991,386.) As stated in the evaluation statement of work (SOW, a copy of which is included in Annex I), “The evaluation purpose is to draw lessons to inform future program design and improve implementation of community-based [national resources management] activities.” A summary of the evaluation questions follows: 1) To what extent and how has the NRT model improved conservation? 2) What were the perceptions of men, women and youth regarding successes and challenges of peace dividends, economic opportunities, improved rangelands, women’s empowerment and youth empowerment and what differential approaches were targeted to men, women and youth? 3) How has the perception and planning around NRT’s sustainability changed over time and what specific factors influenced the current sustainability plan? Project Background The activity can be considered Phase II of a 16-year USAID investment in NRT. Phase I ran from 2004 to 2008, Phase II (the subject of this evaluation) was from 2008–2015 and the follow-on Phase III began in 2015 and continues to 2020. The loss of biological diversity has been cited as a global environmental threat to natural ecosystems and iconic wildlife species, and efforts are increasing to engage communities around the Ewaso ecosystem in conservation through incentive-based approaches. The United States Agency for International Development/Kenya and East Africa (USAID/KEA) Mission launched the second-phase grant to the Northern Rangelands Trust to enable NRT to scale up interventions focused on biodiversity conservation, NRM and livelihood improvement, and to strengthen peace-building and community-based organization (CBO) in its constituent areas. NRT’s community development model is driven by communities’ needs and aspirations with a focus on three key target areas: wildlife and environment management, improving and diversifying income and providing security. The activity is implemented across nine counties, and its central purpose is to improve the livelihoods of conservancy communities through wildlife conservation. USAID, one of the principal donors, funds core operations, infrastructure and capacity-building activities, while NRT maintains partnerships with a large portfolio of other benefactors. Through USAID support, NRT has expanded its constituent conservancy membership from nine conservancies in 2005 to 33 conservancies in 2015, and has promoted more peaceful coexistence among livestock, wildlife and people. Evaluation Purpose and Evaluation Questions Evaluation Purpose The evaluation purpose is to draw lessons to inform future program design and improve implementation of community-based NRM activities. Overall, the evaluation must assist USAID/KEA’s Environment and Natural Resource Management Office to understand what aspects of the activity were most effective in Phase II that could be scaled up and replicated in future programming, and how this could be done. Further, the evaluation seeks to understand the challenges encountered in implementation and what could be done in the future to address them. The evaluation must generate important learning to inform Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 5 Agency and Mission management about sustainability, peacebuilding, governance, gender and enterprise development within the community conservancy model. The key audience for this evaluation is the USAID/ KEA. They will use it to inform future programming. Another key audience is the current project’s technical team, who will use the evaluation to inform the design of possible new interventions and/or revise or modify their current work plan. The USAID/Washington Office of Forestry and Biodiversity within the Bureau of Economic Growth, Education and Environment may also be interested in the broader lessons learned from this evaluation. USAID will share the final report with NRT and its target communities, other donors/funding partners, the Government of Kenya (GOK) and county governments, the Kenya Wildlife Service (KWS) and the Kenya Forest Service (KFS). This audience will likely use the evaluation results to inform future planning for biodiversity conservation. Evaluation Questions Evaluation Question 1 To what extent and how has the NRT model improved conservation? In answering this question, the evaluation team must examine: 1) How NRT’s anti-poaching communications system reduced the potential for crime and threats against wildlife populations within NRT conservancies. This includes any pre-defined systems as well as those that were developed and/or evolved over the life of the activity. 2) The effectiveness of NRT’s community ranger network and how this network has contributed to improved surveillance and wildlife protection, reduced banditry and livestock theft, and greater conservation awareness within NRT conservancies and constituent communities. 3) How NRT has improved conditions and diversity of biological resources over the life of the project and the factors that facilitated and/or hindered improved wildlife and rangeland health. This includes how well constituent communities have adopted biodiversity conservation practices as a result of the activity. 4) How is NRT's community conservancy governance model perceived by the community and stakeholders to have contributed to conservation, peace and livelihoods. 5) Any other systems or factors within the NRT model that led to improved conservation across NRT conservancies. Evaluation Question 2 1) Considering separately the perceptions of men, women, and youth (male and female), which of the following aspects of the NRT model were considered successful by communities, and in what ways were they considered successful? Why? a) Peace dividends b) Economic opportunities c) Improved rangelands d) Women’s empowerment e) Youth empowerment 2) Considering the same program areas (a-e, immediately above), describe whether the program applied differential approaches to engage men, women and youth, and why. What were the effects of such inclusive efforts? In so doing, to what extent did NRT take advantage of emerging opportunities for the inclusion of men, women and youth? Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 6 Evaluation Question 3 How has the perception and planning around the sustainability of NRT and its member conservancies changed over time and what specific factors influenced the current sustainability plan? The answer to this question, the contractor must focus specifically on the effectiveness of NRT’s original sustainability plans and how they changed over the life of the activity to include the evolving role of county governments. The question must also explore the target communities’ perceptions about the extent to which and in what ways the activity’s effects can be sustained following its closing. In answering this question, the evaluation team must examine three things in particular: 1) Strategies for increasing revenue from the county governments 2) Strategies for increasing commercial revenues (e.g., scaling up beadwork, ecotourism, livestock-to￾market programs, microenterprise, etc.) 3) Strategies for increasing capacity to access donor funding Evaluation Methods and Limitations Data collection took place in November 2016 across a sample of five member conservancies (Leparua, Nakuprat-Gotu, Namunyak, Kalama, Jaldesa) in four counties: Laikipia, Isiolo, Samburu and Marsabit. Conservancy sampling purposively targeted a diverse representation of community-owned conservancies within the four evaluation counties. Sampling took into account key criteria, including age of conservancy, ownership status, operating budget, ethnic composition, size, governance and performance, and flagship wildlife species. The evaluation utilized qualitative data collection and analysis methods that were most appropriate for the acquisition of rich detail and triangulation across perspectives, trends and themes, and further relied on a variety of secondary sources including project documents. A total of 56 interviews took place, with representatives from USAID, other NRT donors, government, working partners and conservancy constituents (elders, youth, men, women, boards, management committees and rangers) included. A few limitations were noted, in particular selection bias, as informants were largely selected and provided to the evaluation team by USAID/KEA and NRT. Use of various (primary and secondary) data sources triangulated the findings in an attempt to minimize the effects of these biases. Language barriers were also experienced in the process of data collection, though the use of translators mitigated this. While the evaluation provides useful findings for understanding NRT’s performance in certain conservancies, it offers limited applicability outside the areas visited with respect to peace, culture and resources that are highly context-specific. Finally, due to the volume of information the evaluation sub￾questions generated and the limitations of time and space, the depth in analysis and recommendations is also restricted. Findings and Conclusions In the global context where success in community-based natural resource management projects is very difficult, NRT’s work, is inspiring. In a short time, NRT has used conservancies to help communities in large and important swaths of Kenya transform their lives by alleviating insecurity from once￾ungoverned spaces. With this foothold, NRT has helped communities reinvigorate traditional natural resource management systems, resulting in increased resiliency and further reinforcing security gains. People can travel in former no-go zones, manage conflict over resources and work with former adversaries and better survive environmental shocks. These human, political and natural resource management changes have created conditions for the return of wildlife that counter international trends, create a possibility for tourism revenue, and have fostered Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 7 what appears to be community-wide commitment to conservation-driven development. NRT has patiently ensured that the process is community-driven and is fully supported by counties and the GOK. It is an appropriate model for the North (the area studied by the evaluation.) The model has potential for sustainability if NRT expands its existing, already sound sustainability plan to include attracting community and GOK contributions in addition to its current emphasis on county, donor and commercial sources. On the commercial side, NRT/Trading has made important progress innovating its beads and cattle business lines, although massive expansion of both, along the lines detailed in the report, is key to supporting sustainability. The tourism piece of the NRT/Trading effort requires immediate attention with respect to transparency, systematic support, and revenue collection to get back on track. NRT has successfully included youth in its programs on economic benefits and some other programs, although the trust could make slightly more progress in engaging youth in decision forums. While cultural realities make it challenging to progress on gender issues in the North, NRT needs to invest more effort in engaging women fully in its programming, as this report suggests. NRT is also a victim of its own success. It experienced a 50 percent year-on-year budget increase and increased coverage from five to 33 conservancies, and estimates that community demand and Council of Elders’ endorsement will result in NRT likely supporting a total of 50 conservancies in the next four years. Such spectacular growth has made NRT and conservancies relevant, but it has also put strains on NRT’s ability to support conservancies. Underinvestment in conservancies is evident in all aspects of programming. This puts hopes for sustainability on hold and could lead to loss of some of the challenged conservancies. The central institutional dilemma facing NRT as it moves forward is how to adequately service existing conservancies, using a nuanced approach that recognizes the wide range of conservancy needs, as balanced against its institutional commitment to meet community demands to get more important land under conservancy management. At the conservancy level, the pressing need is to empower conservancy boards and staff to succeed on their own. NRT’s bold beginning, dedicated efforts and huge successes bode well for the organization’s ability to meet these challenges. The evaluation team was repeatedly impressed with the openness of NRT and its willingness to learn. This combination places NRT on an excellent path to help the human and natural communities to which it is fully dedicated. Recommendations The evaluation generated the following highlighted recommendations: 1) NRT/conservancies should consider improving the rangers’ work conditions, including pay, welfare and appropriate equipment (weapons, camping gear, uniforms), consistent with sustainability. This will not only boost their morale, but also enhance their effectiveness. 2) USAID/NRT, in collaboration with relevant partners, should develop capacity for rigorous analysis of biodiversity, rangeland, water and forest resources. 3) NRT should invest more heavily in conservancy board strengthening to address limited funding and human and capacity constraints as an urgent aspect of sustainability through completion of management plans and training of conservancy boards. Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 8 4) NRT/conservancies should enhance meaningful board accountability mechanisms to the community by improving communication between the two. Efforts could include fully funding an expanded array of both the annual general meetings (AGM) and other meetings, in addition to other efforts. 5) NRT should increase the participation of women and youth with grazing committees and in decision-making roles through initiating buy-in from traditional elders, enshrining fixed levels of their participation in the conservancy constitution and increasing the number of female NRT staff who interact with conservancies. 6) To ensure sustainability, NRT/conservancies should continue the county fundraising strategy as currently implemented, expand to seek GOK funding, introduce donors to conservancies directly (as is the case of the new Namunyak conservancy elephant orphanage site), in addition to including community contributions. 7) NRT/Trading (NRT/T) should immediately ensure that all conservancies fully monitor and enforce tourism agreements, including community, NRT and investor obligations; revenue flows; and income reporting. 8) At the strategic corporate level, NRT should resolve the tension between allocating resources to support existing conservancies and continuing to channel resources to expand to new conservancies. 9) For management and donor relations purposes, NRT may want to track precise implementation activities (and challenges) in each conservancy and maintain this information on a database that is easy to access and query. Activity Background A brief background of the activity follows. The full activity background is in the evaluation statement of work (SOW), Annex I of this report. Development Problem The loss of biological diversity has been cited as a global environmental threat to natural ecosystems and iconic wildlife species.1 In northern Kenya, communities struggle with frequent droughts, poor healthcare, sparse government services and threats posed by cattle rustling and ivory poaching. Ethnic rivalries also continue to threaten peace and development.2 This area, known as the Ewaso Nyiro ecosystem, contains two large-scale processes that have been selected by development partners as long￾term targets for conservation: dry-season river flow and elephant migration.3 This diverse region includes the entire Laikipia and parts of adjacent Samburu, Meru, Isiolo and Marsabit counties. To mitigate biodiversity loss, Kenya’s national government and its development partners have created linkages between biodiversity conservation, sound natural resource management (NRM) and sustainable economic growth.4 The study “Conserving Wildlife in Kenya” by Nicholas Georgiadis indicates that as far back as the 1980s, the concept of conservation was limited to protecting wildlife in particular areas, but the paradigm began to shift at the end of that decade toward recognizing humans as an integral component of biodiverse landscapes. Over time, national parks and game reserves became too restrictive, especially for the long-term viability of wildlife. Hence there is a need to open up spaces for habitat and biodiversity conservation in areas that were not formally protected, most of which were communally owned or used. In addition, 1 USAID Kenya Country Development Cooperation Strategy 2014 – 2018 2 The Story of The Northern Rangelands Trust (NRT, 2013) 3 Georgiadis, Nicholas J., editor. “Conserving Wildlife in African Landscapes: Kenya’s Ewaso Ecosystem.” Smithsonian Contributions to Zoology, number 632, 2011. 4 USAID Kenya Country Development Cooperation Strategy 2014 – 2018 Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 9 conservation efforts need to account for human livelihood needs as well as protect wildlife and land among other natural resources. The study further found that in the 1990s, the concept of promoting wildlife as a source of income was initiated among ranchers (particularly in Laikipia), which led to the creation of management associations under the stewardship of the Kenya Wildlife Service (KWS). Laikipia Wildlife Forum was launched in 1991 as a company of private and communal landholders with a shared interest in wildlife management and conservation. By the early 2000s there was a proliferation of non-consumptive wildlife use including ecotourism enterprises on private and communal properties in Laikipia and the greater Ewaso ecosystem. However, human-wildlife conflict persisted from the pressure of opening space for wildlife due to expanding human settlement, land subdivision, the spread of incompatible land uses such as cultivation, woodland clearing, bush meat consumption, and the proliferation of firearms.5 The results of these pressures included crop raiding by wildlife, attacks and in some instances death of both humans and wildlife. As a result, in the late 1990s, a variety of conservation initiatives in northern Kenya sought to mitigate human-wildlife conflict, including community-based conservation efforts. USAID and Kenya Wildlife Services (KWS) played pioneering roles in these efforts, and USAID followed with support to Lewa Wildlife Conservancy (LWC), initially for the development of Sera Wildlife Conservancy in 2004 and later to support the development of the Northern Rangelands Trust (NRT) in 2005. Community-based conservancies aimed to create incentives for landholders and land users to conserve wildlife and natural resources. These incentive-based approaches, where economic rationale provided the impetus for changing conservation strategies with landholders and land users, focused on realizing the market value of wildlife conservation via ecotourism, trophy hunting and live animal sales. Community conservancies produced mixed results, including positive development and persistence of the initial problems. Kenya’s Vision 2030 development strategy includes conservation and sustainable management of the country’s natural capital as an important strategy for the country’s economic growth, as well as attaining a clean, secure and sustainable environment.6 Toward this end, USAID promotes long-term community resource management by helping Kenyans who live around Kenya’s national forests and parks establish nature-based enterprises that preserve, rather than deplete natural resources.7 It does this through various mechanisms, including: the Laikipia Biodiversity Conversation program, support to the Lewa Wildlife Conservancy – Northern Rangelands Trust (NRT) Support Program ,working with the International Development Law Organization to support policy formulation, implementation and monitoring as a contribution to biodiversity conversation, support to The Nature Conservancy (TNC) through the Kenya Wildlife Conservation Association (KWCA), the Low Emission and Climate Resilient Development project, the Climate Resilient Water Towers activity and others focused on wildlife protection and climate resiliency among other areas. Activities funded within the USAID’s Environment and Natural Resource Management (ENRM) portfolio seek to develop incentives and effective structures whereby communities and government entities can conserve forests, biodiversity and manage climate change.8. Support for NRT involves implementing community grazing plans and rehabilitation of degraded areas across the northern rangelands.9 5 Georgiadis, Nicholas J., editor. “Conserving Wildlife in African Landscapes: Kenya’s Ewaso Ecosystem.” Smithsonian Contributions to Zoology, number 632, 2011. 6 USAID KEA Environment and Naturals Resources Management (Kenya) – Factsheet 7 Ibid 8 USAID Kenya Country Development Cooperation Strategy 2014 – 2018 9 USAID KEA Environment and Natural Resources Management (Kenya) – Factsheet Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 10 USAID Activity Strategy and Tasks The United States Agency for International Development’s Kenya and East Africa Mission (USAID/KEA) launched the second phase grant (2008-2015) to the Lewa Wildlife Conservancy – Northern Rangelands Trust (NRT) Support Program(Award Number: AID-623-A-00-09-00011) with the purpose of enabling NRT scale-up interventions focused on biodiversity conservation, NRM, livelihood improvement, peacebuilding and community-based organization (CBO) strengthening in its constituent areas. NRT also has partnerships with a large portfolio of donors including The Nature Conservancy (TNC), the French Development Agency (AFD), the Danish International Development Agency (DANIDA) and other international non-governmental organizations (NGOs) and foundations, as well as with other stakeholders such as the Kenya Forest Service (KFS), Kenya Wildlife Service (KWS), Kenya Police and other national and county government partners. Target Areas and Groups NRT’s community development model is driven by communities’ needs and aspirations with a focus on three key target areas: peace, livelihoods and conservation. The activity is implemented across nine counties (Samburu, Isiolo, Lamu, Marsabit, Laikipia, Tana River, Meru, Garissa and Baringo), supporting 27 conservancies as of 2015. The full details of NRT conservancies are in the SOW (Annex I). Intended Results To achieve the goal of improving the livelihoods of conservancy communities through wildlife conservation in the rangelands of northern Kenya, NRT focused on achieving the following results: 1) improved management systems and institutional structures of community-based organizations managing conservancies, 2) improved condition and diversity of biological resources, 3) increased economic benefits and improved livelihoods for northern rangelands pastoralist households, 4) increased community capacity to resolve resource-based conflicts and improve natural resource governance in the larger NRT region and 5) enhanced sustainability of the NRT executive structure and operations. Objectives NRT’s central purpose is to improve the livelihoods of conservancy communities through wildlife conservation in the rangelands of northern Kenya. As such it is concerned with biodiversity conservation, with USAID, one of its principal donors, funding core operations, infrastructure and capacity building in conservancies. Through USAID support, NRT has rapidly expanded its constituent conservancy membership from nine conservancies in 2005 to 27 conservancies in 2015. This model continues to support the creation of new conservancies in response to increased demand from surrounding communities, environmental degradation, and the prevalent threat of poaching, increasing the area of community-owned conservancies in northern Kenya. Further, it seeks to maintain the values of its member conservancies, their vision for the coexistence of livestock, wildlife and people, and peaceful coexistence of communities. NRT uses wildlife conservation as a vehicle for development among the politically and socially marginalized pastoralist communities of northern Kenya. EVALUATION METHODS AND LIMITATIONS The final performance evaluation of Lewa Wildlife Conservancy – Northern Rangelands Trust (NRT) Support Program Phase II sought to understand the extent of the NRT model’s improvement to conservation, the key factors that contributed to successful implementation and the perceptions around Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 11 sustainability. Annex II fully describes the methodology and Annex III contains data collection instruments. The evaluation team consisted of team leader Mark Renzi and local experts James Ndung’u and Everlyne Amile. USAID and NRT assisted the team with identifying and contacting individuals for interviews, as well as compiling a complete list of NRT documents for review. The evaluation team collected data from November 2-26, 2016, in five member conservancies (Leparua, Nakuprat-Gotu, Namunyak, Kalama and Jaldesa), across four counties (Laikipia, Isiolo, Samburu and Marsabit). Conservancy sampling purposively targeted a diverse representation of community-owned conservancies within the four evaluation counties. Sampling took into account key criteria including age of conservancy, ownership status, operating budget, ethnic composition, size, governance and performance, and flagship wildlife species. Based on consultations with NRT during the early phase of fieldwork, the team replaced Lekurruki and Mpus Kutuku conservancies with Kalama and Nakuprat￾Gotu, respectively, to increase the utility of findings.10 Given the nature of the evaluation questions and the stakeholders involved, this evaluation used qualitative data collection and analysis methods. Qualitative methodologies are the most appropriate because they facilitate acquisition of rich detail regarding how and why things did or did not happen and triangulation across perspectives to assess emerging trends and themes. The team’s approach allowed exploration of the processes behind the results, which is what interested USAID in this evaluation. Data Collection Methods and Sources The evaluation SOW includes detailed description of data collection, analysis and sampling methods by evaluation question. While the evaluation team refined this methodology, it did not make major changes. A summary of methods used and data sources follows. The specific collection methods used include secondary data review, key informant interviews (KIIs) and group interviews (GIs). For the desk review, the evaluation team reviewed all activity documentation provided by USAID and NRT, as well as additional resources obtained online from stakeholders. Secondary documents reviewed include the following: activity award and modifications; activity descriptions, work plans, monitoring and evaluation (M&E) plans, performance reviews, strategic plans, conservancy management plans, baselines and assessments, mid-term evaluation and periodic progress reports. Annex IV contains a list of the key documents reviewed. Findings from the initial document review identified information needed to completely answer each evaluation question and guide discussions about sources for that information. Those discussions helped shape both the list of individuals to be interviewed and the semi-structured interview guides. To identify interviewees, the team began with the list of organizations and titles included in the SOW and consulted with both the USAID officers commissioning the evaluation and NRT to identify, locate and schedule KIIs and GIs. During the three weeks of data collection, the team conducted 21 KIIs and 35 GIs with 260 total respondents. The sex and affiliation of respondents is summarized below; in addition, a full list of interviewees is in Annex V. 10 Lekurruki Conservancy was replaced with Kalama because NRT indicated that Kalama demonstrated potential for solar power generation and because NRT had successfully resolved some governance issues. NRT further advised that Nakuprat-Gotu would make a better target than Mpus Kutuk because it represented greater ethnic diversity, a more sedentary population and is located in the Lamu Port-South Sudan-Ethiopia￾Transport (LAPSSET) corridor. Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 12 TABLE 1: SUMMARY OF INTERVIEWS Affiliation GIs KIIs Male Female Total Respondents USAID 1 0 1 1 2 NRT 2 3 10 1 11 Development partners 0 5 0 5 5 Working partners 1 13 14 1 15 Board members 5 0 30 5 35 Elders 4 0 30 0 30 Management committees 5 0 17 0 17 Grazing committees 2 0 10 0 10 Rangers 5 0 41 2 43 Men only groups 1 0 13 0 13 Women only groups 5 0 0 46 46 Youth only groups 4 0 27 6 33 Total 35 21 193 67 260 Data Analysis Methods Data analysis methods included content analysis, comparison and mixed-methods data integration. The evaluation team integrated information from the desk review, KIIs and GIs by evaluation question to identify common trends and themes related to NRT’s performance, perceptions of the activity’s success, as well as feedback on how to strategically improve the NRT model. Preliminary data analysis began during the fieldwork to ensure that the team was capturing the information necessary to fully address the evaluation questions. During data collection, the team held weekly debriefings to identify common themes among interviews by evaluation question. The team refined these themes as it progressed through data collection. After completing data collection, the team reviewed all interview notes to finalize the list of key themes, which it used to code each set of interview notes. With coding completed, the team analyzed the interview data using content analysis and pattern analysis in gathering and comparing the perspectives of various stakeholder groups to answer the evaluation questions. A tally sheet was used to conduct content and pattern analysis of key themes. The tally sheet captures the frequency of key themes and underlying evidence from the interview notes. Limitations As with any assessment or evaluation, the team experienced a few limitations that it addressed through various methodological approaches. Selection bias existed in the form of contacts provided by USAID/KEA and NRT, which meant the team likely heard from respondents who had direct interaction with NRT and were more likely to have had positive experiences. Use of various (primary and secondary) data sources triangulated the findings in an attempt to minimize the effects of this bias. Two respondents were unavailable for their interviews: representatives from the Ministry of Environment and Natural Resources and the NRT peace coordinator. Language barriers also occurred during data collection, though the use of translators mitigated this challenge. The most effective approach for combating any and all of these biases was using multiple layers of triangulation. By combining information found in documents and interviews from multiple sources, any Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 13 single piece of biased data did not unduly skew the analysis. The use of different data collection methodologies (group and individual interviews) and researcher triangulation to ensure the reliability and validity of results supported this. More generally, while the evaluation provided useful findings for understanding NRT’s performance in certain conservancies, it provides limited applicability outside the areas visited with respect to peace, culture and resources, which are highly context-specific. In addition, because of the volume of evaluation sub-questions, the ability to provide depth in analysis and recommendations for so many multi-dimensional evaluations — given the limitations of time and space — was also restricted. Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 14 FINDINGS AND CONCLUSIONS The findings and conclusions are organized by the three evaluation questions. Evaluation Question 1 To what extent and how has the NRT model improved conservation? The findings and conclusions are organized by the five sub-questions referenced in the evaluation purpose and questions section (see p. 1). While this section focuses on specific components of NRT’s model, the evaluation recognizes the fundamental role that Lewa Conservancy plays in the success of the overall model, particularly with regards to fundraising, security, governance, and infrastructure. NRT would not exist without the ongoing support from and collaboration with Lewa. 1.1. NRT’s Anti-Poaching Communication System Findings NRT’s anti-poaching communications system was designed to address crime and threats against wildlife populations within NRT conservancies. NRT supplied hand-held radios (often with a base radio and radios for vehicles) to partner conservancies, which could communicate within the conservancies and to NRT headquarters in Lewa. Where operational, radios are highly valued by those who rely on them for law enforcement, as reported by four of five rangers’ 11 groups, four of five community management committees and four of five conservancy boards. They reported that radios improved conservancy ability to identify and communicate security threats; respond to rustling, poaching and banditry; and maintain ranger security. KWS and police informants also valued radios for law enforcement (five of seven interviews.) Several conservancies reported that radios were very useful for alerting neighboring conservancies when morans12 are en route to steal cattle and to track and recover stolen cattle across conservancies. Many community groups cited examples of how emergencies (e.g., childbirth, snake bite) had been effectively addressed by using the radio network to gain access to the conservancy vehicle to get people to health care facilities. Nevertheless, most conservancy management committees (three of five), conservancy boards (three of five) and rangers (four of five groups) thought current radio equipment was inadequate. In all but one conservancy, respondents generally agreed that more radios were needed. Even with the available stock (all five conservancies visited had radios), functionality was a challenge. One conservancy had only three functioning handheld units and no vehicle radios and two other conservancies completely lacked radio service. Cases of acute dysfunctionality were reported to be due to units not being repaired and to lack of functioning repeater networks. All communities visited were looking forward to receiving new digital radios (funded under Phase III NRT funding plan) for increased functionality (GPS, for example) and 11 NRT does not employ community conservancy rangers. These rangers are employed and managed by the conservancies, while NRT provides oversight, training, support, government liaison and other guidelines and standards. This report focuses exclusively on these non￾specialized community rangers. NRT does have special 9.1 rapid response ranger teams, but members of this force were not interviewed for this evaluation. 12 Morans are young, unmarried warriors in the Maasai and Samburu ethnic groups who are traditionally involved with pastoralism. Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 15 because they felt the upgraded equipment would improve range, transmission quality and reliability. Conservancies expressed frustration at delivery delays of the new digital system from NRT.13 Rangers lacking radio functionality relied on mobile phones for communication, a solution they considered sub-optimal, since phone network coverage dead spots remain in their areas. In all conservancies visited, residents at group interviews had mobile phones. They reported that they were able to access network, although sometimes to make calls they would have to walk to specific spots known to be on the grid, such as on top of a hill. Conclusions Rapid, reliable communication is necessary for security, conservation and community emergencies in the large areas covered in conservancies, particularly in coordinating efforts among conservancies. Maintenance challenges and a lack of supporting infrastructure reduced the utility of radios purchased in Phase II to reduce the potential for crime and threats against wildlife. Given that many conservancies now have considerable network coverage, it may be useful to complement the digital radios that are on order with other devices — such as smartphones and tablets — to be able to also send non-voice data such as photos and monitoring data, and possibly to track tagged animals. 1.2. Effectiveness of NRT’s Community Ranger Network Findings Rangers received universal praise. Group interviews with the conservancy management committees and boards revealed consistent agreement that rangers were effective, a perception that most KWS and police respondents shared. In group interviews with elders and members of boards and committees, respondents observed that the rangers were more effective than formal law enforcement agencies in areas where rangers operate.14 Informants felt that rangers had halted warring among pastoral groups, stopped poaching and greatly reduced banditry. They had been able to reduce cattle-rustling in most cases and recover animals that were stolen where cattle theft continued. Reasons cited for their effectiveness were: well-maintained and well-financed vehicles, understanding of local culture (and familiarity with individuals involved in nefarious deeds, with some of the rangers being reformed poachers), local language skills, personal commitment and knowledge of local terrain and exit routes. KWS and police stated that NRT effectively expands the reach of law enforcement and greatly appreciate the vehicles, communication, staff and other resources that NRT contributes. As impressive as these accomplishments were, respondents also emphasized what rangers did not do: Despite working with the government for the Kenya Police Reserve (KPR) to provide arms to young men from traditionally warring ethnic groups, there were no reports of abuse of power or authority among a ranger force with little or no previous law enforcement experience. That is an impressive testimony to the discipline and commitment instilled in ranger forces. The main risk facing this competent force is adequate NRT investment. Apart from the shortfall in operational radios noted above, all seven conservancy elders and committees interviewed felt the ranger force was underequipped. The universal feeling was that rangers were too few, significantly underpaid and underequipped (with specified shortfalls relating to uniforms, camping gear, weapons, ammunition and housing). In some cases, community members felt rangers were underinsured. This perception was shared by three of five conservancy boards, three of five conservancy management committees and three of five youth groups. It was slightly less of an issue for women (two of five groups.) Indeed, the pay (approximately $100-$120/month), while above the Kenyan minimum wage, did appear to the evaluation 13 NRT claims the delay is due to issues related to value added tax (VAT) exemption procedures. 14 This question was not part of the interview guide, but this view was consistently volunteered by these groups. Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 16 team to be low. NRT also noted that, if possible, it would like to at least double the pay to reflect the rangers’ contribution and the danger and hardships these men and women face. Group discussions revealed no formal agreement encompassing KWS, police, NRT and conservancies that clearly outlined relationships during an engagement, or arresting and reporting protocols.15 This already led to at least one case where a ranger was exposed to liability for an operation conducted in this grey area, guided by good faith and loose agreements.16 Police and KWS note that some rangers are KPR members; they are armed, have clear police status, fit into the hierarchy and can make arrests. Others lack police or KWS status and are not armed. There were multiple accounts of difficulties in communication and respect of authority between the conservancy civilian (Board and Conservancy Manager) and uniformed ranks. Civilians sometimes do not understand the requirements of ranger work or communicate in ways that respect its hierarchy. Also, the ranger ranks are inconsistent — for example, a KPR who has little formal training (but carries a gun) may be senior to staff trained at the KWS law enforcement academy. This is reported to cause dissent. Conclusions The work of the rangers and NRT is extraordinary: Huge tracts of previously lawless land are now under control, citizens feel safe to travel freely, livestock can graze outside their traditional areas and wildlife is returning as a result of the halt to poaching. NRT has exerted a deft and firm hand to make the ambitious, high-risk effort a success. It is the jewel of the program, regionally and locally. But years of underinvestment in equipping and paying rangers is likely to impede their effectiveness. More troubling, such low investment could reduce morale to the point where motivated staff become disillusioned and susceptible to compromise by poachers and other illicit actors. Since peace and security is at the heart of NRT’s success, this would be troubling. Finally, the lack of formal agreements among NRT, conservancies (there are already existing agreements between NRT and conservancies), police and KWS present risks to rangers and NRT. 1.3 Improved Biodiversity Practices and Factors Findings Biological changes: The NRT model posits that such dramatic improvements in attitudes and practices should lead to improvements in biodiversity and rangelands health. The midterm evaluation of Phase I recommended that NRT improve its monitoring systems to be able to monitor such biophysical 15 Information from respondents representing KWS and Police suggested that there are no written agreements between Police and NRT, or between KWS and NRT. However, there are operational agreements and the three parties collaborate frequently. 16 NRT claims it is advancing on this complex issue by negotiating with the GOK Police for an NRT liaison officer so it can improve understanding, chain of command and clarity of roles between conservancy rangers and the police. Community rangers’ quarters, Namunyak conservancy, Samburu County. Mark Renzi/MSI Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 17 changes. While NRT has made some efforts in that regard and reports on plans to work with other groups to be able to develop that capacity, NRT could not provide the evaluation team with scientific, time-series data. In the absence of such data, the evaluation team can only note that community groups consistently perceive such improvements in wildlife numbers and diversity.17 By far, the most frequently￾cited species increases were non-grazing wildlife, including elephant, giraffe, lions, ostrich, leopards and even gerenuk. Grazers (Burchell's zebra, Grevy’s zebra and various antelope species) were less frequently cited. To understand improvements in biodiversity, in terms of the variety of wildlife species, groups were asked if they had noted the return of certain wildlife species — those that had disappeared for years — since conservancies were established. Return of wildlife was noted by most conservancy management committees (four of five), rangers (four of five), youth (five of six), women (four of five) and six of seven elders and grazing committees, and by two of five boards consulted. Significant changes in rangelands quality were not consistently noted. Attitudes and practices: Almost universally, interviewees reported that community attitudes toward conservation have improved since conservancies began.18 Poaching was reported to have almost disappeared in the conservancies visited, except for Namunyak, which had recently experienced significant levels of elephant poaching. Similarly, almost all groups interviewed reported that systems to manage grazing in wet and dry seasons have been adopted and are largely observed.19 But conservancies also noted a lack of funding to fully develop and implement rangeland plans, to hold meetings of grazing committees and to enforce the rules. Progress in systematically managing water and wood product resources was reported in some conservancies but not in others. The evaluation team observed the early effects of rangeland restoration efforts in one conservancy and noted the dramatic response of the land to these efforts to remove invasive species and combine reseeding with cattle bomas and bunching20 techniques. This was a pilot effort over a small area. NRT reports that similar pilot efforts are underway in five other conservancies. 17 Groups expressing that opinion include all five conservancy management committees, all five boards, five of seven elders and grazing committees, five of five women, five of six youth and four of five rangers. 18 The Social-COMMS Attitude Survey and the State of Conservancies Report 2015 reflect similar results. 19 This view was expressed by four of five boards, four of five conservancy management committees, all seven elders and grazing committee members, four of five rangers, four of five youth three of five and women. 20 The cattle boma technique involves the establishment of a temporary cow shed within an eroded area, where cattle are able to break the dense hardpan soil layer, allowing water to percolate, improving absorption, enhancing soil fertility with manure, and aiding in the overall stabilization of soils regeneration of grass and vegetation. Similarly, the bunching technique involves bringing together herds of cattle in one grazing block for the same purpose of improving rangelands. Buffalo herd in Nakuprat-Gotu conservancy, Isiolo County. Mark Renzi/MSI Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 18 Conclusions As implemented by NRT, conservancies are a highly effective model for conservation, promoting striking changes in attitudes, practices and biological change — the latter based on anecdotal data collected and analyzed. The apparent slower return of browsers could be the result of a lack of good grazing land, as 70 percent of the area covered by NRT is said to be degraded and it takes many years for such systems to recover. Therefore, if NRT could fund grazing committee efforts more fully and successfully scale up rangelands restoration, grazing resources may be able to rebound. This would provide habitat to support the return of significant numbers of browsers (and related predators), thus enhancing biodiversity and simultaneously supporting pastoralism. Rigorous scientific data is needed to understand whether or not this hypothesis is valid, and to generally support biodiversity management decisions. It would appear that NRT has not yet provided the same focused efforts on managing other resources as it has on wildlife and rangelands. 1.4. How is NRT’s Community Conservancy Governance Model Perceived? Findings In individual interviews, respondents almost universally reported support for the conservancy concept and its approach to governance. Naturally, individuals participating in conservancy management, such as the board and committee members, had a stronger understanding of the system. The understanding of youth and women was limited to the value of peace and the general idea that anti-poaching supports wildlife return, which they felt would lead to tourism revenue. Communities also appreciated the development investments of conservancies (such as funding for school bursaries, health clinics and school infrastructure). Interestingly, this was noted more often in group interviews with boards, elders, women and management committees,21 and less often in group interviews with younger community members; only two of five youth groups and one of five ranger groups (themselves mostly youth) stated their appreciation. The governance structure in the NRT model places enormous responsibility on its management committees and boards for planning, fund management, fund oversight, fundraising, negotiating with outsiders, county interface, etc. With such important roles, their skills and motivation are key to conservancy success. Also, their powerful positions enable them to take conscious steps that can either aid or obstruct progress, depending on how actions align with individual and collective interests. 21 Such funding was appreciated by three of five management committees, four of five boards, five of seven elders and committees and four of five groups of women. The cattle boma technique reverses erosion in Kalama conservancy, Samburu County. Mark Renzi/MSI Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 19 Without specific questioning, majorities in each interviewed community group volunteered that elders provide important leadership and oversight of conservancy functions.22 Elders were perceived to be in control of conservancies and received a great deal of respect from all community segments interviewed. Board members’ skills varied widely, with some reportedly lacking formal education and others being well trained. Some were new to community leadership positions, while others demonstrated skills and sensitivity in community mobilization. Some boards appeared passive and were at least perceived to be motivated by individual gain (e.g., seeking jobs for family members or gains from businesses), while others proactively sought to expand possibilities for their communities by reaching out to external stakeholders. Board members described a range of evolving challenges. These were related to the specific nature of their conservancies and current activities, such as their ethnic diversity, geographic size and characteristics, whether they were seeking to develop or manage tourism revenue, negotiation with and monitoring of investors, status of land ownership, structure of the conservancies vis-à-vis group ranches, urban/rural splits and the characteristics of their resource endowment, among other variables. Board members appreciated the training NRT provided as they came into office. However, they also noted that it was general in scope, whereas the challenges they faced were specific, and that they had not received follow-up to the training to help them address issues as they arose. All communities interviewed noted limitations in the capacity of their conservancy boards, particularly with respect to their accountability to communities, which has contributed to erosion of board credibility. In three of five conservancies visited, annual general meetings (AGMs) theoretically provide at least some community accountability check to boards — a basic tenet of the NRT model — but are not held every year, due to reported lack of funding. When held, it can be logistically difficult to achieve effective community representation, given the costs of feeding and transporting people who are dispersed over large areas. Budgets constraints also limited committee and board meetings in some cases. Lack of regular meetings combined with weak board capacity in some cases has resulted in communication gaps between conservancy leadership and community members. This was reported by four of five boards, four of five ranger groups, four of seven elders and committee groups23 and three of five youth groups. Interestingly, such gaps were not emphasized as much by conservancy management committees (only two of five) or women groups (just one of five), presumably because of their direct involvement in the former case and low expectations in the latter. A lack of communication and transparency about financial management can lead communities to mistrust. Indeed, significant perceptions of corruption in the board were shared in two communities visited, specifically in the two that received significant revenue from a tourist lodge. Overall, almost four times more groups interviewed at the community level described their boards as ineffective (19 groups) than said they were effective (five groups). The team noted that conservancies with single ethnic groups did not report problems with neighbors who are members of other ethnic groups. Conservancy integrity remained intact while peace and security efforts managed inter-ethnic conflict issues, both proactively and in response to acute events. In contrast, elders of the two multi-ethnic conservancies visited reported that although peace remained intact, differing attitudes toward conservation and perceived differential treatment by ethnicities within their conservancies greatly complicated conservancy management.24 Each stated that the NRT 22 As reported by six of seven elders and committee members, three of five conservancy management committees, three of five boards, three of five rangers, three of five youth and three of five groups of women. 23 The seven groups in this case include those who were formally gathered as elders and what we had originally identified as committees. The latter were also all considered elders, culturally. 24 Inadequate communication is likely a complicating factor. The evaluation team was not tasked with investigating such allegations, and can only report perceptions with any degree of confidence. Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 20 involvement being provided was not sufficient to help manage these tensions. Elders in these communities suggested that their conservancies should be split into separate conservancies, based on ethnicity, if differences between the ethnic groups could not be resolved. Finally, partners and county and conservancy management informants widely noted the importance of finalizing conservancy management plans. The following were cited as reasons: meet legal requirements, guide resource management, monitor progress, provide a framework for negotiating with investors and seeking county support, communicate with others, develop budgets and raise funds. Conservancies noted considerable delays in the final production and printing of the plans by NRT. One investor tourism agreement frequently references the conservancy management plan as a basis for ongoing investor/conservancy decision-making, but the plan remains incomplete two years after signing. Conclusions The overall governance model is sound and is widely appreciated in conservancies — even among those not engaged in core management or that do not feature as prime direct beneficiaries, such as women. The key challenge rests with leaders below the position of elder (who retain high credibility) — namely with the delivery capacity of boards and, perhaps to a lesser extent, management committees. The NRT model — as with the majority of community-based natural resource models in Africa, in the experience of the evaluation team — places conservancy boards as the critical leadership and decision￾making node for most of the critical sustainability issues faced by communities seeking to manage their natural resources. Conservancy management committees play an important executive support function to the board. None of the boards visited appeared to have the communication, capacity and credibility to guide their conservancies to sustainability at this time. Effectiveness was constrained by locally specific issues, as well as common factors such as limited funding and human and institutional capacity constraints. Meaningful board accountability to the community appeared faint across the conservancies visited. Inadequate communication between conservancy leadership and the community at large appears to drive the problem. These challenges seemed apparent even where the team witnessed talented board members who seemed ready to make a difference and were keen to have their capacity developed. This is a critical gap to conservancy functioning, independence and sustainability. Given the diversity of capacity on boards, the wide range of skills required and the variety of challenges buffeting boards over time, a “one size fits all” approach to capacity building can have only limited impact on the ability of boards to do their jobs effectively. Board capacity was found to be a key constraint to achieving sustainability in the two older conservancies visited (both of which had significant income streams). The two conservancy boards facing multi-ethnic challenges also were not adequately empowered to sort out internal issues without external assistance. The conservancy with five different ethnicities seemed to need significant assistance, whereas the one with two ethnicities could benefit from relatively lighter, but still firm, support. Multi-ethnic conservancies face challenges in remaining intact without special attention. This observation could be isolated to the two multi-ethnic conservancies visited (there are 10 total), or it could be a systematic challenge. 1.5. Other Systems or Factors Within the NRT Model The above evaluation questions seek information on specific aspects of the NRT model thought to be important to the success of the NRT model. In seeking answers so those questions, the team identified other issues that also appear to influence conservancy experience. Items below are based on findings Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 21 and conclusions made elsewhere in this report and, likewise, feed into recommendations in other sections. Thus, this section is not separated into findings, conclusions and recommendations. Rather, the “other system or factor” will first be identified (in italics), with brief comments following to place the system or factor in context.  Using peace and security as a foundation for conservation is a successful approach in the regions visited. This proved useful in areas that have a history of resource-based violence and retribution, such as those evaluated. Thus, it is not clear that the peace program, which was such a driver of success in NRT Center and NRT North East, would have as strong a utility in other regions.  Helping rangers achieve KPR (Kenya Police Reservist) status was strategic to the success of the ranger program. That status provided an immediate link with police and enabled rangers to carry arms and apprehend lawbreakers. While managing armed men within an environment of intense ethnic conflict no doubt presented its own challenges to NRT, having KPRs greatly expanded the credibility and the power conservancies could project.  The strong NRT/community, county, police, KWS partnership is synergetic for impact and sustainability. The evaluation team was impressed with the depth and quality of the partnerships NRT has developed with county, police and KWS officials. This multiplied enforcement, conservation and security impacts and is now evolving into a source of political, institutional and financial sustainability. NRT has facilitated the communities’ access to government services to which they are entitled.  NRT leadership has bolstered the regional and national conservancy movement. The magnitude of area supported by NRT, the innovations it has created, the power of its examples on the ground in the North and in the board rooms of Nairobi and its leadership on the Kenya Wildlife Conservancy Association and related policy efforts have helped define conservancies in Kenya. It was also reported that NRT injected a more indigenous influence into the national conservancy movement.  The NRT focus on working through communities has been critical. In all areas, NRT’s approach is to have conservancies lead the process. While frustrating and occasionally time-consuming, this approach yields far greater ownership and potential sustainability. Conversely, it has proven to be vexing to the communities and NRT when other NGOs, perhaps driven by a need to meet donor targets, intervene outside of that community-managed sphere — for example, by digging boreholes in conflict with conservancy planning.  Inter-conservancy linkages for learning, policing, motivation and grazing. The evaluation team found examples everywhere it traveled of communities being inspired to form conservancies based on witnessing neighbors’ successes, of community leaders visiting other conservancies to learn techniques, of sorting out grazing conflicts across conservancies, of an “all hands on deck” approach to having rangers from one conservancy pitch in and help sister conservancies with cattle-rustling events and of the value of annual NRT-based meetings of conservancy staff and leaders.  Group ranches provide secure land ownership, a history of community governance and supplemental ranch funding. Conservancies initiated on group ranches already had an established tradition of AGM participation, understood board management and had financial resources from the group ranch that could fill in while the conservancy awaited NRT funding. They also benefited from more secure land ownership than some other conservancies. At the same time, a potential elitism-based problem surfaced with group ranch members receiving greater conservancy rights than other residents. Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 22 Evaluation Question 2 Considering separately the perceptions of men, women and youth (male and female), which of the following aspects of the NRT model were considered successful by communities, and in what ways were they considered successful? a) Peace dividends b) Economic opportunities c) Improved rangelands d) Women’s empowerment e) Youth empowerment Findings and conclusions are organized by the five topics above. The emphasis in this evaluation question is on understanding differential perceptions and programming. 2.1 Peace Dividends Findings The NRT peace model is fostering opportunities, developing skills and providing logistical support so that elders from different communities can join together to negotiate issues related to peace. Not surprisingly, those who manage conservancy affairs — and best understand the NRT program and how conservancies take advantage of it — considered elders to be key to establishing and maintaining the peace, as follows: four of five conservancy management committees, four of five conservancy boards and three of seven elders and committee members. But youth (two of five) and women (two of five) less often mentioned the importance of elders to peace initiatives. This could be because they feel that such traditional engagement was obvious and didn’t need mentioning, or that they were less informed of the actual process followed, due to being less directly involved in conservancy affairs than men. All groups visited reported that before conservancies were established, violence and theft related to natural resources and livestock (and, in one case, road banditry) was prevalent, but that now it was thankfully a thing of the past. There was enormous appreciation for this change in their lives. Groups of men, women and youth in all conservancies visited emphasized that peace was the key trigger to the benefits they have received and the others that they fully expect to emerge from conservancies. Some of the more frequently mentioned benefits that peace triggered were the following:  The ability to travel freely across different ethnic domains in the northern landscape. Many respondents noted that they could now walk in other communities’ villages and, in one case, even inter-marry;  The return of wildlife, which community members felt would attract tourists;  The enabling conditions to discuss and collaboratively plan for grazing during the wet and dry seasons within and outside the conservancy;  Being able to negotiate for access to pasture and graze their livestock over wider areas outside their conservancies, thanks to peace agreements, supported by ranger policing;  The willingness of tourists to begin to visit their areas, thanks to greater peace and prevailing security; and  Safety for their youth and women, especially from cattle rustlers. Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 23 Once peace had been established in the two multi-ethnic conservancies visited, boards and conservancy managers paid strict attention to equity among ethnic groups in benefits provided by the conservancy to continue promoting harmony. This was evident in the equal number of ranger posts and board member positions allocated to each ethnic group, as well as in apportioning, as equally as possible, other benefits. This concern appeared well justified, as men, women and youth interviewed in those conservancies discussed this issue in fine detail. The evaluation team did not observe proactive approaches to including women in peace efforts. A few reports surfaced of women’s participation locally, but none outside the community. Some informants indicated that it would be culturally difficult for women to play visible roles in negotiating peace. For one thing, it was reported that women were generally not permitted to travel far from home; men also felt women would not be effective in brokering talks between warring morans. However, one woman ambassador was praised by informants for her peace crusade across the conservancies. These perceptions were consistent with information on gender dynamics in these communities described in the USAID-funded NRT gender study (NRT Gender Analysis, 2016). In contrast, four of five conservancies described specific Youth Peace Ambassador/Key Moran efforts to engage youth in peace programs. In this case, youths were strategically targeted because of their central role in creating conflicts and staging retaliations, as well as the perspectives they shared with youths from other conservancies. Conclusions NRT seems to have sensitively framed the peace program within prevailing cultural norms in the regions visited. Elders are the drivers of the peace program, consistent with their traditional roles in their own and neighboring communities. Youth are targeted for less central — but important — roles because they initiate most conflicts and are the key to halting them. Women, on the other hand, are not included in a significant way in the peace program. This appears to be because it is unlikely — apart from the natural interest in including women in activities wherever possible — that their direct participation in conflict management would be productive or welcome. NRT and conservancy leadership wisely strive to maintain equity among different ethnic groups in multi￾ethnic conservancies as a way to maintain peace and promote conservancy harmony and productivity. In the absence of equity, the clamor is for each ethnic partner to hive out a piece of the conservancy into an independent unit, which would otherwise erode the gains on peaceful co-existence. Communities appear to fully own the peace program, which leverages respected — and locally committed — authority and traditional systems. NRT’s work in this complex dynamic represents a masterful balance of support and leadership. 2.2 Economic Opportunities During the phase of NRT evaluated, the chief efforts to promote community livelihoods included the LivestockWORKS and BeadWORKS programs, through which NRT facilitates the sales of livestock and bead products. Tourism is another key means through which NRT promotes livelihoods, but since the team visited only two conservancies with significant tourism activity, they were unable to explore the differential benefits to women, men and youth more in depth. NRT also promoted economic opportunities through a few smaller programs, such as enterprise groups and business finance efforts, which, similarly, could not be thoroughly evaluated. LivestockWORKS (which reached fruition in Phase III) was appreciated by men, women and youth in all conservancies as an outlet for sales and for the associated contributions to conservancy operational Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 24 costs (for each cow sold, the seller contributes $10 and NRT contributes $20 to the operating budget).25 The total value of cattle purchases in 2015 was $1.7 million, a significant sum and an increase over the prior year. Suggestions for improvement from men were that the program should be more consistent, buy more animals, provide sufficient notice of sales, be more frequent, should not be limited to bulls and should exclude middlemen from the process. All conservancies reported that income from cattle sales generally was used to purchase more cattle: “sell one, buy three” or a variant was the common refrain. Since youth typically manage cattle, NRT reports there was no need to target them specifically. Youth did report sales. Although no specific efforts were made to target women through LivestockWORKS and women do not — at first — appear to benefit directly from cattle purchases, women’s groups unanimously endorsed the program, apparently because they are able to sell their cattle through male surrogates at NRT events. Women suggested the program should be expanded to include sheep and goats, which are often managed by women. The most focused targeting of women for economic benefits is through sales of bead products and, increasingly, for employment in conservancy ranger posts. No women were employed in the conservancy management committees visited. Women in two conservancies were very unhappy with inconsistent NRT efforts with bead enterprises during Phase II, although all women in all cases appreciated whatever income they could gain from beads. The post-grant BeadWORKS program (leveraging new funds for NRT/T) is highly valued (although lack of follow-through was again noted in one case.) Demand from women to participate appears high for this woman-targeted program. Total sales in 2015 were $17,000, although significant increases are projected when the 2016 numbers come in. Overall, youth economic benefits were significant; they gained employment as rangers and conservancy management staff, were able to sell cattle and were specifically targeted for training and financing for new business (although the number of participants in this latter program is quite small.) Conclusions The commercial benefits of cattle sales are significant at both the household and conservancy levels. The commercial benefits of bead products are significant at the household level but modest at the conservancy level. Conservancy employment is a highly valued benefit wherever it occurs. It is well targeted to youth and much less so toward women. Overall, youth appear to be relatively well targeted with economic benefits. NRT has improved its targeting of economic benefits to women through an improved BeadWORKS program. The new BeadWORKS program demonstrates NRT/T’s ability to recognize when a program is operating below expectations and then to innovate to succeed. BeadWORKS appears to be on the right course. While all of these benefits were appreciated, the income streams do not yet appear to be helping to diversify significantly communities from reliance on livestock or to promote destocking. This is likely a longer-term effort as the circumstances of pastoralists are influenced by low literacy levels and environmental conditions, which limits the options for land-based production and access to gainful employment. 25 As mentioned by four of five management committees, four of five boards, four of seven elders and committees, four of five youth and all five women (and mentioned by at least one group in each conservancy). Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 25 2.3 Improved Rangelands Findings Satisfaction with rangelands management is reasonably uniform among men and youth: three of five conservancy management committees, four of five conservancy boards, four of seven elders and committee members and three of five youth groups. Not surprisingly, women were less enthusiastic (two of five groups), considering their limited role in creating or implementing grazing plans, as described below. In all but one conservancy (where ethnic cultural differences and tensions appear to be at play), all groups (including women) reported a high level of adherence to grazing directives, even where the grazing plans were not formally written and distributed. All but one conservancy felt that the grazing plans were helping them avoid livestock loss, especially during dry seasons, when herders were allowed to access grazing within the conservancy on a rotational basis. Compared to women, youth have a greater understanding of the details of grazing plan implementation since they are in the bush with cattle, whereas the women are confined to the home and grazing near their bomas. At least one conservancy used “grazing warriors” to implement its grazing plan, taking advantage of the fact that morans were at the front edge of grazing movements. With the exception of women collecting and selling grass seeds to support rangelands restoration in one conservancy (an example of leveraging an opportunity) little differential focus has been placed on women in rangelands management. Women do not sit on any of the conservancy-wide grazing committees in our sample (one conservancy requires one woman be in each village level grazing committee). This is true even though women in four of five conservancies visited tend to sheep and goats (although they often do not own those animals) and to some of the cattle left behind as men take herds far away in search of grazing in dry periods. Overgrazing by sheep and goats is reported to cause greater long-term damage to rangelands than by cattle, since the former animals (collectively called “shoats”) are reported to “eat everything,” including the roots of grasses, making grazing resource recovery difficult. Conclusions Grazing rules are generally appreciated and understood by men, women and youth. Participation in rangelands management is greater for youth than for women, although elders participate in planning and monitoring compliance much more often than women or youth do. Greater participation by women may improve quality of grazing planning and implementation, particularly with respect to shoats management. 2.4 Women’s Empowerment26 Findings In the traditional, male-dominated cultures visited, there is little tradition of women’s participation in leadership and decision-making bodies. In fact, it is difficult for women to speak freely among groups of men. As an example of the challenges to involving women, one elder in a visited conservancy said, “Why should we include women when it just limits our power?” Nevertheless, NRT has encouraged conservancies to include women on their boards. Of the sampled conservancies, women have been 26 Women’s empowerment is often considered to include economic, political and human rights dimensions. The former is discussed in section 2.2, Economic Opportunities. The latter two dimensions are addressed in this section (2.4). Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 26 elected to three boards and co-opted to two. (See also grazing committee representation described above.) There appear to be few non-traditional role models for women in the rural areas visited by the evaluation team. Except for NRT’s peace ambassador and a few radio operators in the ranger ranks, all NRT staff referenced and seen in the conservancies during evaluation visit were men. USAID recently funded a gender assessment of NRT’s work, which was highly valued by NRT staff. It recommended including minimum female participation requirements in conservancy constitutions as well as focusing on reproductive health as ways to empower women through NRT’s work in conservancies. That study describes in detail the cultural constraints (also voiced by many community informants) to full women’s participation in the region. Conclusions NRT appears to have tried to leverage Kenya’s new constitution to increase women’s participation in elected bodies, though those targets had not been achieved in the conservancies visited. NRT does not yet appear to have included a significant number of women as employees in its field work. Integrating women in conservancy decision-making systems appears to be an important avenue toward gender equity. While it is important to continue, and diversify, this approach, such efforts are likely to be difficult, require investment by NRT of political capital to convince elders of the value of such efforts, and require time. In spite of these challenges NRT has had limited, but significant, success in getting women on boards. In such a gender-split society, it may make sense to complement the above-described integration approaches (likely to yield results only over a long time horizon) with “gender-parallel tracks” that may produce more rapid results and at the same time prepare males and females of all ages to engage in a more gender-equal society in the future. The evaluation team felt the gender assessment’s recommendations (described above) were sound. 2.5 Youth Empowerment27 Findings Boards of four of the five conservancies visited include at least one youth board member, although there are always fewer youth than elders. One conservancy visited had a youth as chairman of the conservancy board. As described in Evaluation Question 1, youth are involved in the peace program, an example of NRT leveraging emerging opportunities to empower youth. Most rangers are youth, a position that has considerable influence over conservancy welfare. As discussed above, many conservancy management committee staff — an even more influential and prestigious role than rangers — are also youth. Conclusions By far, the greatest differential approach is to hire youth as rangers and conservancy management committee staff. In the case of rangers and conservancy management staff, this has both economic and political empowerment dimensions. 27 Youth empowerment has both economic and political dimensions. The former is discussed above in the section 2.2 on Economic Opportunities; the latter is addressed in this section (3.1). Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 27 Youth engagement in the peace program and overall governance is limited but has begun, and could provide a useful framework for greater youth involvement over time. Evaluation Question 3 How has the perception and planning around NRT’s sustainability changed over time and what specific factors influenced the current sustainability plan? The evaluation team examined three strategies: a) Increasing revenue from the county governments b) Increasing commercial revenues (e.g., scaling up beadwork, ecotourism, livestock-to￾market programs, microenterprise, etc.) c) Increasing capacity to access donor funding Findings and conclusions are organized by sub-question, with other issues critical to sustainability not specifically mentioned in the evaluation question. 3.1 Evolution of Sustainability Planning Findings NRT has placed increasing emphasis on sustainability over time. Originally, NRT appears to have focused on becoming operational and succeeding on the ground (“proof of concept”). Sustainability does not seem to have been a central part of NRT’s orientation. By the 2012-2017 Strategic Plan, however, NRT listed “NRT growth and conservancy sustainability” as one of its objectives, with creation of a trust fund as a sub-objective. Although no other mention was made of sustainability in the plan, it seems that at around the same time, NRT worked with Oxford University’s Said Business School to plan more concretely for sustainability at the conservancy level. They produced a Conservancy Financial Sustainability Plan 2012-2022, which projected that new conservancies could become sustainable in eight years at funding roughly in line with what NRT has been providing. Said’s plans focused on financial sustainability, anticipating revenues primarily from tourism, but also from NRT trading, especially livestock purchases, and from the GOK. They also cited future potential income streams from bio enterprises, carbon trading and payments for ecosystem services. The plans contain sound advice for extracting more value from the prime income streams to drive sustainability. NRT appears to have adopted some of those recommendations to increase sustainability income from beads and livestock, but less so for tourism.28 The business school also produced individual sustainability plans for the three NRT-supported conservancies that already received sizeable tourism income (Westgate, Kalama and Sera), based on the same conceptual framework. The financial sustainability plan for Kalama conservancy (sampled in the evaluation) projected financial sustainability by 2016, if their recommendations, such as increasing tourism beds and conservation fees, were followed. More recently, NRT has produced its own sustainability plan, based predominantly on three meta income streams: county funding, commercial revenue and donor funds, including an endowment fund. Thus, the new strategy appears to build on earlier thinking, but adds the potential for revenue through active engagement with counties (made possible with Kenya’s decentralization, a transformation that occurred after the prior strategic plan drafting), while reducing interest in central government revenues. The balance of this section will use the new sustainability strategy as a framework for discussion, as implied in the Evaluation Question 3 sub-questions. 28 NRT may want to revisit some of the recommendations for enhancing tourism revenue to consider whether the time may be ripe to pursue some of them. Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 28 The biggest visible shifts in NRT fund-seeking behaviors have been to try to tap into county resources and developing an endowment fund with TNC (The Nature Conservancy) support. 3.2 Sustainability (County) Findings NRT, development partners and conservancy staff all report that NRT and conservancy leadership place increasing importance on achieving county funding in their plans for sustainability. There is a strong sense among those best versed in sustainability issues that significant potential exists for obtaining county funding to support sustainability: eight of 10 development partners, two of three counties all five NRT informants and four of five conservancy management committees. NRT and development partners feel that NRT’s security and conservation work is a public good that warrants county funding. On the county side, as devolution takes hold, a window of time appears to be opening where counties have funding and are seeking appropriate ways to invest it in their communities. However, they lack the social infrastructure to reach down to the community level to identify needs. Development partners and counties both identified conservancies as a useful community-based platform that counties could use to access and support communities. Although only a minority of each community group interviewed — two of five conservancy management committees, one of five boards, one of seven elders and committee members, zero of five youth and no women of the five — reported that their conservancies were receiving county funding, it appears that the majority actually were benefiting from direct county investment in conservancies through construction of ranger posts and ranger housing or contributions from county wildlife reserves. However, in several cases, county funds were less than expected, delayed or spent on projects other than those requested by the conservancy. It seems that communities either already take such sums for granted or don’t yet link them to conservancy sustainability. Those closest to the ground, the conservancy boards, were less sanguine that county funding could support sustainability, with only one of five boards stating that counties were an important source of future sustainability funding. NRT is actively lobbying county officials, providing valued technical assistance to counties that have themselves begun to form conservancies, and is helping conservancies submit proposals to raise county funds — in one case combining conservancy proposals into a unified proposal to county government to facilitate approval. NRT County Liaison staff have promoted NRT and conservancies with county officials and have become trusted partners to help develop counties. Counties were unanimous in their appreciation for NRT’s support to conservancies, for the quality of technical assistance NRT provides county officials and in their commitment to supporting conservancies in the future. As noted, counties are funding infrastructure and providing operational support in most of the conservancies visited, while also funding operational and infrastructure costs in county-established conservancies. Conclusions Counties recognize the utility of the conservancy platform in helping them provide county services at the community level. They are prepared to provide financial support to conservancy infrastructure and operational costs. NRT’s integrated approach to seeking county funding — lobbying, developing proposals, getting conservancy costs in county strategies and plans and providing technical assistance — is a comprehensive and strategic approach to access this emerging county interest in supporting conservancies. The strategy is bearing fruit in political commitment, operational support and funding. Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 29 Based on wide perceptions, county behavior and NRT-sustaining actions, it is feasible to anticipate that counties will provide increased funds in the future to support conservancy costs, although progress may be uneven. 3.3 Sustainability (Commercial) Findings Commercial sustainability seeks to attract a range of potential sources of market-based income streams to support conservancy long-term viability. Ideas have been floated, such as solar farming, honey and gum and resin collection. This section of the report will focus on the three main areas that were present in Phase II and that were referenced in NRT’s sustainability plan and are actively supported by NRT Trading (NRT/T). Established in 2014, NRT/T is a for-profit social enterprise owned by NRT. Its mission is to identify, incubate, pilot and grow sustainable businesses within conservancies supported by NRT. In a relatively short time, NRT/T has managed to consider, and in some cases test and roll out, creative solutions for different products and business models. In the regions visited by the evaluation team, NRT/Trading is working to scale up businesses in beads production and marketing, cattle marketing and tourism. None of the product lines were operating at scale or without large subsidies during Phase II, but they have since progressed and provide varying possibilities for future sustainability. NRT/T recognized the shortcomings of the former beads program and worked hard to overhaul the program. Although still profitable to NRT, revenue diminished in 2015 during the period of transition. The BeadWORKS program that emerged provides raw materials to a three-tiered network of community bead workers and supervisors to produce bead products that are then sold to NRT at set prices. NRT subsequently sells the products overseas. The improved BeadWORKS business model appears sound. NRT/T was able to rapidly increase the number of women participating in the 2016. Women engaged in the program were almost universally enthusiastic about the new program design, reporting that they very much value the income it provides them. While 5 percent of sales revenue is contributed to conservancy coffers, NRT is seeking to expand its ultimate market for beads through the website and “champion agents.” NRT/T has also invested considerable effort to improve the model, now called LivestockWORKS, used to purchase cattle from conservancies NRT deems to be well-performing. The model has introduced a potentially transformational element into the NRT model: community contributions to sustainability. For each cow sold, NRT contributes approximately $20 to fund conservancy operational costs and the seller contributes $10. All conservancies appreciated this contribution to the operations of conservancies and none complained about the financial burden — as long as the selling price was right. NRT/T noted that while they felt that they had made progress improving LivestockWORKS, they estimated that they would need roughly to double production (purchases were 6,000 head from nine conservancies in 2015) to financially break even. Communities expressed willingness to sell far more cattle than that — although they would prefer to sell a wider quality range of cattle than NRT currently specifies. One major limitation to the current LivestockWORKS model reported by NRT/T is access to pasture to fatten the cattle it buys to add value prior to resale (a key element of the current model.) NRT is Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 30 trying to resolve this issue, but has not yet succeeded. It is also considering other parts of the cattle value chain, including establishing relationships with abattoirs (where additional value can be realized). NRT/T’s website describes the third principal thrust as follows: “it is the aim of AdventureWORKS to help put northern Kenya on the tourism maps and to help facilitate travel for people of many backgrounds and interests in this unique and exciting destination … AdventureWORKS is helping conservancies to develop long￾term agreements with investors and promote tourism in northern Kenya.” NRT/T is also trying to leverage a Wild North national marketing program and promote adventure tourism. Tourism is seen to have the potential to provide significant sustainability revenue in a few conservancies in the NRT group, while potential is reported to exist for other — but not all — conservancies. The evaluation team visited two conservancies that had lodges. Both were impressive, relatively high￾end facilities, run by persons that are reported to be committed to conservation. One investor (the son of the prior contract holder) was recently granted a 30-year agreement and is expanding operations through establishment of an elephant orphanage and increasing the number of beds from 18 to 50, with the construction of two more facilities. The other has recently begun what was termed a relatively small-scale trial investment in a neighboring conservancy with a rhino sanctuary In both lodges, tourism benefits are shared by the lodge paying conservancies through a “bed night levy,” a fixed fee (the amount differs between the two lodges, and by tourist age and whether the tourist is a Kenyan resident/citizen or not) paid for each night that a tourist spends in the lodge. The advantage of this approach is that fees are simple to monitor and calculate. Properly executed, it supports transparent accounting and open communication between investor and conservancy. In each of the two conservancies, informants indicated that the investor was understating the number of bed nights, resulting in lower revenues to the conservancy. In one conservancy, the community had not been monitoring bed nights, but instead relied on investor reporting. In that case, the board appeared to be divided on the issue: some felt it was best to trust the investor; others felt the board had an obligation to monitor bed nights strictly to ensure the conservancy was maximizing revenue from the lodge establishment. In neither case, were the allegations proven, but the number of bed nights were not effectively monitored and publicly displayed to community members. In this information vacuum, suspicion arose and is undermining confidence in each of the conservancy boards. It was clear that NRT was not executing its responsibilities to support (and serve as honest broker) of community/investor agreements. Specifically, in the one contract the evaluation team reviewed (two years after the contract’s execution), NRT was expected to hold regular investor community meetings (there had only been one), was supposed to establish a bed night levy monitoring system (but had not), Saruni Samburu, an ecotourism facility at Namunyak conservancy, Samburu County. Mark Renzi/MSI Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 31 to demarcate core area boundaries (but had not) and to complete a conservancy management plan that was an important foundation for the agreement (but such plan was still not finalized.) The agreement also called for the investor to notify the community when occupancy rates fell below 22 percent, but rates on average were only 11 percent, a statistic that should have warranted community and NRT inquiry. Finally, when the evaluation team brought the possible bed night shortfall to the attention of NRT, the investor discovered that he had failed to adjust the foreign exchange rate, resulting in a cumulative loss to the community of $10,000 (which the investor is now preparing to pay the communities.) Conclusions BeadWORKS provides important benefits to women and is likely to enhance their commitment to conservancies. But, given market constraints and the relatively low dollar value of the trade, it is not likely to provide a significant sustainability stream. Even greater social and sustainability impact could be achieved if the almost-proven labor outsourcing model could be applied to other goods – and then scaled. This could begin with other traditional cultural products, and then expand to include other hand￾made goods. LivestockWORKS is a potential sustainability game changer, if NRT/T can adapt model to buy at a much larger scale, thereby mobilizing community funds to promote financial and governance sustainability Although there are long-standing community tourism projects, NRT/T’s AdventureWORKS is just getting started. Supporting tourism product marketing is a useful niche for NRT, unless it could be outsourced to enhance effectiveness and efficiency. But, such marketing efforts should be a secondary focus to maximizing current revenue. It appears that AdventureWORKS is not adequately monitoring and supporting existing investor contracts. Some sentiment exists that at least new contracts may not have been well structured to meet needs of communities. Lodge revenue collection was not transparent. It is not possible to discount the possibility of underpayment by lodges to communities. Any shortfall here greatly impedes sustainability potential and undermines NRT and conservancy leadership credibility. Reporting and meetings were not thorough. NRT/T does not appear to have invested sufficiently in this area. 3.4 Sustainability (Donor) Findings NRT has managed an enviable growth rate from its original inception as a way to implement Lewa Conservancy’s original vision, to the point where it has an annual income of approximately $11 million29 and is supporting diverse activities throughout a large and important swath of Kenya. Since just 2010, it has grown over 19-fold, increasing by half-again each year (average year-on-year increases of 49.8 percent.) Since the NRT model contains potentially robust components of security, economic growth, conservation, decentralization and climate change — all of which are inextricably linked — NRT can justifiably market itself (and individual conservancies) in donor niches, arguing that any investment in a single area also leverages benefits in the others. NRT produces professional marketing materials and is able to maintain important marketing ties, such as with Lewa Conservancy (since its inception) and The Nature Conservancy (TNC), which appears committed to a long-term alliance, including helping to establish and partially fund an endowment for long-term sustainability. 29 NRT Financial Statement, year ended December 31, 2015. Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 32 NRT has managed to develop a diverse and apparently loyal donor base, including long-term development agencies (USAID has committed to consistent funding over a 13-year period, which is uncommon for the Agency). These include international NGOs, individuals and government programs from Netherlands, France and the United States (including multiple agencies of the latter two.) In 2015, NRT received support from 30 organizations issuing KES 649.8 million worth of restricted funds, and five donors issuing KES 290.5 million worth of unrestricted funds. Tullow Oil was the largest contributor of restricted funds (33.5%), followed by USAID (11.4%), The Tusk Trust (9%), and The Nature Conservancy (9%), while the French Development Agency (AFD) and DANIDA each contributed roughly 6.8%. Three donors contributed over 98% of the unrestricted funds: USAID (38.3%), The Nature Conservancy (35.5%) and the Royal Netherlands Embassy (24.6%).30 Donors interviewed as part of this evaluation process were universally laudatory of NRT and appeared to feel that their funds were being directed to critical challenges effectively by NRT. Rather than simply using NRT as a local partner to implement a particular set of donor-driven activities, each of the large bilateral development donors expressed their commitment to supporting NRT’s program, while understanding that achieving results was a long-term process. There was no sense of “donor fatigue” evident in our discussions with donors with respect to NRT. Most conservancy management committees shared NRT’s desire to reduce conservancy dependency. But at the community level, no youth or women expressed a desire for more independence, and only three of the men’s groups said that they wanted to reduce dependency on NRT. No women or youth interviewed expressed awareness of how donor funds could be accessed for sustainability beyond NRT. Men in conservancies visited were slightly more aware (fewer than a third of the groups) that donors beyond NRT could be tapped for funding; only two of the 17 male community groups interviewed were aware of efforts by NRT to help conservancies raise funds from donors. NRT has worked with conservancies to make proposals to access its Livelihoods Fund and to seek funds from county governments, an experience valued in communities. Conclusions NRT’s success at raising long-term donor funds to support its multi-faceted, important, and timely mission is extraordinary. It is well placed to attract funds for the donor portion of its sustainability plan. This is an extremely valuable competency that NRT brings to conservation in Kenya. Fundraising niches for conservancies, Lewa and NRT may well be complementary. Although conservancies currently lack significant fund raising skills, they appear eager to learn how to obtain donor funding. Working on proposals with NRT is a valuable first step to access funding through diverse fundraising vehicles as TNC, Lewa Conservancy, NRT and individual counties. 3.5 Sustainability (Other) Findings Not surprisingly, community groups — especially below the conservancy management committee and conservancy board levels — were not well-versed in sustainability concepts or NRT’s approach to supporting conservancy sustainability. In fact, for many such community groups interviewed, sustainability was not first and foremost about raising funds. Sustainability was dependent on such basic enabling conditions as secure land tenure (three groups); or on improving community conservation awareness (six cases); or on improved conservation and natural resource management (six groups); or 30 NRT Financial Statement, year ending December 31, 2015. Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 33 peace (three groups). Some thought fundamental structural issues needed urgent attention to achieve sustainability; for example, groups in two conservancies identified the need to resolve ethnic issues or divide their existing conservancy into new conservancies along ethnic lines. In some cases, it was about the institutional capacity of conservancies, including overall management, such as the need to improve communication between the conservancy board and the community (seven cases); or within the ranger sphere, with some seeking more rangers (four cases), better pay and benefits for rangers (eight cases) or more ranger outposts (two cases). Two groups said sustainability would be enhanced by mobilizing more women’s and youth groups. Three-quarters of NRT’s largest donors expressed concern about NRT’s rate of expansion — although each has supported NRT’s ongoing expansion financially. NRT staff also expressed concern that expansion dilutes impact while amplifying the importance of growth. Leadership of the two older conservancies visited felt expansion was detracting from the services it was receiving from NRT as they have noticed declines over time. Every conservancy visited evidenced the need for greater investment. Widely observed shortcomings included: low ranger pay and equipment (inadequate firearms, limited ammunition, rations, kit, radios); lack of direct funding for board meetings and AGMs; lack of follow-up for rangelands meetings; lack of funding for costs to produce a grazing plan and monitor compliance; and lack of adequate funding to complete conservancy management plans. The budgeting process is also complicated by lack of funding. All conservancies reported that their requested annual budgets were not met and that during the course of the year they were not certain if agreed-upon items (such as AGMs) would actually get funded. Often, once staff are paid and vehicles are maintained, little is left for community participation, governance, rangelands restoration or other investments that would support ecological, social, institutional and political sustainability. NRT reports that — while getting more area under conservancy management is important — expansion costs are starving implementation of existing conservancy programming, a frustration for all. Numerous reports cited poor NRT follow through. These include delays in conservancy management plan production; follow-up on tourism, bead and cattle work; delays in repair/replacement of radios; and effective follow-up to training. Programs for economic opportunities (BeadWORKS, LivestockWORKS and AdventureWORKS) all currently operate at small scale. With 33 conservancies currently operating and the portfolio anticipated to grow to nearly 50 by 2020 in response to community demand,31 the management of so many units can be a challenge. It was clear that NRT only provides each of its various services (WORKS suite, grazing plans, restoration, etc.) to fewer than half of the conservancies, due in part to funding constraints. At the beginning of the data collection stage for this evaluation, the team asked NRT to prepare a matrix that tracked NRT services across conservancies. Despite repeated calls and emails to three of NRT’s top managers, it was never provided. It is not clear how NRT and USAID monitor what support is given to each conservancy, apart from the marketing descriptions. Conclusions The new sustainability plan is sound, and an improvement over previous thinking. Conservancies could also seek sustainability funding from county governments and their own community members, beginning with the WORKS suite of products. 31 The State of Conservancies Report 2015; NRT Phase III Program Description Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 34 But, given the central role of boards in seeking and managing funds, negotiating and working with partners, the lack of capacity at the conservancy board level is the most immediate constraint to sustainability. This was apparent even at the two conservancies the evaluation team visited, which were in relatively strong financial positions. Lack of investment at the conservancy level has meant that boards have minimal autonomy over operating cost budget decisions, restricting the ability for boards to learn and grow. Inadequate investment at the conservancy level and the subsequent lack of conservancy services delivered could be a result of expansion, but also stem from inadequate management systems. If the conservancy funding constraints resulting from expansion are not addressed by efficiency improvements or additional funding, conservancies will be starved for development beyond provision of basic security. NRT’s emphasis on communications to support marketing may limit clear understanding of precise accomplishments in each conservancy and cloud perceptions of overall effectiveness. Donors who rely on such material may not have an accurate sense of what is actually going on. Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 35 RECOMMENDATIONS The evaluation team offers the following recommendations for consideration, based on the findings and conclusions presented. In addition, the team used these recommendations to develop key strategic insights to inform USAID and NRT in future planning (see Annex II). 1) NRT/conservancies should determine optimal communications requirement for the rangers, and then supply – and develop a maintenance plan for – digital radios and ancillary equipment. Further they should procure other ICT devices to complement the radios32 These devices may include tablets with capability to capture geotagged images of animal sightings, while enabling rangers to record information such as the type, gender, and age of wildlife so as to provide better wildlife distribution data. However, NRT would need to devise highly secure approaches to using such technology in view of potential wildlife crimes, and should avoid doing so for critically endangered wildlife including elephants and rhinos. 2) NRT/conservancies should consider improving the rangers’ conditions of work including pay, welfare, appropriate equipment (weapons, camping gear, uniforms) – consistent with sustainability. This will not only boost their morale but also enhance their effectiveness. 3) USAID/NRT in collaboration with relevant partners should develop capacity for rigorous analysis of biodiversity, rangeland, water, and forest resources. This will result in effective strategies and efforts to support grazing plans and scaling up rangelands restoration, possibly by minting a new “GrassWORKS” program. This would be an enterprise that would use chainsaws, seeds collected by women, and local labor to restore rangelands at a much more rapid pace and greater scale than the pilot efforts – based on technologies developed there. 4) To improve the rangeland conditions, NRT should tie the livestock purchase and access to funding and enterprise microfinance to destocking of the rangelands. Demonstration sites where conservancy members can be trained practically could be a useful addition to the rangeland restoration efforts. 5) NRT should consider expanding attention to resources beyond wildlife and rangelands, such as water and wood products. Such efforts should explore other economic opportunities that are resilient to weather conditions, including climate change, such as the BeadWORKS program. 6) NRT should invest more heavily in conservancy board strengthening to address limited funding, human and capacity constraints as an urgent aspect of sustainability, through the following actions: a) Completion of management plans for all conservancies as an urgent priority, and encouragement of their use in planning, monitoring, tourism management, funding, and fund raising. b) Training the board while supplementing with ongoing mentoring and accompaniment. Assistance should be customized to the unique conditions of each conservancy. 7) NRT/conservancies should enhance meaningful board accountability mechanisms to the community by improving the communication between the two. Efforts could include fully funding an expanded array of AGM and other meetings, and other efforts; signboards distributed throughout 32 NRT claims that the new communication network is designed for this functionality, that at least one-half of the rangers already have smartphones and that the JOCC (Joint Operations Communication Center) in Lewa will manage and synthesize these data for the entire NRT network. Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 36 conservancies, updated by locals receiving SMS; hiring staff devoted to improving communications at NRT and conservancies; it is critical to conservancy vitality. 8) Considering the success of NRT’s use of the peace crusade as an incentive for conservation of wildlife and reduction of grazing conflicts, NRT should consider collaborating with other agencies whose core mandate is peace building or outsource this role to local peace NGOs. This is so as to ensure continuity and sustenance of the peace building effort as it concentrates on wildlife conservation and promoting community livelihoods within the context of conservancies and pastoralism. 9) Increase participation of women and youth on grazing committees, given their important roles in herding cattle and shoats, respectively. 10) NRT should encourage the conservancies to increase women’s integration into decision making through: a) Discussions with traditional elders how to increase women’s participation in leadership and governance. Such discussions will be difficult, and may only yield progress over time. However, elders must eventually buy in for lasting progress to take hold; b) Enshrine fixed levels of women’s participation in decision-making bodies through conservancy constitutions. Patience is required here; impact of real participation may only be visible over the medium term; c) Identify leaders, provide skill development, leadership training, and mentoring – combined networking of women leaders in and outside the NRT world – to empower those women to succeed in difficult conditions; and d) Increase the number of female NRT staff who interact with conservancies, especially with women community members. The importance of providing women and girls living examples of other paths to self-realization should not be underestimated, particularly if women from the comparable cultures can be hired. e) Establish an NRT Council of Women to ensure women’s issues are considered in NRT-wide deliberations and report back to women in their conservancies (similar to the way in which the Kenya Wildlife Conservancy Association has approached this issue); f) Develop a new class of conservancy employee, “Community Resource Monitors,” to be filled by women to collect data and perform basic analysis and community feedback as NRT improves its monitoring science. This would be an expansion to what female rangers are doing currently that works to improve NRT’s science to monitor changes in the natural resource base. g) NRT/T should consider women enterprise development programs, similar to that provided to youth and possible expansion of BeadWORKS, as currently designed as an important women’s economic benefit and minor support to conservancy financial sustainability. They should further explore whether the BeadWORKS outsourcing model could be applied to other goods (such as cultural products) as well. They should also consider establishing a women’s GrassWORKS program. h) The evaluation team endorses recommendations of the recent NRT gender assessment, including the observation that access to reproductive health care could alleviate a key barrier to women’s empowerment (though most effectively supported through a specialized partner institution), as well as its suggestion to enshrine female representation on decision-making bodies (as above). Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 37 11) NRT/Conservancies should continue with the youth program while strive to increase engagement in boards and in peace efforts. 12) To ensure sustainability, NRT/Conservancies should continue the county fund raising strategy as currently implemented. However, it should be expanded to seek GOK funding, beginning with police and KWS, based on extreme GOK satisfaction with NRT and the important governance services that NRT provides by expanding police and KWS reach. 13) NRT/T should apply the same innovative zeal it invested in developing BeadWORKS to continue to improve LivestockWORKS to become an economically viable model. It should operate at a scale sufficient to buy enough animals (including eventually sheep and goats) to provide significant benefit to sellers and at the same time significantly support conservancy financial sustainability. This would ingrain the concept of citizen responsibility to support conservancies financially. NRT/T should consider how to meet sellers’ desires for increased frequency, predictability, and market integrity. 14) NRT/T should immediately ensure that all conservancies fully monitor and enforce tourism agreements, including community, NRT and investor obligations; revenue flows; and income reporting. NRT cannot afford to “leave money on the table”. Monitoring efforts should include regular meetings with investors and communities. Adequate controls on conservancy monitoring should be employed (such as checking on the monitors, changing the monitors frequently, and spot checks.) NRT/T should help conservancy staff and boards to ensure that such visits enhance, rather than detract from, the tourist experience. 15) NRT/T in collaboration with conservancies should review existing, new, and planned tourism agreements for adequacy. It may be useful to employ outside expertise, such as Kenya Wildlife Conservancy Association (KWCA.) Results should be communicated to communities at the next AGM (if this has not already been done recently), together with a detailed explanation of benefits derived in the past year 16) NRT/TNC should introduce donors to conservancies directly (the new Namunyak conservancy elephant orphanage site would appear to be a perfect example, once the new lodge is completed), transfer its marketing skills to conservancies, and explore the various marketing niches described above. NRT should develop a comprehensive strategy to apply the full spectrum of its marketing instruments to squeeze every last dime from the various donor niches. 17) NRT’s sustainability plan should be expanded to include community contributions (beginning with the WORKS suite, and other local services, such as sand harvesting) as an important source of sustainability funding. This creates a virtual cycle where communities have greater interest in holding conservancy leadership accountable as a result of having invested their own funds in the conservancy. 18) NRT should consider reviewing its management systems to improve efficiency, quality and responsiveness. Recent devolution of management responsibility to NRT regions appears to be a positive step in this direction. 19) At the strategic corporate level, NRT should resolve the tension between allocating resources to support existing conservancies versus continuing to channel resources to expand to new conservancies. 20) For management and donor relations purposes, NRT may want to track precise implementation activities (and challenges) in each conservancy and maintain this information on an easily accessible and queried data base. Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 38 ANNEXES Annex I: Evaluation Statement of Work SECTION C – STATEMENT OF WORK C.1. BACKGROUND INFORMATION 1.1 Identifying Information  Program: Environment and Natural Resource Management  Project: Northern Rangelands Trust Project (NRT)  Award Number: AID-623-A-00-09-00011  Award Dates: December 2008 to December 2015  Period to be Evaluated: December 2008 to December 2015 (Phase II)  Funding: $6,991,386  Implementing Organization: Lewa Wildlife Conservancy (Lewa)  Contracting Officer’s Representative (COR): Beatrice Wamalwa 1.2 Development Context 1.2.1 Problem or Opportunity Addressed The loss of biological diversity has been cited as a global environmental threat to natural ecosystems and iconic wildlife species.33 In northern Kenya, communities struggle with frequent droughts, poor healthcare, sparse government services and threats posed by cattle rustling and ivory poaching. Ethnic rivalries also continue to threaten peace and development.34 This area, known as the Ewaso Nyiro ecosystem, contains two large-scale processes that have been selected by development partners as long￾term targets for conservation: dry-season river flow and elephant migration.35 This diverse region includes the entire Laikipia and parts of adjacent Samburu, Meru, Isiolo and Marsabit counties. In an effort to mitigate biodiversity loss, Kenya’s national government and its development partners have created linkages between biodiversity conservation, sound natural resource management (NRM) and sustainable economic growth.36 In a study ‘Conserving Wildlife in Kenya’, Georgiadis indicates that as far back as the 1980s the concept of conservation was limited to protecting wildlife in particular areas, but the paradigm began to shift at the end of that decade toward recognizing humans as an integral component of biodiverse landscapes. Over time, national parks and game reserves became too restrictive, especially for the long-term viability of wildlife. Hence there is a need to open up spaces for habitat and biodiversity conservation in areas that were not formally protected, most of which were communally owned or used. In addition, conservation efforts need to also account for human livelihood needs in addition to protecting wildlife and land among other natural resources. The study further found that in the 1990s the concept of promoting wildlife as a source of income was initiated among ranchers (particularly in Laikipia), which led to the creation of management associations under the stewardship of the Kenya Wildlife Service (KWS). Laikipia Wildlife Forum was launched in 1991 as a company of private and communal landholders with a shared interest in wildlife management and conservation. By the early 2000s there 33 USAID Kenya Country Development Cooperation Strategy 2014 – 2018 34 The Story of The Northern Rangelands Trust (NRT, 2013) 35 Introduction: Conserving Wildlife in Kenya’s Ewaso Landscape (Georgiadis, 2010) 36 USAID Kenya Country Development Cooperation Strategy 2014 – 2018 Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 39 was a proliferation of non-consumptive wildlife use including ecotourism enterprises on private and communal properties in Laikipia and the greater Ewaso ecosystem. However, human-wildlife conflict persisted from the pressure of opening space for wildlife due to expanding human settlement, land subdivision, the spread of incompatible land uses such as cultivation, woodland clearing, bush meat consumption, and the proliferation of firearms.37 The results of these pressures included crop raiding by wildlife, attacks and in some instances death of both humans and wildlife. As a result, in the late 1990s, a variety of conservation initiatives were initiated in northern Kenya to mitigate human-wildlife conflict, including community-based conservation efforts. Community-based conservancies aimed to create incentives for landholders and land users to conserve wildlife and natural resources. These incentive-based approaches, where economic rationale provided the impetus for changing conservation strategies with landholders and land users, focused on realizing the market value of wildlife conservation via ecotourism, trophy hunting and live animal sales. Community conservancies produced mixed results including positive development and persistence of the initial problems. Kenya’s Vision 2030 development strategy includes conservation and sustainable management of the country’s natural capital as an important strategy for the country’s economic growth, as well as attaining a clean, secure and sustainable environment.38 Toward this end, USAID promotes long-term community resource management by helping Kenyans who live around Kenya’s national forests and parks establish nature-based enterprises that preserve, rather than deplete natural resources.39 It does this through three fronts: (i) The Laikipia Biodiversity Conversation program, (ii) support to NRT, and (iii) working with the International Development Law Organization to support policy formulation, implementation and monitoring as a contribution to biodiversity conversation. Activities funded within the USAID’s ENRM portfolio seek to develop incentives and effective structures whereby communities and government entities can conserve forests, biodiversity and manage climate change40. Support to NRT involves implementing community grazing plans and rehabilitation of degraded areas across the northern rangelands.41 NRT’s central purpose is to improve the livelihoods of conservancy communities through wildlife conservation in the rangelands of northern Kenya.42 As such it is concerned with biodiversity conservation, with USAID, one of its principal donors, funding core operations, infrastructure and capacity building in conservancies.43 Through USAID support, NRT has rapidly expanded its constituent conservancy membership from 9 conservancies in 2005 to 27 conservancies in 2015. This model continues to support the creation of new conservancies in response to increased demand from surrounding communities, environmental degradation, and the prevalent threat of poaching, increasing the area of community-owned conservancies in northern Kenya. Consequently, NRT has become a catalyst for the development of community-based conservation initiatives with pastoralist communities. It has embraced the approach of community-governed institutions as forums for sharing successes and experiences, and acts as an advisory and fundraising body for its members. Further, it seeks to maintain the values of its member conservancies, their vision for the coexistence of livestock, wildlife and people, 37 Introduction: Conserving Wildlife in Kenya’s Ewaso Landscape (Georgiadis, 2010) 38 USAID KEA Environment and Naturals Resources Management (Kenya) – Factsheet 39 USAID KEA Environment and Naturals Resources Management (Kenya) – Factsheet 40 USAID Kenya Country Development Cooperation Strategy 2014 – 2018 41 USAID KEA Environment and Naturals Resources Management (Kenya) – Factsheet 42 NRT’s Strategic Plan (2008 – 2012) 43 www.nrt-kenya.org/partners Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 40 and peaceful coexistence of communities.44 NRT uses wildlife conservation as a vehicle for development among the politically and socially marginalized pastoralist communities of northern Kenya.45 1.2.2 Target Areas and Groups NRT’s community development model is driven by communities’ needs and aspirations. The model builds capacity in governance, financial management and security, which promotes independence and long-term sustainability. Its intervention areas are three-pronged: a) Wildlife and Environment Management – NRT assists community conservancies to design and implement community conservation programs, and to improve rangeland conditions, livestock and water management programs. b) Improve and Diversify Income – NRT promotes income diversification through ecotourism and nature-based enterprises to improve livelihoods and increase economic benefit for pastoralist communities. c) Provide Security – the supported community conservancies institute security operations that provide stability for wildlife, residents and visitors in their respective areas. No. County Conservancies 1 Samburu Kalama; Ltungai; Meibae; Sera; Namunyak; Westgate (6) 2 Isiolo Mpus-Kutuk; Leparua; Biliqo-Bulesa; Nakuprat-Gotu; Nasuulu (5) 3 Lamu Pate Island; Awer; Kiunga; Hanshak-Nyangoro (4) 4 Marsabit Jaldesa; Songa; Shurr; Melako (4) 5 Laikipia Naibunga; Lekurruki; Il Ngwesi (3) 6 Tana River Ndera; Lower Tana Delta (2) 7 Meru Ngare Ndare (1) 8 Garissa Ishaqbini (1) 9 Baringo Ruko (1) 44 The Story of The Northern Rangelands Trust (NRT, 2013) 45 NRT Program Description Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 41 FIGURE 1: NRT IMPLEMENTATION AND EVALUATION COUNTIES 1.3 Intended Results Goal Building on the success of an initial USAID grant (2004 – 2008) that strengthened and developed the capacity of Sera Wildlife Conservancy and supported the development of NRT, the purpose of the second phase grant (2008 – 2015) was to enable NRT to scale-up interventions focused on biodiversity conservation, NRM, livelihood improvement, peace-building and community based organization (CBO) strengthening in its constituent areas. Results In order to achieve the goal of improving the livelihoods of conservancy communities through wildlife coservation in the rangelands of northern Kenya, NRT defined the following results areas: 1) Improved management systems and institutional structures of community-based organizations managing conservancies; 2) Improved condition and diversity of biological resources; 3) Increased economic benefits and improved livelihoods for Northern Rangelands pastoralist households; 4) Increased community capacity to resolve resource-based conflicts and improve natural resource governance in the larger NRT region; and 5) Enhanced sustainability of the NRT executive structure and operations. Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 42 Key activity accomplishments NRT’s innovative model of conservation and peacebuilding has resulted in transformational change in the northern arid and semi-arid lands as evidenced by the following indicative outcomes:46  Increasing the area of community-owned conservancies in northern Kenya by 833,933 hectares (ha). By the end of 2012, community conservancies operating under NRT’s umbrella were managing over two million ha. of land and safeguarding the future of a wide range of species, from elephant to Grevy’s Zebra and Hirola to Rhinoceros.  Expanded constituent conservancy membership from nine in 2005 to 27 in 2015.  Community conservancies have improved the lives and livelihoods of more than 280,000 people, by creating more than 500 permanent jobs. They have also secured peace across the rangelands in nine counties.  The conservancy approach has attracted millions of dollars of private sector eco-tourism investment that directly benefits these communities. NRT supported conservancies earn more than $1 million annually from wildlife tourism, livestock investment, and women￾owned bead craft enterprises. The earnings have benefitted more than 58,000 stakeholders through education grants, water projects, employment, and medical care.  Expanded infrastructure development within conservancies including airstrips, roads, schools, hospitals, and water points.  More than 1,300 people received training in NRM and conservation, developing present and future community leaders. 1.4 Approach and Implementation Management and governance capacities of community conservancies were identified by development partners, including USAID as potential areas for improving livelihoods of beneficiary communities. Recognizing the need for an organization to bridge the emerging conservancies’ capacity gaps, Lewa Wildlife Conservancy (LWC), with funding from USAID, began promoting the development of technical competence, organizational maturity and financial stability for these community conservation initiatives. In 2004, the Northern Rangelands Trust (NRT) was established as an umbrella organization with a mission to develop resilient community conservancies that transform people’s lives, secure peace and conserve natural resources.47 NRT began as an outgrowth of the LWC.48 Over a seven-year period (2008-2015), NRT has supported 27 community conservancies in northern Kenya covering 4.4m ha over 11 counties. These conservancies are home to about 450,000 people. Its membership includes community conservancies, private ranches, representatives of county councils and government ministries/agencies. Working with the community conservancies, NRT’s oversight and facilitative strategic leadership is led by a Council of Elders from diverse areas of northern Kenya who are supported by operational staff. To achieve its mission of developing resilient community conservancies that transform people’s lives, secure peace and conserve natural resources, NRT used a number of approaches including (i) partnerships, (ii) incentive provision, and (iii) empowering women as economy-builders. NRT has partnerships with a large portfolio of organizations and stakeholders including:  Principal donors: USAID, The Nature Conservancy, AFD, DANIDA 46 USAID KEA Environment and Naturals Resources Management (Kenya) – Factsheet 47 NRT State of Conservancies Report 2014 48 Mid Term Evaluation of USAID ENRM Projects (dTS, 2012) Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 43  Long term donors: Fauna & Flora International, Zoos South Australia, US Fish & Wildlife Service, San Diego Zoo, Taronga Zoo, Tusk, The Zoo Oklahoma City, Zoos Victoria, Running Wild (Safaricom marathon), International Elephant Foundation, and Saint Louis Zoo  Government of Kenya: The Kenya Forest Service, The Kenya Wildlife Service and The Kenya Police  County Governments: Samburu, Isiolo, Laikipia and Marsabit  Other partners (Foundations and NGOs): Butler Conservation Fund, Lewa Canada & Lewa USA, The Liz Clairborne and Art Ortenberg Foundation, Lundin Foundation, Metcalf Foundation, Millennium Water Alliance, Ol Jogi Ranch Incentive provision for better community-based NRM management encourages well governed community conservancies to benefit from niche markets in livestock trade and establish small-scale enterprises. Through this approach, NRT not only links economic reward to wildlife protection performance, but also reinvests 25% of the mark-up on livestock sales into development activities within the communities. Empowering women as economy-builders within NRT Trading (the commercial arm of NRT) offers a collective marketing strategy with a particular focus on supporting women’s groups through business training to enable them to create and run their own enterprises. C.2. EVALUATION RATIONALE 2.1 Evaluation Purpose The evaluation purpose is to draw lessons to inform future program design and improve implementation of community-based NRM activities. Overall, the evaluation must assist USAID/KEA’s Environment and Natural Resource Management Office to understand what aspects of the activity were most effective in Phase II (2008 – 2015) that could be scaled up and replicated in future programming, and how this could be done. Further, the evaluation seeks to understand the challenges encountered in implementation and what could be done in the future to address them. The evaluation must generate important learning to inform agency and mission management about sustainability, peace-building, governance, and gender and enterprise development within the community conservancy model. 2.2 Audience and Intended Use The key audience for this evaluation is the United States Agency for International Development (USAID)/Kenya and East Africa (KEA). They will use this evaluation to inform future programming. Another key audience is the current project’s technical team, who will use the evaluation to inform the design of possible new interventions and/or revise or modify their current work plan. The USAID/Washington Office of Forestry and Biodiversity within the Bureau of Economic Growth, Education and Environment may also be interested in the broader lessons learned from this evaluation. USAID will share the final report with NRT and its target communities, other donors/funding partners the national and county governments, KWS and Kenya Forest Service (KFS). This audience will likely use the evaluation results to inform future planning for biodiversity conservation. 2.3 Evaluation Questions Evaluation Question 1: To what extent and how has the NRT model improved conservation? In answering this question, the evaluation team must examine: Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 44 a. How NRT’s anti-poaching communications system reduced the potential for crime and threats against wildlife populations within NRT conservancies. This includes any pre-defined systems as well as those that were developed and/or evolved over the life of the activity. b. The effectiveness of NRT’s community ranger network and how this network has contributed to improved surveillance and wildlife protection, reduced banditry and livestock theft, and greater conservation awareness within NRT conservancies and constituent communities. c. How NRT has improved conditions and diversity of biological resources over the life of the project and the factors that facilitated and/or hindered improved wildlife and rangeland health. This includes how well constituent communities have adopted biodiversity conservation practices as a result of the activity. d. The effectiveness of NRT's community conservancy governance model and how it has contributed to improved conservation, peace and livelihoods. e. Any other systems or factors within the NRT model that led to improved conservation across NRT conservancies. Evaluation Question 2: What kinds of drivers and incentives have contributed to NRT’s success and how? The answer to this question, the contractor must focus specifically on the aspects of the activity that were most effective. The question must also explore the target communities’ perceptions about the extent to which and in what ways the activity achieved success. In answering this question, the evaluation team must examine five things in particular: a. Peace dividends b. Economic opportunities c. Improved rangelands d. Women’s empowerment e. Youth empowerment This question must examine how men, women and youth were involved in the activity, and whether different approaches were used to engage them and why. The evaluation must also explore the overall effects of their inclusion and the activity’s contributions to women’s and youth empowerment within the conservancy communities. It must examine the extent to which NRT took advantage of emerging opportunities for the inclusion of men, women and youth. Evaluation Question 3: How has the perception and planning around NRT’s sustainability changed over time and what specific factors influenced the current sustainability plan? The answer to this question, the contractor must focus specifically on the effectiveness of NRT’s original sustainability plans and how they changed over the life of the activity to include the evolving role of county governments. The question must also explore the target communities’ perceptions about the extent to which and in what ways the activity’s effects can be sustained following its closing. In answering this question, the evaluation team must examine three things in particular: a. Strategies for increasing revenue from the county governments b. Strategies for increasing commercial revenues (e.g. scaling up beadwork, ecotourism, livestock￾to-market programs, microenterprise, etc.) c. Strategies for increasing capacity to access donor funding Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 45 C.3. EVALUATION DESIGN AND METHODOLOGY 3.1 Evaluation Design This is a final performance evaluation aimed at understanding lessons learned from Phase II of the NRT activity. The evaluation must inform future USAID/KEA conservation and NRM programming. 3.1.1 Selection of Counties NRT reported a total of 3.184 million ha under conservancy in 2014.49 Of this, over 80 percent of the area is located in three central counties, Isiolo, Laikipia and Samburu, and the northeast county of Marsabit (NRT North East).50 Within these four counties are some of the most established conservancies, yet it is also the area that has seen the greatest growth in total area under conservation management and population served. The livelihoods in this region are predominantly livestock-based and they rank amongst the poorest in Kenya.51 Of the declared 2014 NRT operating budget, 78 percent is earmarked for these four counties. Despite the wide area of rangeland grazing, the land productivity has been in decline and cattle migrations along the many livestock movement routes have triggered conflict. Within these selected counties (Table 1) are conservancies that illustrate relatively high standards of governance measured on NRT’s criteria of accountability, financial management and conservancy operations, while also containing some of the poorest governed. It is anticipated that the evaluation will be able to examine peace and wildlife protection initiatives, in addition to challenges in community conservancy support to securing peace, transforming lives and conserving natural resources. TABLE 1: SELECTED COUNTIES No. County Name Conservancies 1 Laikipia Kalama; Ltungai; Meibae; Sera;Lekurruki ; Westgate (6) 2 Isiolo Mpus-Kutuk; Leparua; Biliqo-Bulesa; Nakuprat-Gotu; Nasuulu (5) 3 Samburu Naibunga;Namunyak ; Il Ngwesi (3) 4 Marsabit Jaldesa; Songa; Shurr; Melako (4) 3.1.2 Sample Selection of Conservancies Conservancy sampling must purposively target a diverse representation of community-owned conservancies within the four evaluation counties. Sampling must take into account key criteria including: age of conservancy, ownership status, operating budget, ethnic composition, size, governance and performance, and flagship wildlife species. Purposive sampling allows for deeper comparative analysis, such that, to the extent possible, major areas of thematic importance including peace, economic opportunities, improved biodiversity and women and youth empowerment are studied. This sample must allow for exploration of the processes behind the results, as well as unexpected outcomes across conservancies with varied composition, focusing on the acquisition of rich detail regarding how and why things did or did not happen in each community. Table 2 provides a summary of key features for each selected conservancy.52 A full list of the 27 NRT member conservancies is provided in Annex VII. A 49 NRT State of Conservancies Report 2014 50ibid 51 ibid 52 Information in this table is compiled from the State of Conservancies Report (2014), The Story of the Northern Rangelands Trust (2013) report, and the conservancy profiles on the NRT website. Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 46 detailed description of each selected conservancy, highlighting the diversity that is represented within the sample, is provided below. Lekurruki in Laikipia County represents one of the conservancies with a high governance score (56)53, and is also among the oldest and most established members of the NRT network. It has a group ranch status54, and a medium size budget relative to other conservancies. Lekurruki is an important wildlife corridor as it surrounded by three other conservancies. In terms of ecosystem, the conservancy is home to one of the largest indigenous forests in East Africa, the Mukogodo Forest, and forest conservation is a key component of the NRM work being carried out in this community. Rangeland rehabilitation is also a core activity within this area, which has long suffered from overgrazing. Despite being largely inhabited by only one community, the Mukogodo Maasai, it has a history of cattle incursions and has experienced several recent violent conflicts with the neighboring Mpus Kutuk conservancy. Lekurruki has actively participated in the Bead Works program, where women from the community have received training in craft-making, product development, basic accounting and leadership, with the support of NRT Trading. Leparua in Isiolo County has a relatively low governance score of 39 and is somewhat new to the NRT network, having been registered in 2011. It has the smallest budget among the sampled conservancies. Leparua is set on community owned land and is highly ethnically diverse, with Ndorobo, Turkana, Somali, Borana and Samburu nomadic and semi-nomadic pastoralist populations inhabiting the conservancy. Competition for grazing pastures among these groups has frequently resulted in conflict in the past, and community inclusion and cooperation is a key component of the ongoing NRM work in this conservancy, particularly where grazing is concerned. Leparua is home to the last remaining populations of Eland in Northern Kenya, a species that has been heavily targeted by meat poachers, as well as the endangered Grevy’s Zebra. It is also an important wildlife corridor, as it connects a number of other community conservancies (Il Ngwesi, Lekurruki, Nasuulu and Mpus Kutuk). The conservancy also hosts tourists and participates in the Beef Works program that links local livestock owners to ready markets. Mpus Kutuk in Isiolo County was established on land owned by the Ministry of Agriculture, Livestock and Fishery’s Livestock Management Division in 2007, a division which no longer exists. This status has created a great sense of anxiety among the communities (Samburu and Turkana) who live and graze on this land, as they fear possible eviction should the Ministry reclaim the land at any point. Further, this conservancy has experienced a variety of challenges related to local politics, community leadership and overall management, and subsequently received the lowest governance score (25), representing the most poorly governed conservancy in the NRT network. Security is a key concern for this conservancy, and rangers from both the Samburu and Turkana communities have been recruited to manage conflict and raise conservation awareness. Mpus Kutuk borders the Ewaso Nyiro River and hosts migratory elephant populations that are heavily threatened by poaching. The community participates in the Bead Works program and also hosts tourists in a few campsites. 53 Governance scores are based on NRT’s metrics for measuring performance in the State of Conservancies Report (2014) and are based on a scale of 0 (poor) to 100 (best). The lowest score awarded to any conservancy was 25, while the highest score was 67. 54 “A group ranch is a livestock production system or enterprise where a group of people jointly own freehold title to land, maintain agreed stocking levels and herd their livestock collectively which they own individually (Ministry of Agriculture, 1968). It is noteworthy that selection of members to a particular group ranch was based on kinship and traditional land rights. The group ranch concept was implemented through Kenya Livestock Development Policy (KLDP) I and II (Kenya Government, 1980)” [source: Food and Agriculture Organization (FAO) http://www.fao.org/Wairdocs/ILRI/x5485E/x5485e0t.htm] Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 47 TABLE 2: REVISED LIST OF SELECTED CONSERVANCIES Conservancy Name Region County Registration date Area (ha) Population (2009 Census) Ownership No. of staff 2014 Budget (KES) Ethnicity Governance Scores Livelihood Key Economic Activities Flagship Wildlife Species Lekurruki NRT center Laikipia 1999 15,872 4,820 Group Ranch 39 7,392,794 Mukogodo Maasai 56 Pastoralist BreadWorks program Elephant, Reticulated, Giraffe, and rare forest species of plants, butterflies and birds Leparua NRT center Isiolo 2011 32,835 8,590 Community Land 19 4,072,714 Ntorobo, Turkana, Somali, Borana 39 Agro￾pastoralist Tourism, Beef/Works program Elephant, Grevy's Zebra and Eland Mpus-Kutuk NRT center Isiolo 2007 91,039 8,416 Ministry of Agriculture, Livestock and Fisheries 23 6,185,921 Samburu, Turkana 25 Pastoralist Tourism, Beef/Works program Elephant, Giraffe, Grevy's Zebra, Lesser Kudu, Cheetah, Eland, Oryx, Lion Namunyak NRT center Samburu 1995 288,228 26,643 Group Ranch and Trust Land 92 23,752,966 Samburu 61 Pastoralist Tourism, Beef/Works program Elephant, Wild Dog, Leopard, rare De Brazza Colobus Monkey Jaldesa NRT North East Marsabit 2013 52,079 13,565 Community Land 13 4,764,675 Borana 39 Pastoralist N/A Elephant, Grevy's Zebra, Buffalo, Giraffe, Leopard, Antelope Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 48 Namunyak in Samburu County is among the oldest NRT members, and one of the first community conservancies established in northern Kenya (1995). It is the third best performing in terms of governance with a score of 61, and has the largest operating budget and staff size of all NRT member conservancies. Namunyak is home to the Samburu, and represents a combination of group ranch and trust land. It includes the Mathews mountain rage, and is home to indigenous forest and abundant wildlife, including a high concentration of elephants. The conservancy has also benefited from a special force, the mobile anti-poaching unit known as 9-1, who are involved in reducing on-going poaching threats and cattle raiding. Further conflict in the area has been driven by cattle migrations from outside the conservancy. Namunyak participates in the Beef Works program and also hosts a high end luxury tourist lodge, Sarara Camp, which brings significant revenue to the conservancy and helps fund community projects. Jaldesa in Marsabit County is among the newest members, having been registered in 2013 when NRT formed three new community conservancies in the county. The conservancy is located between the Marsabit National Reserve and Songa and Shurr conservancies, which make it a critical wildlife corridor for migrating animals, including the endangered Grevy’s Zebra. The area has experienced a variety of conflicts including deadly inter-ethnic tensions among the Borana, Gabbra and Rendille communities, camel and cattle theft, and illegal poaching. A combination of these challenges has resulted in a relatively low governance score of 39. The conservancy is home to diverse landscapes and natural landforms, including ancient sacred and cultural sites, but also suffers from rangeland degradation and destruction of the Marsabit forest. Rangers from the community have been recruited and trained. The community is in the process of exploring alternative livelihoods, which have not yet been established. 3.1.2 Sample Selection of Stakeholders Stakeholder sampling for key informant (KI) and group interviews (GI) will purposefully target relevant stakeholders, beneficiaries and conservancy constituents who have valuable information on the activity or who played a key role in its design and/or implementation. 3.2 Data Collection and Analysis Methods 3.2.1 Data Collection Methods Evaluation data must be gathered from both primary and secondary sources. The evaluation team must review and refine the methodology as part of the work plan development. The evaluation design must use a mix of data collection and analysis methods to generate answers. Following a summary of each data collection and analysis method to be used, Table 2, below, summarizes how each of the three evaluation questions will be answered. Desk Review The evaluation team must review documentation provided by USAID/KEA and NRT on the activity. They must also use online resources and other reports relevant to activity efforts. Secondary data sources will mainly be from activity implementation documents and other periodic progress reports. Other secondary sources will include NRT’s strategy and operational plans and a range of other program documents related to conservation, peace, rangelands and enterprise development. The product of the team’s document review must be an organized presentation of the information found relevant to each of the evaluation questions. The evaluation team must present initial findings to the Kenya Support Project (KSP) internally as part of the Team Planning Meeting (TPM) at the beginning of the evaluation. Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 49 Key secondary documents are summarized here: o Activity Award and Modifications o Activity description document o Annual work plans o Activity M&E Plans o Annual reports o NRT Strategic Plan 2012 - 2017 o State of Conservancies Report 2014 o Environmental monitoring and mitigation reports o Community conservation reports o Conservancy management plans o Conservancy sustainability plans o Wildlife monitoring reports Key Informant Interviews Key informants, in this case, are people who are knowledgeable about the NRT activity in their specific areas of involvement. The sampled participants will provide information on their experiences with and perceptions of the various activity components addressed in the evaluation. These stakeholders are, therefore, relevant to providing information that will guide the evaluation in terms of NRT’s processes and outcomes. The evaluation team must identify key informants based on the document review, key contacts provided by USAID/KEA and information received from NRT. However, Table 3 below indicates the stakeholders proposed for KIIs by location. Key informant interviews (KIIs) with the implementing partners’ staff must target the activity senior management teams, including the Chiefs of Party and/or managers and the staff responsible for key activity components. KIIs must be conducted using a semi-structured interview tool that will be developed as part of the TPM. Group Interviews Group interviews bring three to six individuals together who have a common interest. These differ from key informant interviews in that the discussion often reflects both the individual perspectives and the groups’ consensus on issues. The discussion among group members in response to a question is a key component of the information captured. Sampling for the group interviews must also be purposive, only targeting the conservancy level institutions and constituents who have valuable information on the activity or who played a key role in activity implementation and outcomes. Table 3 below indicates the proposed stakeholder interviews by location and their impact on Level of Effort (LOE). The key below describes the calculations. 3.2.2 Data Analysis Methods Some key approaches to the data analysis that the evaluation will use include the following: Content Analysis The contractor must document narrative responses at a sufficient level of detail to permit a systematic content analysis of these qualitative data. Narrative reviews of interview and discussion responses are expected to provide an in-depth understanding of beneficiary and stakeholder experiences and perceptions. Qualitative data analysis begins with note writing. Each interview will be written up so that it can be shared among team members so that everyone has as complete a picture as possible of all the information obtained. In addition to note writing, the team will hold debriefings (in person and/or remotely) at the end of each data collection week during which they will begin to identify common themes that will be used later for developing the coding framework, which will be used to code the collected data during the formal data analysis process. The team will also use this coding process for the content analysis of all qualitative secondary sources with a focus on the issues most salient to the Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 50 evaluation questions. This will be done first as part of the document review prior to the fieldwork to help identify missing information that can only come from primary data collection. Pattern Analysis From the content analysis, the team must examine interview and discussion notes for patterns to determine whether some responses appear to be correlated with other factors, such as geography, respondent group, age, gender, etc. This process must also identify key responses that were mentioned by more than one person or group. Those that have the highest level of response must be identified as key findings. TABLE 3: ILLUSTRATIVE LIST OF STAKEHOLDER INTERVIEWS (LOCATION AND ESTIMATED DURATION) Stakeholder Nairobi /Calls Laikipia Samburu Isiolo Marsabit Namunyak Leparua Lekurruki Mpus Kutuk Jaldesa USAID Activity AOR/COR 0.5 ENV Team Lead 0.5 Core Implementers NRT 0.5 Lewa Wildlife Conservancy 0.25 NRT Board of Trustees/Elders/Regional Councils 0.5 NRT Trading 0.5 Anti-Poaching Unit 0.5 Working Partners Ol Pejeta Conservancy 0.17 Grevys Zebra Trust 0.17 Ewaso Lions 0.17 Development Partners DANIDA 0.17 The Nature Conservancy 0.17 The Liz Claireborne and Art Ortenberg Foundation 0.17 US Fish and Wildlife Service 0.17 San Diego Zoo 0.17 Tusk Trust 0.17 Agence Francaise Development 0.17 National Government Partners Ministry of Environment and Natural Resources 0.17 Kenya Wildlife Services 0.25 0.25 0.25 0.25 Kenya Forest Service 0.17 0.17 0.17 0.17 Kenya Police 0.17 0.17 0.17 0.17 County Government Partners Land or Environment and Natural Resources 0.17 0.17 0.17 0.17 Tourism, Trade and Enterprise 0.17 0.17 0.17 0.17 Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 51 Stakeholder Nairobi /Calls Laikipia Samburu Isiolo Marsabit Namunyak Leparua Lekurruki Mpus Kutuk Jaldesa Other Kenya Wildlife Conservancies Association 0.25 Conservancy Level Representatives Elders 0.25 0.25 0.25 0.25 0.25 Conservancy Manager/Management Committee 0.25 0.25 0.25 0.25 0.25 Grazing/Livestock/Rangelands Sub-Committee 0.25 0.25 0.25 0.25 0.25 Tourism Sub-Committee 0.25 0.25 0.25 0.25 0.25 Community Rangers 0.25 0.25 0.25 0.25 0.25 Peace committee/coordinators 0.25 0.25 0.25 0.25 0.25 Constituents (Men, Women and Youth) Women 0.25 0.25 0.25 0.25 0.25 Men 0.25 0.25 0.25 0.25 0.25 Youth 0.25 0.25 0.25 0.25 0.25 Days per location 2.95 1.1 0.93 3.18 0.93 2.25 2.25 2.25 2.25 2.25 Total number of days 20.34 Key: 0.17 = KII with half the team - based on 3 KIIs per day; 0.25 = GI with half the team - based on 2 GIs per day; 0.5 = GI with full team Preliminary primary data analysis must begin during the fieldwork so the team can be sure of capturing the information necessary to fully address the evaluation questions. Near the end of the data collection process, the team will conduct an open coding process to identify key themes that emerged in the interviews, beginning with the list of preliminary themes identified during the fieldwork. The key identified themes for each evaluation question will form the basis for the coding framework. The evaluation team will then work in two sub-teams to code the data from a sub-set of all the interview notes. This entails identifying which themes are present in each set of notes. To enhance researcher triangulation, each team member will code the interview data collected by the other team. Once the sub-team members have completed their individual coding, they will compare their coding and agree on how to deal with any differences in perspective. Through this process, the entire team will develop a solid common understanding of the various perspectives that emerged among different stakeholders, which will help ensure that the evaluation findings do not rest on the perspectives of one or two people. Comparative Analysis Results from the pattern analysis based on the document review, interview and discussion notes that emerge will be compared across data type or sources. This approach facilitates both within case (within each stakeholder group and secondary data source) and across case comparisons. In this way key pieces of evidence from the various interviews, conservancies and documents are compared and triangulated to identify the main evaluation findings that respond to the evaluation questions. Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 52 TABLE 4: DATA COLLECTION AND ANALYSIS METHODS FOR THE THREE EVALUATION QUESTIONS Evaluation Question1. To what extent and how has the NRT model improved conservation? Data Collection Methods Secondary Data Group Interviews (GIs) Key Informant Interviews (KIIs) X X X Secondary data from routine activity data collection and reports will yield information on whether NRT’s community conservation model improved conservation. This data was selected as a source because it is available, relevant, and representative of the activity and provides key information on the activity’s accomplishments. These secondary data sources documenting the interventions and results achieved in anti-poaching, community ranger networks, improved surveillance, improved biodiversity and improved community conservancy governance will demonstrate the extent to which the NRT model improved conservation in the second phase of the activity. The documented data will also guide development of interview tools aimed at evaluating the processes through which the activity outputs and outcomes were achieved. To the extent possible, within the limits of time and availability, the team will also review other reports about community-owned natural resource and biodiversity management activities relevant to NRT’s work. This will provide an additional window on the extent to which NRT has improved conservation from an “outsider” perspective, which will balance NRT’s own reporting and others’ perspectives. KIIs will be conducted with key stakeholders including USAID/KEA staff, NRT staff, Lewa Wildlife Conservancy, government partners such as the Kenya Police, KWS, KFS, Ministry of Environment and Natural Resources, DANIDA, The Nature Conservancy, county government partners, etc. All these stakeholders will have relevant information and insights on how NRT’s design and implementation enabled achievement of the activity’s overall goal and objectives around improved conservation. KIIs with the key members of KWS, KFS, Ministry of Environment and Natural Resources, Kenya Police and county governments will provide valuable information on how the activity’s design enhanced the national and county governments’ community conservation approaches, policies, guidelines, systems and resource allocation so as to assess NRT’s contribution to the wider issue of community conservation in northern Kenya (e.g. increased engagement in conservancy planning and development, legislation and by-laws to support community conservation, and active support for community conservancy). Group interviews with conservancy managers, conservancy committees (e.g. peace, rangelands, livestock and peace), and community members will provide information as to how NRT’s community conservation model engaged conservancy staff and constituents in anti-poaching, wildlife protection, biodiversity conservation, governance and peace building (e.g. ranger based wildlife and vegetation monitoring, multi-ethnic peace events, community-led decision-making, mobile response teams). They will be asked about how NRT supported their work and how conservancy staff and constituents benefited from NRT’s model of engagement. Both USAID/KEA and NRT developed the initial concept of using wildlife conservation as a mechanism for achieving the activity’s goal of improved livelihoods among northern Kenya pastoralist communities. Both, but especially NRT and its core partners, were also involved in implementation and regularly monitoring the activity’s performance. They are, therefore, critical informants about the activity’s accomplishments including any deviations in the designs that supported or hindered achievement of improved conservation. GIs with women, men and youth from NRT conservancy communities will provide deeper insight into their knowledge and perceptions of how NRT’s model supported conflicting communities in Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 53 shared management of resources and improved biodiversity conservation practices more generally. These group interviews will further seek to understand how communities responded to NRT’s interventions of creating peaceful and well managed conservancies. Data Analysis Methods Content Analysis Pattern Analysis X X Qualitative data analysis begins with note writing. Each interview will be written up so that it can be shared among team members so that everyone has as complete a picture as possible of all the information obtained by the team. This is important as the evaluation team will work in two separate sub-teams while collecting some of the data. In other words, at least one team member and a facilitator will be present at each interview so that two interviews can be conducted concurrently. The team will hold a debriefing at the end of each data collection day during which they will begin to identify common themes that will be used later for developing the coding framework that will be used during the formal data analysis process. The entire team will meet to debrief in person or remotely at least once a week. After the data collection is completed, beginning with the list of themes compiled during the fieldwork, the team will conduct an open coding process to identify key themes and issues that emerged in the interviews. Once the coding framework has been developed using the list of themes, working independently to enhance researcher triangulation, each team member will code the interviews separately. This process entails identifying what themes and issues emerged in each interview. Once the team members have completed their independent coding, they will compare their coding and come to consensus about how to deal with any differences in perspective. Through this process the entire team will develop a solid understanding of the various perspectives that emerged among different stakeholder groups with respect to the extent to which NRT’s model improved conservation. Of particular importance is identifying the key themes and issues that emerged so as to ensure that the evaluation findings do not rest on the perspectives of one or two people. Nonetheless, divergent and unique perspectives will be investigated to assess whether or not they merit consideration in the analysis. If so, the team will specify why. Once the key themes and issues that emerged in the various sets of interviews and documents are identified, the team will then conduct a pattern analysis, where the perspectives both within and among the various stakeholder groups are compared, and used to map out the patterns that emerge. These data will also be compared with data from the content analysis based on the documents. In this analytical approach, each type of data is analyzed in parallel, and then across data sources. Put another way, this approach facilitates both within case (within each stakeholder group and secondary data source) and across case comparisons. In this way key pieces of evidence from the various interviews and documents are compared and triangulated by data collection method, source of information and researcher to identify the main evaluation findings that respond to the question. Evaluation Question 2. What kinds of drivers and incentives have contributed to NRT’s success and how? Data Collection Methods Secondary Data Group Interviews (GIs) Key Informant Interviews (KIIs) X X X Work, M&E plans and annual reports will provide insight into the various approaches and partnerships NRT developed to provide benefits to conservancy communities, and the extent to which these approaches were successful. This information is critical to understanding how NRT set Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 54 out to achieve its results, which is fundamental to assessing which of the approaches facilitated NRT and which ones may not have been so helpful. These secondary data sources documenting how NRT reached out to men, women and youth to promote economic opportunities, peace-building, and improved rangelands will be helpful in guiding the development of interview tools aimed at evaluating the effectiveness of NRT’s drivers and strategies for community engagement. KIIs will be conducted with key stakeholders including USAID/KEA staff, NRT staff, Lewa Wildlife Conservancy, national government partners such as the Kenya Police, KWS, KFS, and Ministry of Environment and Natural Resources, DANIDA, The Nature Conservancy, county government partners, etc. and others working directly with NRT who have had critical experiences as primary targets of its interventions. They will, therefore, also have relevant information and insights on the drivers and incentives used and the extent to which they worked and delivered results. Moreover, they can provide insight on the factors that may have contributed to the activity’s role in reducing resource-based conflicts, increasing economic opportunities and benefits for constituents (e.g. beadworks, livestock to market opportunities, ecotourism, warrior enterprises, and other business), and engaging pastoralist communities in improving rangeland health. These KIIs are also critical to understanding the extent to which NRT’s partners and constituents are satisfied with the activity’s model, and their perceptions about the strengths and weaknesses of NRT’s approaches to delivering social and economic benefits to communities. Group interviews with representatives from the core implementers (e.g. NRT staff and board of trustees, NRT Trading, Lewa Wildlife Conservancy), community conservancy governance bodies (e.g. Council of Elders, Regional Council, Regional Grazing Committee), and county government partners will provide first-hand information on how NRT’s interventions, such as building capacity of conservancy institutions and enhancing commercial trading, made tangible changes for constituents as far as building peace and security for people and wildlife, promoting conservancy-related economic growth, and preservation and improvement of rangeland conditions. County governments as key stakeholders are also well positioned to provide insights on how NRT’s approaches contributed to the reduction of livestock and wildlife related clashes (e.g. banditry, livestock theft, poaching) and non-violent conflict resolution, and how these approaches were adopted by the county constituents. These informants will also provide valuable information on how NRT has contributed to helping county governments advance their commitment to improving security and livelihoods in the region, and promoting ecotourism and private sector investments in the county. They will be asked what factors facilitated the attainment of NRT results with respect to peace, enterprise development, and rangeland health (e.g. monitoring of grazing resources/forage availability and review of grazing by-laws), and their perceptions of the successes of these initiatives. County governments will further provide information about how equitably community benefits were perceived to be shared, and how successfully NRT partnered with and leveraged county governments in mitigating conflict, accessing resources and increasing revenues. Interviews with Kenya Police, KWS, KFS and the Ministry of Environment and Natural Resources will provide insights on how NRT’s approaches enhanced community policing of natural resources, and built the capacity of conservancy staff and constituents to provide stability for wildlife, residents and visitors to the area. These informants will also be asked about the successes experienced within the conservancies as relates to NRT’s initiatives (e.g. job creation, rangeland rehabilitation, land management, enhanced intelligence/mobile response), and how NRT played a role in managing inter￾ethnic conflict and insecurity. GIs with conservancy committees (e.g. peace, rangelands, livestock and peace), women, men and youth will provide deeper insight into their knowledge about NRT and their perceptions about the success of the activity. As target beneficiaries, they will be asked about strengths and weaknesses of NRT’s approach to diversifying incomes, promoting nature based enterprises, engaging the Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 55 community in environmental management, and improving livelihoods. GIs with community constituents including beneficiaries such as grazing and livestock sub-committees, elders, and community rangers will be critical to understanding the extent to which these constituents are satisfied with NRT’s approaches for livelihood benefits (e.g. improved security of humans, livestock and wildlife, and diversified incomes). GIs with women and youth, in particular, will focus on understanding how women and youth were engaged and supported in peace-building, rangeland conservation, and economic growth. These interviews will further explore specific NRT interventions (e.g. youth moran programs, warrior enterprises, beadworks, scouts/ranger programs) that targeted women and youth, and examine the extent to which these approaches were successful, and how they were perceived among beneficiaries. In addition, these respondents will provide valuable information about how the activity created opportunities for women and youth, either in employment or enterprise development, and how it contributed to their social and economic empowerment. This information will be further triangulated against other interviews with members of the community conservancy management structure (e.g. board of trustees, elders and committees). As with the KIIs, the GIs will yield rich textual data through which key trends and themes can be identified. This information will be triangulated with other sources, particularly the KII data to assess key points of convergence and divergence in perspective among the various stakeholder groups. Data Analysis Methods Content Analysis Pattern Analysis Comparative Analysis X X X These KIIs and GIs will yield narrative content, which will provide in-depth insight to identify key trends and themes. The information generated through these interview data will be triangulated with other sources to arrive at the findings, conclusions and recommendations addressing this evaluation question. Data analysis for this question will follow the same procedures of content and pattern analyses detailed above. In addition, using comparative analysis (for data source triangulation), data from the content analysis based on the document review and interviews will be compared across data type or source. This approach will facilitate comparison within each stakeholder group and secondary data source and across various groups and documents, particularly across the individual conservancies. In this way key pieces of evidence from the various interviews and documents will be compared and triangulated to identify the main evaluation findings that respond to the question. Evaluation Question 3. How has the perception and planning around NRT’s sustainability changed over time and what specific factors influenced the current sustainability plan? Data Collection Methods Secondary Data Key Informant Interviews (KIIs) Group Interviews (GIs) X X X Work plans, M&E plans and annual reports will also provide insight into how NRT intended to promote the sustainability of its interventions. Of particular importance to answering this question are the activity design documents and modifications that outline the approaches NRT intended to use to establish, develop and support governance and institutional structures. This information is critical to understanding how NRT’s sustainability plans evolved, particularly with the role of county governments, which were established in 2013. The documents will also be critical to examining whether the activity design included sustainability planning. The team will review these secondary data sources as a guide to developing the interview tools and questions aimed at evaluating the extent to which and how sustainability plans were adapted over time. KIIs will be conducted with key stakeholders, including USAID/KEA staff, NRT staff, Lewa Wildlife Conservancy, national government partners (e.g. KWS, KFS, Ministry of Environment and Natural Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 56 Resources), DANIDA, The Nature Conservancy, county government partners, etc. and others working directly with NRT, will provide information about how NRT’s design and implementation addressed sustainability. In addition, these KIIs will provide information about how sustainability was defined and conceptualized and how NRT implemented its goal of enabling conservancies to balance donor, commercial and government support. These stakeholders will provide valuable information about the strategies that NRT used to generate direct support from the counties (e.g. Conservancy Livelihood Funds), and the approaches that were used to establish self-sufficient and financially sustainable community conservancies. USAID/KEA, NRT and core partners will also provide insight about factors that influenced the current sustainability plan and the influence of the new county governments in shaping this planning. Interviews with NRT Trading will specifically explore the current model for revenue generation (e.g. Livestock to Markets program, beadworks, and tourism), the success of the support conservancies received in generating commercial revenue, and the factors that have influenced the outcomes of this approach. These informants will also provide valuable information about the extent to which conservancies have been able to diversify natural resource based enterprises, how they have been supported to increase their capacity to access donor funding, and their perceptions about how these initiatives have contributed to sustainability. Interviews with KWS, KFS, the Ministry of Environment and Natural Resources and Kenya Wildlife Conservancies Association will provide insights on how NRT’s approach to sustainability enhanced local capacity to sustain the activity’s effects following its closing. These informants will also be asked about the successes experienced within the conservancies as relates to the strategies laid out by the activity (e.g. ecotourism), and their perceptions about how effectively NRT planned for and prepared the conservancies for self-sufficiency. Group interviews with representatives of the NRT Council of Elders, members of the regional councils, conservancy managers and board members, and conservancy constituents will provide valuable insights into their knowledge and understanding of how NRT planned to sustain its effects after the end of the activity, and to what extent constituents were engaged in the process. They will also provide critical insights into the changes, if any, they observed or experienced as far as seeking county support and promoting natural resource based enterprises. Conservancy staff will provide important information about the extent to which they received support toward enhancing their ability to access donor funding, and their perceptions about this strategy. They will also be asked about emerging lessons on how best to involve community constituents in sustainability planning. GIs with women, men and youth will reveal deeper insight into their perceptions about the extent to which NRT initiatives can be sustained beyond the implementing partners’ exit. As target beneficiaries, they will be asked about strengths and weaknesses of NRT’s approach to diversifying incomes, promoting nature based enterprises, and empowering communities to directly drive conservation efforts. GIs with community constituents including beneficiaries such as grazing and livestock sub-committees, community members, and community rangers will be critical to understanding the extent to which these constituents are satisfied with NRT’s approaches for promoting financial sustainability (e.g. Livestock to Markets program, beadworks and tourism) and other factors that may facilitate or hinder the sustainability of NRT’s effects post-activity. GIs with women and youth will also explore the extent to which NRT took advantage of emerging opportunities to include them in the management of conservancies following the activity’s closing. The KIIs and GIs will yield narrative content that will provide in-depth insight to identify key trends and themes. The information generated through these interviews will be triangulated with other sources to arrive at the findings, conclusions and recommendations addressing this evaluation question. This information will be triangulated with other sources to assess key points of convergence and divergence regarding NRT’s sustainability planning. Content Analysis Pattern Analysis Comparative Analysis Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 57 Data Analysis Methods X X X Data analysis for this question will follow the same procedures detailed above. Analysis will particularly aim to identify how NRT’s sustainability planning changed over time, and how it evolved to factor in the role of county governments, and the extent to which these strategies differed across conservancies. In addition, using comparative analysis (for data source triangulation), patterns that emerge will be compared across data type or source. This approach will facilitate both within case and across case comparisons, particularly across the individual conservancies. In this way key pieces of evidence from the various interviews and documents will be compared and triangulated to identify the main evaluation findings that respond to the question. 3.3 Gender As per Automated Directive System (ADS) 205.3.6.2, all Missions must identify all evaluation questions for which sex-disaggregated data are needed. All people-level indicators must be disaggregated by sex and collected before activities with beneficiaries (or clients) begin (i.e., at baseline) and when activities with beneficiaries end or at the end of the project, whichever comes first (i.e., end line). Missions should also consider whether key evaluation questions examine the extent to which closing gender gaps has improved project outcomes and whether the project has transformed gender norms and reduced gender gaps. Finally, evaluations should identify whether any particular sub-groups (e.g., different ages, persons with disabilities, etc.) are losing out. Good evaluation practice involves:  Establishing interview teams comprised of both males and females appropriate for the cultural context and data being collected, and  Ensuring that samples consist of both men and women as appropriate to the evaluation questions. TABLE 5: EVALUATION QUESTIONS, GENDER DATA AND DIFFERENTIALS Evaluation Questions Sex Disaggregated Data Gender Specific/Differential Effects: Access and Participation Gender Specific/Differential Effects: Results and Benefits 1. To what extent and how has the NRT model improved conservation?  Yes in conservation activities and monitoring data o NRT’s specific support to community conservation, Lewa Wildlife Conservancy, government partners, other partner – DANIDA, The Natural Conservancy etc. Proportion of men and women who participated in conservation activities 2. What kinds of drivers and incentives have contributed to NRT’s success and how?  Yes in conservation activities and monitoring data o Extent to which NRT involved men and women in conservation processes o Proportion of men and women who participated in conservation and livelihood enterprises o Extent to which men and women were involved in and benefited from specific interventions Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 58 Evaluation Questions Sex Disaggregated Data Gender Specific/Differential Effects: Access and Participation Gender Specific/Differential Effects: Results and Benefits 3. How has the perception and planning around NRT’s sustainability changed over time and what specific factors influenced the current sustainability plan?  Respondents data o Respondents’ experiences and perceptions on NRT sustainability o Proportion of men and women who participated in conservation activities o Extent to which men and women utilized conservation knowledge 3.4 Methodological Strengths and Limitations 3.4.1 Strengths The qualitative methods used provide an important platform through which critical information from the various stakeholders can be collected. Moreover, the information will capture perspectives across varying constituent groups that must be compared and triangulated. Combining both primary and secondary data sources through interviews and the desk review provides an opportunity for inclusion of the various perspectives and perceptions into the evaluation thereby enhancing participatory approaches in the NRT evaluation. The use of different data sources further enables the assessment team to triangulate information before identifying findings and making conclusions and recommendation. The selection of data analysis methods allows triangulation between researchers, which further enhances the reliability and validity of the evidence-based findings. 3.4.2 Potential Limitations and their Mitigation As with any assessment or evaluation, there are biases and other limitations that must be addressed through methodological or analytical methods. First, selection bias in the form of the contacts provided by the implementers can mean that the team only hears from people with positive experiences. A wide and diverse respondent pool including national and county government representatives and working partners will offset this. In addition, the team must obtain information from non-project sources, particularly USAID, about key individuals to include. Second, the availability of desired participants may vary and must work with those available at the specified times, which may mean that some gaps in data are unavoidable. The contractor will try to secure interviews in advance to ensure key respondents are reached. Third, since there is no control group for an impact assessment, the evaluation cannot determine with certainty, whether any changes in community conservancy can be attributed directly and exclusively to the NRT activity. The effect of the activity on community conservation can only be measured in nominal terms, especially considering that there were other players supporting NRM during the same period in the same areas of northern Kenya where NRT implemented interventions. A survey of the target population would provide important quantitative information on the effects of the NRT activity on the population and would be triangulated with the information from the qualitative interviews to validate findings and conclusions on NRT’s contribution to changes experienced. The survey was not warranted, Task Order #AID-623-TO-16-00014 USAID/Kenya and East Africa/Kenya Program Support IDIQ Evaluation of the National Resources and Biodiversity Program NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 59 however, considering the level of resources required for such an effort relative to its value addition to the evaluation. Finally, the most effective approach to combating any and all of these biases is to use multiple data sources, data collection and analysis methodologies to triangulate on an assessment question. By combining information found in documents or interviews from multiple sources, any one piece of biased data will not skew the analysis. [END OF SECTION C] NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 60 Annex II: Strategic Insights Alignment of NRT’s Work with USAID Funding Sources and Policies The left column of Table 1 examines how well the NRT model is aligned with the intents of the various sources of funding and USAID policies that Phase III addresses. The rightmost column also indicates ways in which the current model could be enhanced for even greater alignment. All but the gender portion are found to be well aligned. Detailed recommendations are provided in the table to improve gender alignment. NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 61 Linking Evaluation Results with NRT Strategic Planning At the outset of the evaluation effort, the evaluation team asked NRT how the evaluation could be of most use to them moving forward. Top management indicated that they would benefit from an external perspective to identify the key issues to consider in their upcoming 2017-2022 strategic planning exercise. To that end, the team has shared a number of observations with NRT that go beyond – but build on – the evaluation questions. NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 62 NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 63 NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 64 Annex III: Data Collection Instruments Question List for KIIs by Stakeholder Group Development and Working Partners (General) 1. (EQ1d) How well have communities adopted natural resources conservation practices as a result of NRT programs? 2. (EQ1c) How have NRT's programs led to improved or worsened conditions of biological resources? What about diversity of biological resources? What factors hindered improvement in wildlife numbers and rangeland conditions?  General: when thinking about the factors, consider land tenure system (community vs. group ranch vs. govt/crown land), sociocultural factors, institutional factors 3. (EQ1b) In your opinion, what role has the conservancy ranger force played in improving or hindering the success of the NRT conservation model?  General: protecting wildlife, reducing banditry, reducing livestock theft, conservation awareness, livelihoods improvement 4. (EQ1a) Are you aware of the conservancy ranger radio system used in the conservancies? Do you think it has been an effective tool to reduce the potential for crime and threats against wildlife populations within those conservancies? Why or why not? 5. (EQ1a) Do you have any suggestions for how the radio system could be improved?  General: based on other experiences elsewhere 6. (EQ1e) How effective do you think NRT's governance model is in supporting conservation, peace and livelihoods?  General: compare with conservancies outside the NRT system 7. (EQ1f) Is there something else that NRT does that has helped or hindered conservation and access to resources? 8. (EQ2a) We will now discuss 5 important areas of how the conservancy works in the communities. We are going to talk about (see a-e below): For each, we would like to know what parts you thought were successful and why. Interviewer: introduce one aspect at a time NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 65 a. Peace Program b. Economic Opportunities c. Improving the Rangelands d. Women’s Empowerment e. Youth Empowerment 9. (EQ2b) For each of the 5 areas we discussed, were special efforts made to include women, men and youth? What were they? Note: we realize that your understanding of the activity/project may be limited, so this question may not apply to all. a. Peace Program b. Economic Opportunities c. Improving the Rangelands d. Women’s Empowerment e. Youth Empowerment 10. (EQ3) What do you think should be done to ensure that the conservancies are sustainable?  General: revenue from county governments, commercial activities, donor funding 11. (EQ3) How has NRT sustainability planning changed over time? What factors influenced those changes?  General: revenue from county governments, commercial activities, donor funding LEWA 1. Why and how was LEWA involved in the creation of NRT? 2. What were the key components of the USAID grant? 3. Following the creation of NRT how satisfied is LEWA with the end product? 4. What is the current working relationship between LEWA and NRT? 5. How can the model be improved? 6. How has NRT affected LEWA? 7. What are the sustainability plans for both LEWA and NRT? Counties 1. What is the nature of your collaboration with NRT and the conservancies in your county? a. (List them) 2. How do conservancies fit with county efforts to govern communities? a. Governance – Board/Elders, Conservancy Management Committees b. Peace and Security – Rangers, Peace Programs NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 66 c. NRM – grazing, wildlife, water, forestry 3. Would you be pleased to see more conservancies operating in your county? 4. Would you (or are you already) be willing to pay some (more) conservancy costs? a. Operational b. Capital 5. How can conservancies become more sustainable? 6. From where you sit is there anything that conservancies/NRT could do to improve their effectiveness? KWS 1. What is the nature of your collaboration with NRT and the conservancies in you county? 2. How do conservancies support or hinder your efforts to implement the KWS mandate? a. NRM b. Wildlife 3. In your opinion, what role has the conservancy ranger force played in improving or hindering the success of the NRT conservation model? 4. Are you aware of the conservancy ranger radio system used in the conservancies? Do you think it has been an effective tool to reduce the potential for crime and threats against wildlife populations within those conservancies? Why or why not? 5. Do you have any suggestions for how the radio system could be improved? 6. Would you be pleased to see more conservancies operating in your county? 7. How can conservancies become more sustainable? 8. From where you sit is there anything that conservancies/NRT could do to improve their effectiveness? Question List for GIs with Community Stakeholders 1. (EQ1d) What new ways have you adopted to manage your cattle, water, pasture, wildlife and other resources?  Interviewer: note if a practice described does not appear to come from the conservancy model  Interviewer: probe for cattle, water, pasture, wildlife and other resources separately • General: use of natural resources regulations and by-laws • Grazing/Rangeland Committee: charcoal burning, holistic grazing • Conservancy Management: how well management plans are implemented 2. (EQ1c) In what ways has your access to pasture, water and other resources improved as a result of the conservancy? How has it gotten worse?  Interviewer: “access: means improved resource quality and ability to use the resource) NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 67 3. (EQ1c) Have you noticed some types/species of wildlife increasing or decreasing in the last five years? What about new sightings? 4. (EQ1c) What components of the conservancy’s work have brought about the positive changes in access to resources? What components of the conservancy’s work have slowed progress?  General: “components” include peace, tourism, grazing, wildlife, beads, range, social program  Grazing/Rangelands Committee: increasing ability to cope with drought for livestock committee, whether improved security and wildlife numbers have led to greater tourism satisfaction and increased tourism revenue  Conservancy Management Committee: use of information (scientific or traditional) 5. (EQ1b) Has the conservancy ranger force helped the people in the conservancy? Why do you say it has (or it hasn’t)?  General: protecting wildlife, reducing banditry, livestock theft, livelihoods impact, conservation awareness, two-way communication between the communities and the conservancy rangers, extension role for conservancy rangers  Women’s Group: job opportunities  Youth Group: job opportunities 6. (EQ1a) Has the conservancy ranger radio system been useful to you and your community? If yes, how? Interviewer: think about direct and indirect usefulness  General: crime, wildlife, health, governance  Ranger Group: efficiency/effectiveness/productivity/real-time response 7. (EQ1a) Have you noticed if the conservancy ranger radio system has gotten better or worse over time? How can the communications system be improved to better serve the community and conservancies?  Ranger Group: use of technology/equipment/training in telecommunication, designated radio community channel 8. (EQ1e) We will now discuss the effectiveness of conservancy governance. By “governance” we mean, how well the elders, board, conservancy management, committees, staff and rangers work together with the community. NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 68 Overall, how effective do you think the conservancy governance system has been in supporting conservation, peace and livelihoods?  Women’s Group: committee decision making  Youth Group: committee decision making 9. (EQ1f) Is there something else that the conservancy does that has helped improve conservation and access to resources?  General: partnerships/collaborations, social programs 10. (EQ2a) We will now discuss 5 important areas of how the conservancy works with the community. We are going to talk about (see a-e below): For each, we would like to know what aspects you thought were successful and why. Interviewer: introduce one aspect at a time a. Peace Program b. Economic Opportunities c. Improving the Rangelands d. Women’s Empowerment: (beads, micro credit, training, participation in decision making, leadership/responsibilities) e. Youth Empowerment: (micro credit, training, youth, skills transfer etc., participation in decision making, leadership/responsibilities, ranger recruitment) 11. (EQ2b) For each of the 5 areas we discussed, were special efforts made to include women, men and youth? What were they? Note: We realize that your understanding of the activity/project may be limited, so this question may not apply. a. Peace Program b. Economic Opportunities c. Improving the Rangelands d. Women’s Empowerment  Men’s Group: how has the project influenced men to consider women participation in conservancy management  Conservancy Management Committee: inclusion of female rangers e. Youth Empowerment  Men’s Group: how has the project influenced men to consider youth participation in conservancy management 12. (EQ3) What do you think should be done to ensure that conservancies continue to benefit communities for future generations? NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 69 Annex IV: Sources of Information The evaluation team reviewed the following NRT project documents and other related literature as well as several external reports and other information. The main documents consulted are listed below. Award Documents and Modifications  NRT Co-operative Agreement No. AID No. 623-A-00-09-00011USAID Report Oct 13-Sep 14  Technical & Cost Application to USAID for Northern Rangelands Trust Support Program Phase II (Co-operative Agreement: 623-A-005-00010-00) Revised November 14th 2008  Performance Monitoring and Evaluation Plan for Lewa Wildlife Conservancy Northern Rangelands Trust Support Program Phase II – NRT Phase II  NRT Environmental Review Form  NRT Establishing Community Conservancies guideline 2015 Work Plans  NRT work plan –Year 2 (October 2009-September 2010)  NRT work plan –Year 3 (October 2010-September 2011)  NRT work plan –Year 4 (October 2011-September 2012)  NRT work plan –Year 5 (October 2012-September 2013)  NRT work plan- Year 6 (October 2013-September 2014)  NRT work plan –Year 7 (October 2014-September 2015)  NRT Year 3 Phase II Work plan  NRT Operational Plan 2013 Project Reports and Plans  NRT Annual Report Year 1 (December 2008-September 2009)  NRT Annual Report Year 2 (October 2009-September 2010)  NRT Annual Report Year 3 (October 2010-September 2011)  NRT Annual Report Year 4 (October 2011-September 2012)  NRT Annual Report Year 5 (October 2012-September 2013)  NRT Annual Report Year 6 (October 2013-September 2014)  NRT Annual Report Year 7 (October 2014-September 2015)  NRT Annual Report 2013  NRT Centre Operational Plan 2016  NRT Operational Plan 2014  NRT Operational Plan 2015  NRT Strategic Plan 2012-2017 Final Reports  Annual USAID NRT final report year 1-year 7 Evaluation Reports  KWS Final Performance Evaluation Report 2013  MWF Final Performance Evaluation Report 2013  Mid Term Evaluation of USAID ENRM Projects 2012 NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 70  Final Performance Evaluation Report for Community-Based Natural Resource Management and Biodiversity Implemented by the Laikipia Wildlife Forum, 2013 Internal Reports  NRT Elephant Mortality Reports 2009-2015  NRT Wildlife Management Plans and Monitoring Reports  NRT Wildlife CoMMS Documentation  NRT Rangeland Plans and Reports  NRT Peace Program Plans and Reports  NRT Enterprise Development Reports Management and Sustainability Plans  NRT Conservancy Financial Stability Plan 2012-2022  Kalama Conservancy Financial Sustainability Plan 2012-2022  Nakupratt Gotu Conservancy Management Plan 2015-2019  NRT Pooled Conservancy Fund Document Other Reports  Laikipia County Integrated Development Plan 2013-2017  Samburu County Integrated Development Plan 2013-2017  Great Grevy’s Rally Report 2016  Lessons Learned from the Danida Fast Start Climate Change Program (2011-2014), Kenya  USAID Kenya Strategy Statement  NRT Environmental Monitoring and Mitigation Report Year 1- Year 6  NRT Climate Change and Conflict Report 2015  NRT Conservation, Crime and Communities Report 2015  NRT Evaluating the Effectiveness of Community Conservation in Northern Kenya report 2010  NRT Mix of Formal and Traditional Community Institutions in Conservation 2016  NRT The Value of Conservation 2012  NRT State of Conservancies Report 2014  NRT State of Conservancies Report 2015  NRT Gender Analysis Report 2016 NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 71 Annex V: List of Interviewees Key Informant Interviews Name of Interviewee Institution/Affiliation Position/Title Respondent Category Male Female 1 Ann Angwenyi DANIDA Program Manager Development Partner 1 2 Belinda Lows Grevy Zebra Trust Co-Founder & Executive Director Development Partner 1 3 Munira Anyonge The Nature Conservancy Kenya Program Director Development Partner 1 4 Sara Watson Tusk Trust Director of Programmes Development Partner 1 5 Gitahi Nyokabi AFD Development Partner 1 6 Mike Harrison NRT CEO NRT 1 7 Antony Wandera NRT Senior Research and Monitoring Officer NRT 1 8 Frank Bora NRT Chief Finance Officer NRT 1 9 Mike Watson Lewa Conservancy CEO Working Partner 1 10 Mike Gichure KWS Senior Warden, KWS Isiolo County Working Partner 1 11 Peter Hertz Laikipia Wildlife Forum CEO Working Partner 1 12 Patrick Lenawasae Isiolo County Government Tourism Officer Working Partner 1 13 Mathew Leakono Samburu County Government Director, Toursim, Wildlife and Conservation Working Partner 1 14 Joseph Wanyama Samburu Police Senior Sergeant Working Partner 1 15 Mr. Ndolo KWS Samburu Station Deputy Park Warden; Working Partner 1 16 Paul Wambi KWS Marsabit County Deputy Park Warden Working Partner 1 17 Hambule Chupale Marsabit County Government Asst. Director Conservancy Working Partner 1 18 Hawo Abdullahi Marsabit County Government CEC, Water, Environment and Natural Resources Working Partner 1 19 Mark Wanjala Marsabit Central Police Station OCPD Marsabit Central Working Partner 1 20 Jeremy Bastard Sarara Lodge Owner Working Partner 1 21 Dickson Simiren Ole Kaelo KWCA Chief Executive Officer Working Partner 1 Total 15 6 NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 72 Group Interviews Group/Affiliation Nature of the Group Respondent Category Men Women 1 USAID USAID USAID 1 1 2 NRT NRT Team NRT 3 1 3 NRT NRT Trading NRT 4 4 Isiolo County Police Working Partner Working Partners 2 5 Leparua Conservancy Management Committee Conservancy respondents 2 6 Leparua Conservancy Board Conservancy respondents 12 1 7 Leparua Consevancy Rangers Conservancy respondents 11 8 Leparua Conservancy Women Conservancy respondents 13 9 Leparua Conservancy Men Conservancy respondents 13 10 Leparua Conservancy Elders Conservancy respondents 7 11 Leparua Conservancy Youth Conservancy respondents 7 12 Nakuprat Gotu Conservancy Management Committee Conservancy respondents 4 13 Nakuprat Gotu Conservancy Rangers Conservancy respondents 13 14 Nakuprat Gotu Conservancy Board Conservancy respondents 6 15 Nakuprat Gotu Conservancy Elders Conservancy respondents 5 16 Nakuprat Gotu Conservancy Youth Conservancy respondents 9 1 17 Nakuprat Gotu Conservancy Grazing Committee Conservancy respondents 3 18 Nakuprat Gotu Conservancy Women Conservancy respondents 14 19 Kalama Conservancy Rangers Conservancy respondents 5 1 20 Kalama Conservancy Elders Conservancy respondents 10 21 Kalama Conservancy Management Committee Conservancy respondents 4 22 Kalama Conservancy Youth Conservancy respondents 7 23 Kalama Conservancy Women Conservancy respondents 5 24 Kalama Conservancy Board Conservancy respondents 3 2 25 Namunyak Conservancy Management Committee Conservancy respondents 4 26 Namunyak Conservancy Board Conservancy respondents 5 27 Namunyak Conservancy Rangers Conservancy respondents 5 1 28 Namunyak Conservancy Women Conservancy respondents 5 29 Namunyak Conservancy Youth Conservancy respondents 2 1 30 Jaldesa Conservancy Management Committee Conservancy respondents 3 31 Jaldesa Conservancy Rangers Conservancy respondents 7 32 Jaldesa Conservancy Board Conservancy respondents 4 2 33 Jaldesa Conservancy Women Conservancy respondents 9 34 Jaldesa Conservancy Elders Conservancy respondents 8 35 Jaldesa Conservancy Youth Conservancy respondents 2 4 36 Jaldesa Conservancy Grazing Committee Conservancy respondents 7 Total 187 52 NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 73 Annex VI: Disclosure of Any Conflicts of Interest NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 74 NORTHERN RANGELANDS TRUST: FINAL PERFORMANCE EVALUATION 75 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523