MIDTERM EVALUATION OF THE FISHERIES INTEGRATION OF SOCIETY AND HABITATS (FISH) ACTIVITY September 2017 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Sergio Cambronero, Steve Nelson, Dezio Banda, Gracian Chimseu Jim Phillips and Merrill Anne Jordan of International Business & Technical Consultants, Inc. (IBTCI). MIDTERM EVALUATION OF THE FISHERIES INTEGRATION OF SOCIETY AND HABITATS (FISH) ACTIVITY FINAL AID-OAA-I-15-00022/AID-612-TO-17-00001 International Business & Technical Consultants, Inc. (IBTCI) 8618 Westwood Center Drive, Suite 400 Vienna, VA 22182 DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Cover Photo: Fishers in their boats at Lake Chiuta, Machinga. TABLE OF CONTENTS Executive Summary .................................................................................................................................................... 1 Evaluation Purpose & Evaluation Questions ................................................................................................. 11 Project Background ................................................................................................................................................... 13 Evaluation Design, Methods, & Limitations................................................................................................... 14 Approach to TOC Analysis .................................................................................................................................... 20 Findings, Conclusions and Recommendations ............................................................................................. 23 Evaluation Question 1 .................................................................................................................................................... 23 Evaluation Question 2 ................................................................................................................................................... 25 Evaluation Question 3 .................................................................................................................................................... 27 Evaluation Question 4 .................................................................................................................................................... 29 Evaluation Question 5 ................................................................................................................................................... 32 Evaluation Question 6 .................................................................................................................................................... 34 Evaluation Question 7 .................................................................................................................................................... 38 ANNEXES Annex 1. Evaluation Statement of Work ......................................................................................................... 1 Annex II. Evaluation Design and Methods...................................................................................................... 15 Annex III. Data Collection Tools ........................................................................................................................ 17 Annex IV. Evaluation Questions Matrix .......................................................................................................... 26 Annex V. Key Informants Interviewed............................................................................................................. 31 Annex VI. Theory of Change Analsysis ............................................................................................................ 32 Annex VII. Gender Context for FISH ............................................................................................................... 41 Annex VIII. Conflict of Interest Forms for Team Members ................................................................. 44 Annex IX. Statement of Differences ................................................................................................................. 47 ACRONYMS BVC Beach Village Committee CA Christian Aid CDCS Country Development Cooperation Strategy CEPA Centre for Environmental Policy and Advocacy CISER Community Initiative for Self Reliance DEC District Executive Committee DFO District Fisheries Officers DoF Department of Fisheries DQA Data Quality Assessment ECRP Enhancing Community Resilience Program EI Emmanuel International FA Fisheries Association FAO Food and Agriculture Association FISAM Fishing Association of Malawi FISH Fisheries Integration of Society and Habitats FSTAP Fisheries Science and Technology Advisory Panel GoM Government of Malawi KII Key Informant Interview LOP Life of Project MGDS Malawi’s Growth and Development Strategy M&E Monitoring and Evaluation NRM Natural Resource Management PAC Project Advisory Committee PFM Participatory Fisheries Management PMEP Performance Monitoring and Evaluation Plan RF Results Framework SAV Submerged Aquatic Vegetation SEA South eastern Arm (of Lake Malawi) SIRs Sub-Intermediate Results SOW Scope of Work STA Sub-Traditional Authority STAP Science and Technology Advisory Panel TA Traditional Authority TOC Theory of Change ToT Training of Trainers UNDP United Nations Development Program USAID United States Agency for International Development VMS Vessel Monitoring System VNRMC Village Natural Resource Management Committees VSLA Village Savings and Loans Association 1 EXECUTIVE SUMMARY USAID’s Fisheries Integration of Society and Habitats (referred to hereafter as FISH) was specifically designed to align with Malawi’s Growth and Development Strategy (MGDS) and to promote sustainable fisheries for improved livelihoods for communities living around Malawi’s main lakes. The project period is 5 years, from 9 September 2014 to 19 September 2019. FISH is one of the key activities under “Development Objective 2 (DO2): Sustainable Livelihoods Increased” of USAID Malawi’s Country Development Cooperative Strategy (CDCS). EVALUATION PURPOSE FISH has completed approximately 2.5 years; therefore, USAID wishes to evaluate the performance of the project at the mid-point of its five-year program. Consistent with USAID’s 2011 Evaluation Policy (and its more recent updates), the primary goal of this mid-term evaluation is to provide evidence so that USAID/Malawi can determine whether FISH is on track to achieve its desired results or not. Other goals of the evaluation are to inform what course corrections, if any, are needed, identifying specific project interventions that can be scaled up further and to determine best ways to ensure sustainability of the activities, institutions and capacities promoted by the activity. EVALUATION QUESTIONS 1. How effectively are FISH components supporting the project goal of “Increased, social, ecological, and economic resilience of freshwater ecosystems and the people who depend on them.” In particular, given past difficulties with the co-management experience in Malawi, are national and local government officials, local communities and fishers actually working together in a way that contributes to improved management of fisheries resources? 2. How widespread in the target area is knowledge of FISH and its messaging, including on halting use of inappropriate fishing gear such as mosquito nets in fishing, improved methods for fish handling/processing, observance of closed seasons, minimum catch size, mesh size, no-fish areas and sanctuaries, and income-generation activities (e.g., orange-fleshed sweet potatoes, beekeeping etc.). To what extent are the local communities actually using these techniques? 3. How effective have efforts such as campaigns against using mosquito nets, no-fishing zones/time periods, fish sanctuaries, and vessel monitoring systems been for countering over-fishing? 4. How sustainable are the Participatory Fisheries Management (PFM) efforts? Are District Council/GOM officials, traditional leaders and local communities likely to have the political will and funding necessary to continue these programs on their own when FISH ends? To what extent have Participatory Fisheries Management Agreements been advanced with BVCs and their subsequent fisheries associations? PFM has been tested and tried in the past and under FISH but the scale and scope of PFM has been variable. What are the key drivers for successful implementation of PFM in Malawi? 5. How well is FISH integrated with: a) other USAID initiatives (e.g., Feed the Future); and b) other DPs efforts in sustainable fisheries, livelihoods, and climate change adaptation? How successful has FISH been in leveraging assistance from other sources and supporting Malawian efforts to obtain funding from other donors? 6. What if any adjustment could be made to improve project effectiveness? Are there lessons learned that have broader applicability for USAID Malawi and beyond? 2 7. Restriction to access or exclusion of further fishers appears to be one of the workable mechanisms to reduce fishing pressure on the freshwater lake. How feasible would it be for BVCs and District Councils to implement a phased approach of allowing surplus fishers to exit and restricting access for newer fishers? What would be the consequences? METHODS Overall, the mid-term evaluation of the FISH activity assessed the program’s ability to achieve its intended targets, where the main focus of all data collection activities was to answer the evaluation questions formulated by USAID and included in the SOW. IBTCI developed guidelines to register the information collected in the interviews and the design of a matrix that assisted the compilation of data collected from primary sources. Annex 3 contains sample templates of instruments developed for the purpose of this compilation. Central to assessing progress, the Mid-term Evaluation conducted Key Informant Interviews (KIIs) and group interviews. Whenever possible and appropriate, focus groups and field observation of ongoing activities was incorporated to selected site visits. SAMPLING AND LIMITATIONS In an attempt to ensure quality and reliability of evaluation results, the evaluation team covered a balanced sample, but also considered specific social, cultural and demographic aspects contained in the geographic spread covered by FISH. This was reflected in the interviews and site visits scheduled to engage beneficiaries/stakeholders, including individuals, academic, governmental, NGO, civil society (community based and grassroots) and diverse private sector entities. Although sampling tracked the scope and nature of the activities undertaken by FISH under the four components, where some activities encompassed larger numbers of participants, the evaluation activities achieved a balance between individual and group interviews. Group interviews ranged from 8 to 10 participants to enable productive interview environments that encouraged meaningful participation and contributions from participants. The evaluation team selected the sites and KIs were based on information, records and publications provided by FISH, including records pertaining to pilot site characteristics, annually reported target accruals, factoring anticipated biases and other limitations that could skew the data collected. Specific biases referring to recall, response and selection were noted and considered in the analysis of the data collected. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS KEY TAKE AWAYS Summary of the FISH project. The FISH project made significant progress in the first 2.5 years. The PACT consortium of NGO partners worked well across four lake communities to implement projects related to fishery and watershed management and adaptation to climate change. FISH collaborates with the University of Rhode Island (URI) to conduct research and provide scientific data to support better natural resource management decision-making. Participatory Fishery Management. FISH works to implement Participatory Fishery Management (PFM), or co-management, by Beach Village Committees (BVCs) to monitor fishing activities in local communities. These efforts are off to a good start and BVCs have developed constitutions and By-Laws to manage fishery and coastal resources. However BVC sustainability will depend on GOM approval and enforcement of By-Laws, adequate sources of funding, and continued training. 3 Governance. FISH conducts an effective governance program that informs and influences important stakeholder groups. Overall, the governance effort has been effective in influencing stakeholders in BVCs, District Councils, and national government. By providing strong advocacy in the last 2.5 years of the project, FISH will improve chances for sustainable co-management. Program Design. By integrating climate change into the program design, FISH adds agriculture and forestry activities to those for fishery management. In this context, FISH represents more of a coastal zone management program rather than a strictly fishery management project. Assuming declining funds for climate change, FISH should emphasize biodiversity conservation as an over-arching theme. FISH may consider branding itself as an integrated coastal zone management program supporting biodiversity conservation and economic development. African freshwater co-management. In a larger context, FISH represents an experiment in fishery co-management for freshwater African lakes. Given the social and economic features of African communities, can local communities manage their own fisheries? Comparable co-management programs are underway in Tanzania and other places. With FISH support for BVCs, co-management shows promise for being a successful model. However concerns remain regarding the sustainability of BVCs after completion of the FISH project. Therefore, FISH should focus on long-term sustainability in the last 2.5 years of the FISH project. FINDINGS FOR EVALUATION QUESTION 1 SYNOPSIS: The four project components (tracking the four outputs of the RF) have worked reasonably well together to achieve initial project goals. There are clear associations between the intended increased access and understanding contributed under Output 1(science, analysis and information) and the enabling environment sought under Output 2, which in turn advances change at the district and community levels. Evidence supports both mainstreaming and replication intended under Outputs 3 and 4 in direct association with new knowledge and an enhanced scientific foundation afforded Output 1. Both the strengthened fishery sections in the District Development Plans and support BVCs and VNMRCs (constitutions, by-laws and management plans) are at the heart of the contribution story rendered under this question. There is also strong evidence that depicts science, analysis and information as instrumental to capacity building of individuals and institutions, contributing to both improved governance structures and community assets. 1. Evidence of progress includes significant milestones achieved over the first two and half years, including: capacity building in local communities; relevant and timely research findings and; good publications. 2. Comparisons of allocations for the four program components suggest relatively large amounts of resources dedicated to research (Component 1) compared to on-the-ground implementation and livelihood development activities (Components 3 and 4). However, activities anticipated by the Pact Malawi team for Years 4 and 5 seem to heavily rely on inputs from Component 1 with direct overlap into Components 2, 3 and 4. 3. There is evidence of progress includes revamped and/or created BVCs and FAs and strong political will to support PFM at community, district, and national levels. KIs in all four local government districts recognized FISH efforts to complete socio-economic profiles that were used to develop District Development Plans with strengthened fishery sections. There is evidence of coordinated institutional work supported by FISH including national and subnational level actors. 4 4. The Fisheries Science and Technology Advisory Panel (FSTAP), an anticipated major contributor to the TOC condition of “ shared, evidence-based objectives and learning” has been slow in fully engaging its role as a “ participatory clearing house mechanism for defining and prioritizing applied science needs” and assumed to be critical to FISH’s exit strategy. 5. At the local level, FISH technicians and Fisheries/Forestry extension agents are working together in planning and implementing FISH project activities. The vast majority of FISH and Fisheries/Forestry staff interviewed agreed that collaboration was best demonstrated by their shared use of resources and collaborative training. CONCLUSIONS RELEVANT TO EVALUATION QUESTION 1 1. FISH has met significant program milestones, strengthened BVCs, and advanced PFM at national, district, and community levels. Some institutional links require strengthening, such as better communication between FISH and DoF and more engagement with Traditional Authorities. 2. FISH components are adequate to achieve the project goal and support the theory of change. “Component 1. Utilization of science” takes up a large percentage of program resources and accounts for almost 50% of the total budget. Accordingly, the anticipated leveraging of Component 1 in Years 4 and 5 towards outputs and outcomes that crossover to the remaining three components seems an effective way to assist the ability of FISH to achieve a higher concentration of activities on the ground. 3. There is visible collaboration among implementing partners, Government and local assemblies to implement FISH projects and activities and to achieve program outputs and outcomes. However, important knowledge gaps and hindrances to efficient data and information flows seem to be prevalent in key stakeholders across the local, district and central levels. There are also important lags in achieving the full activation and contribution of FSTAP, as a key GoM partner at the central level, assumed to remain and expand beyond the FISH LOP. RECOMMENDATIONS RELEVANT TO EVALUATION QUESTION 1 1. FISH should consider: a) developing appropriate advocacy efforts to influence fishery management policy at higher levels of central government; b) reviewing the research strategy; and c) shifting more resources to “boots-on-the-ground” implementation project. FINDINGS FOR EVALUATION QUESTION 2 SYNOPSIS: FISH messages have been effective throughout the project area. Local communities heard these messages mostly through personal contact with FISH Technicians and DoF extension workers. Local communities have implemented practices to reduce illegal fishing and to start new income generation activities, such as honey production and fish processing. Knowledge about the FISH activity, including its objectives, was clearly articulated by both, key stakeholders and community level beneficiaries. At the local level, the work undertaken by the FISH technicians (in collaboration with Fisheries/Forestry extension agents) seems to have addressed the shared learning. Messages clearly overlap topics associated with Outputs 3 and 4 and their corresponding sub-outputs, where evidence points to their dissemination as instrumental to increased widespread awareness. The outputs that drive the contribution story under this question speak to content (focus on practices and income generation) and outreach (targets of dissemination). No substantial evidence was found on actual changes happening through the midpoint of FISH, affecting the density or patterns of application of improved techniques and practices. 1. Knowledge about the FISH activity, including its objectives, was clearly articulated by primary sources, including key stakeholders and community level beneficiaries, across the different community and 5 institutional settings (four districts) that were visited. While key stakeholders included staff from Department of Fisheries, District Councils, Department of Forestry, MALDECO Fisheries and the FISH activity, Community beneficiaries included BVC members, Village Natural Resources Management Committees (VNRMC), bee-keepers, and Village Savings and Loan (VSL) groups. There is evidence of exchange based on messages (source/user collaboration) among key stakeholders at various levels, including the Departments of Fisheries and Forestry, BVC and Natural resources management committees. 2. Across numerous interviews where respondents often used local language, evidence of knowledge introduced by FISH demonstrated a clear understanding of core concepts, including an integrated approach that links lake and catchment ecosystems to fisheries management. KIs confirmed that messages are disseminated through various channels; but most effectively through direct interaction with FISH and DoF technicians. Primary sources offered evidence of important differences and contrasts in the skill sets and professional backgrounds among the FISH Technicians recruited by CISER, WESM, and EI. CONCLUSIONS FOR EVALUATION QUESTION 2 1. FISH messages are adequately and widely disseminated in the project areas and some of these messages have been adopted in practice. However, it is difficult to know whether these messages will lead to widespread adoption of best fishing practices and technologies. 2. Illegal commercial trawlers present threats of overfishing and adverse impacts on the lake ecosystem and aquatic biodiversity. Although DoF licenses the commercial trawler fleet, enforcement remains weak. RECOMMENDATIONS RELEVANT TO EVALUATION QUESTION 2 1. FISH should continue messaging and advocating to reduce illegal fishing; these campaigns should include advocacy and workshops at higher government levels. To reduce fishing efforts, FISH should support the development of alternative sources of income such as those from honey production, fish processing, and possibly aquaculture. These efforts should be coupled with an increase focus on enforcement. FINDINGS FOR EVALUATION QUESTION 3 SYNOPSIS: FISH campaigns to stop illegal fishing have changed perceptions in communities and illegal practices may be declining. However, DoF Frame surveys describe high numbers of illegal, under-meshed nets, especially used as gill nets set in Lake Malawi. Several communities have established sanctuaries and no-take zones to protect critical habitats and they implement land use practices to limit erosion and maintain fish breeding areas. Commercial trawlers present another dimension of illegal fishing. To address this problem, Vessel Monitoring Systems (VMS) have been tested, but not yet required to obtain commercial trawler licenses. In relating the combined effect of outputs to the three conditions for change assumed by the FISH TOC, most positive or negative changes assessed to specific contributions can be cross-referenced to the three TOC conditions, where significant intersections between FISH achievements (associated with each of the four outputs) involving or assisted by dissemination and promotional efforts such as. A good example of this contribution story would be the faster uptake of improved practices detected in enhanced catchment areas (addressed by FISH in tandem with community engagement) has been related a multiplier effect from the use of combined technologies (evidence-based objectives and learning). 1. Most BVC (82%) and fisher’s respondents believe that illegal fishing continues in Malawian lakes, however, most respondents believe it is decreasing. Illegal fishing identified by respondents includes fishing during closed season, mesh size violations (mostly under-meshed mosquito nets), and fishing in 6 restricted areas by commercial trawl fishermen. Secondary sources (DoF Frame Survey) show increasing illegal fishing activity. 2. Local communities on all three lakes know about and/or support sanctuaries and no-take zones, recognizing their importance in protecting fish stocks. Several VNRMCs described the importance of tree planting and forest protection to prevent soil erosion in order to help lake fish stocks. BVC members spoke to their collaboration with DoF and Traditional Authorities to patrol waters for illegal activities and confiscate illegal gear. Vessel Monitoring Systems (VMS) have been tested and they are required as a condition in the license application form. However, implementation delays have been experienced due to procurement requirements. 3. FISH, with CEPA support, proposed a revenue sharing scheme for the fishery sector, but it has not been implemented in the GoM. KIs across the three lake ecosystems demonstrated variable degrees of familiarity with and participation in the FISH program. Interviews with DoF Regional Fishery Officers and District commissioners in all four districts revealed that they have endorsed the new By-Laws, but did not elaborate on an approval timeframe. Fishing communities around Lake Malawi made greater progress implementing BVCs; as measured by progress on the six-step process relative to VNRMCs in agricultural communities around Lake Chilwa and Lake Chiuta. CONCLUSIONS FOR EVALUATION QUESTION 3 1. FISH campaigns to stop illegal fishing practices have reduced some improper activities, but illegal fishing continues. Stronger and greater enforcement is required to stop illegal fishing, and under current laws effective enforcement requires the cooperation of BVCs, DoF, and Traditional Authorities. 2. The FISH project aims to implement an integrated coastal ecosystem conservation strategy that includes the lake and the surrounding catchment extending 10 km from the shore. The community understands the importance of protecting these coastal habitats and some BVCs and VNRMCs are currently implementing and managing protected areas. RECOMMENDATIONS RELEVANT TO EVALUATION QUESTION 3 1. FISH should continue to strengthen capacity for fishery co-management. Specifically, it should ensure the prompt approval and enforcement of BVC By-Laws and advocate for stronger control of the commercial trawl fleet. FINDINGS FOR EVALUATION QUESTION 4 SYNOPSIS: With FISH support, PFM has advanced in local communities and district governments. Sustainability depends on BVC empowerment based on enforceable By-Laws, reliable sources of incomes, continued training, and strong political will among all stakeholders. To date, the work undertaken by FISH towards better ensuring PFM sustainability pivots around Output 2 and in particular, the Sub-output of Institutional and community capacities strengthened is reflected in FISH efforts towards enabling better structured stakeholder contributions and efforts to increase the operational resources of BVCs, Fisheries Associations and District Councils in support of improved co-management. Although it is possible to identify significant causal linkages between outputs and outcomes that relate to an increased prospect for sustainability of PFM achievements to date, evidence collected and reviewed does not yet build a complete argument for PFM success. FINDINGS FOR EVALUATION QUESTION 4 1. Evidence was provided by primary sources at various levels suggesting strong political will to support PFM among BVCs, local governments in all four target districts, and in DoF. Specifically, KIs expressed support Participatory Fisheries Management (PFM) as a way to improve fishery management in Malawi and believe that most, but not all, Traditional Authorities support PFM. 7 2. In terms of factors that could contribute to sustainability KIs noted the legal formalization of BVC By￾Laws and the need for sustainable funding for BVCs, alongside equitable revenue sharing between central and local governments. When asked about possible drivers of PFM success, KIs referred to both the approval and enforcement of BVC By-Laws. However, some respondents further elaborated on the importance of building community ownership when advancing PFM initiatives, highlighting the need to factor contextual differences among the multiple lake settings and how these should guide a sensitive engagement of target communities. 3. Echoing reflections shared by FISH staff, KIs also spoke to the need to effectively pair the advancement and approval of BVC By-Laws with more productive relationships with Traditional Authorities, possibly better linking them to new emerging spaces for discussion and exchange hosted under the collective BVC mandates promoted by rising Fishing Associations. On a parallel possible course of action, secondary sources suggest that FISH has effectively leveraged new knowledge (e.g. new/improved technologies and productivity) that addresses special interests across multiple stakeholder groups. CONCLUSIONS FOR EVALUATION QUESTION 4 1. Illegal commercial trawlers present threats of overfishing and adverse impacts on the lake ecosystem. Although DoF licenses the commercial trawler fleet and is responsible for enforcement, enforcement remains weak. FISH, through FSTAP plans to hold meetings with DoF and Fisheries Association of Malawi (FISAM) to make the case for mandatory membership to FISAM for all commercial stern and pair trawls fishers. Nonetheless, weak policies at higher levels of central government may continue to hinder or weaken critical commitments such as mandatory membership to FISAM to ensure trawlers’ compliance to license conditions. 2. PFM has reasonably good political support from central and local governments and community based organizations, responsive to a decisive and readily acknowledged interaction with FISH at all levels. In this regard, FISH evidences a strong grasp of the inherent complexity of the multiple stakeholders at various levels that must work together to successfully implement PFM. Gender is also at the core of this complex collaboration, where cultural and social norms, supported by community-based governance may not be entirely aligned with the economic contribution of women and its critical importance to sustainability. 3. FISH has advanced PFM by strengthening political will in local governments, building capacity in local communities, and helping to develop BVC By-Laws. However, sustained future building of PFM success depends on equitable revenue sharing among central and local governments and empowered BVCs to be able to co-manage their coastal resources. Longer term PFM sustainability depends on equitable revenue sharing among central and local governments and BVC empowerment based on approved and enforced By-Laws. RECOMMENDATIONS RELEVANT TO EVALUATION QUESTION 4 1. To enhance sustainability, FISH should focus more on income generating activities in local communities, advocate for more equitable sharing at national levels and prompt approval of BVC By￾Laws at the District level. Moreover, FISH should design an exit strategy the leaves behind functioning institutions, such as the FSTAP. FINDINGS FOR EVALUATION QUESTION 5 SYNOPSIS: FISH activities integrate with complementary USAID projects such as PERFORM in forestry. FISH forestry and agricultural activities overlap with similar projects from development partners. The various outcomes under each of the four output areas identified in the FISH Results Framework appear to relate to integration that is either content-driven or based on geographic focus and co-location. While 8 content-driven integration naturally aligns with efforts under Output 1, given FISH’s prominent work on the ground through extension services, geography and co-location track opportunity and is, of course, sensitive to both settings and actors. Therefore, location specific collaboration with other initiatives is mostly driven by the focus of other initiatives in addressing threats (Output 3), and/or supporting adaptation (Output 4). Visible synergies in planning and implementation efforts led by FISH can be readily linked to human resource readiness (extension work involving FISH Technicians and DoF agents) that is tasked with a sound contextual understanding of challenges and opportunities that (shared evidence￾based objectives). Likewise, FISH is in a position to continue building on knowledge assets that are shared across the portfolio USAID, as well as initiatives of other development partners (shared evidence-based learning). 1. FISH has been able to integrate both its content development and site interventions with either the planning efforts, human resources or knowledge assets developed by multiple USAID initiatives. To this end, there is an obvious and natural overlap between FISH’s goal and the contributions driven by its results logic and those of other USAID activities, particularly in the case of PERFORM, Pamawa, WALA and other DP supported ECRP. FISH also has leveraged capacity development previously supported by USAID to strengthen its on-the-ground presence. Collaboration was also noted with upper catchment activities implemented by NJIRA and the IDRC-funded Nsomba ndi Chuma Project. 2. KIs from District Councils identified initiatives such as the USG-funded DREAMS initiative, as well as the Local Development Fund Initiatives and the Lake Malawi Basin Project funded by the World Bank as examples of coordinated interventions that targeted complementary outcomes to those pursued by FISH. Likewise, it has tapped into human resources and on-the-ground experience previously developed by the Wellness in Agriculture for Livelihoods Advancements – WALA (also linked by EI) and supported the recruitment of FISH Technicians. FISH overlaps with other development partners implementing comparable biodiversity and watershed programs, which include FAO-GEF, UNDP-GEF and the Lake Malawi Basin Programme. 3. There is also visible evidence of opportunities opened by overlapping geographic coverage and technical focus. In this respect, FISH has moved forward in better leveraging geographic overlaps with PERFORM’s containment and restoration efforts, where KIs noted a number of strategic and operational advantages. Along the lines of overlapping sites and geographic focus, KIs referred to the Enhancing Community Resilience Program - ECRP with significant coincidences in its focus on food security, reduced vulnerability and strengthened resilience in selected districts that are most prone to natural disasters and climatic hazards. Moreover, CEPA’s role as a technical partner on policy and advocacy as well as knowledge and information, provide ideal grounds for synergies in the mobilization of public sector support aligned with FISH’s work across multiple coincident districts as well as within Central Government. CONCLUSIONS FOR EVALUATION QUESTION 5 1. Although there is evidence of meaningful content-centered collaboration, much of the integration efforts engaged by FISH are based on geographic focus and co-location to avoid redundancies. However, there is high potential for enhanced integration of FISH with USAID–funded PERFORM, particularly in terms of income generation and broader ecosystem impact. 2. FISH has successfully engaged selected watershed management improvements (control of erosion, ecosystem protection and restoration), opening an opportunity for knowledge sharing/transfer and coordinated action with initiatives (bilateral or multilateral) that can converge in addressing existing gaps in comprehensive biodiversity conservation. 9 RECOMMENDATIONS RELEVANT TO EVALUATION QUESTION 5 FISH may consider closer integration with PERFORM focused on income generation and broader ecosystem management. Moreover, FISH may want to emphasize knowledge sharing/transfer when considering coordination with other initiatives (bilateral or multilateral) that target existing gaps in comprehensive biodiversity conservation. FINDINGS FOR EVALUATION QUESTION 6 SYNOPSIS: Implementation activities related to bee-keeping and fish processing need more business management training, marketing expertise, and access to capital. There are some gaps related to biodiversity conservation; especially in considering biodiversity hotspot and climate vulnerability in selecting project sites. BVCs and local communities asked for low-cost equipment like boots, gloves and life jackets to promote a safe workplace. When considering adjustments to improve effectiveness, the FISH results logic clearly supports a desirable shift from the initial emphasis on sharply improving the knowledge base for decision-making and program action to a more immediate and direct threading of science, analysis and information to the advancement of the PFM model in the new economic and governance context of improved fisheries management (Output 2) and the strengthening of the fisheries value chain. Accordingly, the previous combined dynamics hosted under the four FISH areas of output is expected to continue to feed into the three TOC conditions but their focus will now be on direct action, seeking the increased agility of the improved governance/management structures that favor an ecosystem-scaled delivery. Evidence suggests that this is expected to translate into District action and resourcing plans supported by strategies for better integrated use and management. 1. Although there is evidence of community awareness of the value of conservation practices, secondary sources point to significant gaps in advancing a coherent and comprehensive biodiversity conservation strategy that includes the lake and catchment area. Peer-to-peer engagement of target populations were noted by primary sources as important and strategic drivers of initial change that yields opportunity. Along this same rationale, respondents described their needs for low-end equipment to support patrol operations or engage new income generation activities (bee-keeping and fish processing) as opportunities to build more tangible and immediate means to effectively convey and promote prospective longer-term gains. 2. In considering adjustments to improve effectiveness, a central element to consider is the role and scope of activities considered under Output 1, which in turn, may leverage its higher budgetary allocations to priorities identified for Years 4 and 5 of the FISH implementation. Drawing from both primary (FISH senior staff) and secondary (planning documents) sources, these priority tasks cover specific realms across the four program components. 3. At its midpoint implementation, FISH is in a position to strategically facilitate both, the effective flow￾down of these enhanced technical capacities to the key actors of co-management at the local and community level (LFMA), as well as in fostering effective and sustainable feedback loops that directly support decision-making. There are important and persistent knowledge gaps across the different implementation settings that could critically impact any sustainability/exit strategy adopted by FISH during its final implementation. 4. Also linked to a visible need for sustained and strategic feedback, there is a perceived need (emphasized by primary sources at multiple levels) for brokering opportunities at high levels of authority (Central Government) to promote important policy shifts that are relevant to the success of fisheries co-management and specifically the PFM model implemented by FISH. 10 CONCLUSIONS FOR EVALUATION QUESTION 6 1. Overall, FISH implementation has been successful. Despite early program changes, the PACT consortium of NGOs has coordinated work plans, implemented field projects and reported activities reasonably well; as evidenced by achievement of program milestones and targets. There are some important and persistent challenges that are inherent to its complex implementation and the coordination of numerous partners (and their consortia). However, the implementation model also lends a unique opportunity to build on and expand further integration that secures both FISH results and its exit strategy. 2. Some KIs believe that FISH, now in its third year, might shift emphasis more toward on-the-ground implementation (Outputs 3 & 4) rather than research and planning (Output 1). Moreover, to verify program effectiveness, FISH research findings from Output 1 should be clearly linked to management decisions and/or advocacy campaigns. FISH needs to more effectively address significant gaps in the advancement of a coherent and comprehensive biodiversity conservation strategy covering the lake and catchment area. 3. Fish processors and honey makers have developed their products and now seek to market and sell them. At this point in the program, FISH appears well-positioned to consider a sharper focus on income generation activities, such as improved community financing options and more business management training. Likewise, effective integration with other initiatives and development partners may be instrumental to added entrepreneurial capacity and access to financing that better assures sustainable (upkeep and expansion) operations and growth. RECOMMENDATIONS RELEVANT TO EVALUATION QUESTION 6 1. FISH should review its research plan and make sure that scientific findings support management decisions. Consider using Chambo and Usipa stock assessment and fishery management plans to review options for limited entry fishing. As a way to boost political will, FISH should shift more resources toward on-the-ground implementation and support BVC and VNRMC members to continue to build their presence and visibility in the target lake settings. FINDINGS FOR EVALUATION QUESTION 7 Restricting access to the fishery represents one way to reduce fishing pressure. But it will be difficult for BVCs to manage fishery restrictions under current circumstances, especially related to: a) problems resulting from placing new restrictions on a previously open access fishery; and b) imprecise enforcement mandates among BVCs, DoF, and TAs sharing the same area. Findings related to the challenges: 1. Skepticism. District Fishery Officers (DFOs), local government representatives, and some implementing partners expressed skepticism over the ability of BVCs to restrict access to only licensed fishers. 2. Open access fishery. Currently, Malawi maintains an open access fishery and BVCs do not have a strong mandate and enough political power to restrict numbers of fishers operating in their jurisdictions. 3. Legal mandate. Several DFOs pointed out that limiting fishery access would be the responsibility of Traditional Authorities and DoF, not BVCs. 11 Findings related to opportunities: 1. Phased approach. Informant BVCs maintain lists of registered fishers and they request “transfer letters” form migrating fishers. This represents a first step to license the fishery. 2. Alternative livelihoods. Several informants described the lack of alternative livelihoods coupled with rapid population growth as factors to diminish the feasibility of establishing a limited entry fishery. In any case, one of the most important consequences of restricting access to fishing would be the need to create alternative livelihoods in other sectors. CONCLUSIONS RELEVANT TO EVALUATION QUESTION 7 1. Restricting access of small-scale fishers will face resistance both from the fishers affected and from opinion/community leaders, possibly influencing the necessary support from local authorities. The absence of viable alternative livelihood for excluded fishers presents a major hindrance to restricting access. 2. There are weak local controls and monitoring operations of fishing activities. Fishery management must consider the complex social web of fisher communities encompass and understand inherent conflicts with Traditional Authorities and cultural beliefs. Nonetheless, BVCs and District Councils may be able to implement a phased approach to restrict fishers. But it would take continued training, patient advocacy and sustained political will to make it happen. RECOMMENDATIONS RELEVANT TO EVALUATION QUESTION 7 FISH should advocate for restricted access as best practice of sustainable fishery management. Develop messages to explain the benefits of a limited entry fishery and describe the adverse consequences of continuing an open access fishery. Develop stock assessments and prepare fishery management plans for Chambo and Usipa as described in the Year 3 Workplan. Consider introducing fishery limits (or restricted access) as elements of Chambo and Usipa fishery management plans scheduled for Year 3. Get feedback and build consensus. EVALUATION PURPOSE & EVALUATION QUESTIONS EVALUATION PURPOSE The primary goal of this mid-term evaluation is to provide evidence so that USAID/Malawi can determine whether FISH is on track to achieve its desired results or not. Other goals of the evaluation are to inform what course corrections, if any, are needed, identifying specific project interventions that can be scaled up further and to determine best ways to ensure sustainability of the activities, institutions and capacities promoted by the project. There are three major objectives of this evaluation. They are as follows: 1. Review, analyze, and evaluate the effectiveness of the FISH activity in achieving program objectives and contributing to USAID/Malawi’s efforts to mitigate climate change and adaptation in Malawi; 2. Evaluate major constraints in achieving expected project results; 12 3. Provide specific recommendations and lessons learned on strategies and approaches USAID/Malawi should pursue in the remaining period of FISH’s implementation and for future fisheries and climate change adaptation activities. Consistent with USAID’s 2011 Evaluation Policy (and its more recent updates), USAID intends to determine whether the findings, recommendations, and lessons learned from this mid-term evaluation were taken into account during the remainder of the implementation period, and its impact on learning and informed adaptation (as per ADS 201). Finally, this evaluation will provide strategic options for future USAID/Malawi engagement in fisheries and offer lessons learned with wider applicability to other activities in Malawi or beyond. EVALUATION QUESTIONS The following are the principal seven evaluation questions that will be addressed by IBTCI: 1. How effectively are FISH components supporting the project goal of “Increased, social, ecological, and economic resilience of freshwater ecosystems and the people who depend on them.” In particular, given past difficulties with the co-management experience in Malawi, are national and local government officials, local communities and fishers actually working together in a way that contributes to improved management of fisheries resources? 2. How widespread in the target area is knowledge of FISH and its messaging, including halting the use of inappropriate fishing gear such as mosquito nets in fishing, improved methods for fish handling/processing, observance of closed seasons, minimum catch size, mesh size, no-fish areas and sanctuaries, and income-generation activities (e.g., orange-fleshed sweet potatoes, beekeeping etc.). To what extent are the local communities actually using these techniques? 3. How effective have efforts such as campaigns against using mosquito nets, no-fishing zones/time periods, fish sanctuaries, and vessel monitoring systems actually been in countering over-fishing? 4. How sustainable are the Participatory Fisheries Management (PFM) efforts? Are District Council/GOM officials, traditional leaders and local communities likely to have the political will and funding necessary to continue these programs on their own when FISH ends? To what extent have Participatory Fisheries Management Agreements been advanced with BVCs and their subsequent fisheries associations? PFM has been tested and tried in the past under FISH, but the scale and scope of PFM has been variable. What are the key drivers for successful implementation of PFM in Malawi? 5. How well is FISH integrated with: a) other USAID initiatives (e.g., Feed the Future): and b) other DPs efforts in sustainable fisheries, livelihoods, and climate change adaptation? How successful has FISH been in leveraging assistance from other sources and supporting Malawian efforts to obtain funding from other donors? 6. What if any adjustment could be made to improve project effectiveness? Are there lessons learned that have broader applicability for USAID Malawi and beyond? 7. Restriction to access or exclusion of further fishers appears to be one of the workable mechanisms to reduce fishing pressure on the freshwater lake. How feasible would it be for BVCs and District Councils to implement a phased approach of allowing surplus fishers to exit and restricting access for newer fishers? What would be the consequences? 13 PROJECT BACKGROUND Brief Country Context Malawi, a landlocked country in Southern Africa, has had its fisheries resources under economic and environmental threats. Both forestry and fisheries resources have experienced intense overexploitation due to ever increasing population which currently is at 17.9 million. Fisheries is one of the important sectors for economic growth, food security, employment and improvement of people’s livelihoods in Malawi. Capture fisheries prevalent in the five main water bodies of the country, including Lake Malawi (24,208 km2), Lake Malombe (390 km2), Lake Chilwa (1,800 km2), Lake Chiuta (200km2) and lower and upper sections of Shire River (820km2) (FAO, 2005). However, these fishery resources are experiencing enormous pressure, among others, due to high and unsustainable exploitation rates and climate variability. The major habitats providing fish are Lakes Malawi, Malombe, Chiuta, Chiuta and other rivers such as Shire River. The fisheries sector is one of the main sources of employment, food and income to over 1.6 million Malawians. The sector contributes about 4% to the country’s Gross Domestic Product (GDP), 40% of the total protein supply for Malawians; and provides employment to over half million people (Annual Economic Report, 2016). The Malawi fisheries has experienced considerable decline with catch reduction fluctuating from an average of 60,000 metric tons in in the mid-seventies to close to 49,000 by 2003. The fisheries of Lakes Malawi, Malombe, Chilwa, and Chiuta provide employment for 60,000 fishers and another 450,000 individuals involved in fish processing and domestic trade. They also contain 15% of global freshwater fish biodiversity. However, these fisheries are under considerable stress from a growing population, over￾fishing, and environmental degradation, complicated by the effects of climate change. Brief FISH Background USAID’s Fisheries Integration of Society and Habitats (FISH) was specifically designed to align with Malawi’s Growth and Development Strategy (MGDS) and to promote sustainable fisheries for improved livelihoods for communities living around Malawi’s main lakes. The project period is 5 years, from 9 September 2014 to 19 September 2019. FISH is one of the key activities under “Development Objective 2 (DO2): Sustainable Livelihoods Increased” of USAID Malawi’s Country Development Cooperative Strategy (CDCS). FISH has completed approximately 2.5 years; therefore, USAID wishes to evaluate the performance of the project at the mid-point of its five-year program. FISH seeks to address the drivers of over-fishing and degradation by improving mechanisms for local co-management of fisheries, using the following four main components: Component A1. Utilization of science, analysis, and information for decision making. This component focuses on gathering information on Malawi fisheries and making it available on a wider scale through a database; research to understand the current threats faced by the industry and options to improve conservation; and obtaining a better understanding of the effects of climate change on Malawi’s lakes and potential fisheries management options to increase resilience to those climate impacts. Component B2. Enabling environment for conservation and management of freshwater ecosystems enhanced. This component focuses on ensuring an enabling legal framework for sustainable fisheries management and biodiversity conservation; greater transparency, representation, and accountability in 14 decision-making with regard to fisheries; and building institutional and community capacities for shared fisheries management (co-management) between local communities and local/national authorities. Component C3. Priority threats to freshwater ecosystem biodiversity reduced. This component is focused on addressing key threats to fisheries ecosystem biodiversity conservation by implementing best practices in the three interlinked livelihood areas listed below: a. Sustainable natural resources management and agriculture in the catchment. b. Fishery habitat management and riparian zone conservation. c. Sustainable fishing in targeted biodiversity ‘hotspot’ areas. Component D4. Adoption of climate change adaptation measures that support resilience of communities and freshwater ecosystems increased. This component focuses on developing and disseminating strategies, methods, and information that will allow vulnerable individuals and communities to become increasingly resilient in the four target lakes and associated catchment ecosystems. Under D4, FISH will provide viable, climate smart and more environmentally friendly and diversified livelihood practices by promoting the adoption of best practices in CCA identified and tried. An Intensive ecosystem approach services packages for fisheries management and climate smart agriculture will be established in six catchment areas. EVALUATION DESIGN, METHODS & LIMITATIONS The contractual period of performance for USAID’s Fisheries Integration of Society and Habitats (FISH) activity runs from September 9, 2014 to September 9, 2019. Accordingly, the FISH mid-term evaluation covered the activity’s implementation from the start of the Program in FY 2014 through approximately the end of the second quarter of FY 2017, accounting for one-half of total implementation under the present contract. The evaluation design considered an initial phase for Start-up, Design and Field Preparations, followed by a second phase covering the Fieldwork and Data Analysis and a final phase for the development of the evaluation report. The projected time period for the evaluation activities in the field (32 days) that will enable substantive exchanges with USAID on the process and findings, as well as a final stakeholder debrief. A team of four professionals (including two local consultants) conducted all in-country evaluation activities throughout this period, where feedback to USAID was supported by weekly reporting, a preliminary draft report and a final debriefing before departure from Malawi. The Scope of Work approved by USAID, included in Annex 1. As described in Table 1 below, the evaluation process covered initial consultations and discussions with USAID and key FISH staff currently involved in the implementation to enable the evaluation team to identify and prioritize sources and best understand the opportunities or constraints that each potential source represents in terms of the evaluation purposes and questions. In this respect, the design and programming of the field activities considered opportunities and constraints to directly observe FISH approaches under each of its four components, covering outcomes that reach stakeholders and beneficiaries in four key ecological freshwater lake ecosystems of South-East and South-West Arms of Lake Malawi, Lake Malombe, Lake Chiuta and Lake Chilwa, spread over four target districts: Mangochi, 15 Balaka, Zomba and Machinga. Annex 4 provides a detailed description of the team’s approach to crafting a response to each of the seven evaluation questions included in the SOW. Table 1: Evaluation Approach: Challenges/Opportunities First Stage Second Stage Third Stage Review of secondary (document) sources In-country data collection and preliminary data analysis Production of Final Evaluation Report 1. Initial review of context/program documents; 2. Document identification and review; 3. In-brief with USAID and kickoff with FISH; 4. Identification/contacting of evaluation sources and KIs; 5. Finalized schedule of visits; 6. Finalized data collection tools. 1. Field observation and data collection; 2. Ongoing data integration and preliminary analysis; 3. Data triangulation and final facts-checking; 4. Debrief to discuss preliminary findings and enable feedback; 5. Production of draft report; 6. Final in-country debriefing. 1. Final data analysis and final drafting of findings, conclusions and recommendations; 2. Production of draft report; 3. Review, comment and approval of draft report. Challenges/Opportunities in Implementation  Secondary sources and background documents were expanded to cover added inputs to the team’s understanding of both context and process.  Field visit schedule was adjusted and expanded based on emerging opportunities.  Draft and final report production will benefit from advance/added comments and feedback included in the joint discussion of preliminary findings with IP participants. DATA COLLECTION METHODS Overall, the mid-term evaluation of the FISH activity assessed the program’s ability to achieve its intended targets, while closely examining the quality and reliability of the data collected to date. The main focus of all data collection activities was to answer the evaluation questions formulated by USAID and included in the SOW. IBTCI developed guidelines to register the information collected in the interviews and the design of a matrix that assisted the compilation of data collected from primary sources. Annex 3 contains sample templates of instruments developed for the purpose of this compilation. The evaluation team developed a standardized format to register the information collected across all interviews, directly coding all information relating to the evaluation questions and sub-questions from the semi-structured questionnaire by sub-question and thematic keywords. The information was compiled on a daily basis in an Excel matrix (Annex 3) that assisted the targeted review of coverage and the identification of information gaps. The approach of systematic ongoing compilation was also instrumental in enabling the Team Leader to track and follow, often remotely, the progress and depth of data collection activities, while attending to a parallel and separate USAID evaluation. Data drawn from a set of additional interviews (mainly context or topic-driven through a targeted approach) was enabled by splitting the team into a Lilongwe versus target beneficiary sites, has also been incorporated into the review and analysis of project reported data. Any particularly useful data on FISH approaches has been highlighted to assist recommendations that may best support reflection on progress, after-action review and informed adaptation, extensive to future USAID program design. Primary data was collected from a total of 166 individuals via individual and group interviews and focus group discussions (Annex 5). DATA COLLECTION TOOLS The mid-term evaluation tools used to collect and analyze information from beneficiaries were developed to establish useful linkages between responses and the main evaluation questions. This core focus of the research on the evaluation questions was primarily supported by the semi-structured individual interview protocols used in a one-on-one setting (Annex 3). A standardized template associated to the main 16 research questions (as provided by the SOW), and a predetermined set of keywords enabled the team to code responses on an ongoing basis. However, the originally proposed semi-structured approach for the Group Interviews and the related use of questionnaires was not deemed feasible. SAMPLING AND LIMITATIONS In terms of the characteristics of the sample reached by the data collection activities, the evaluation team sought to cover a balanced sample, but also considered specific social, cultural and demographic aspects that are contained in the geographic coverage covered by FISH. See Annex 5 for lists of the key informants interviewed. Sampling tracked the scope and nature of the activities undertaken by FISH with variable numbers of participants when using a group format. The selection of sites and KIs was based on information provided by FISH, including the team’s review of pilot site characteristics, annually reported target accruals, considerations of possible biases and other limitations that could skew the data collected. Associated logistics and deployment distances were an important factor in the final selection. DATA AVAILABILITY The evaluation team encountered some limitations related to data availability. None of the sites visited or KIIs enabled additional quantitative data or access to locally kept records. In addition, hindrances related the application of mini-surveys or questionnaires limited additional triangulation across the scheduled field visits. Despite multiple efforts to reach some KIs, the evaluation team was not always successful in interviewing some individuals (particularly in Lilongwe), translating into time lags or, in some cases, unavailability. Nonetheless, based on the large number of individuals/groups (50 or 110 including individual farm club members, as explained above), the evaluation team believes that its findings accurately reflect the focus of FISH-supported activities. BIAS CONSIDERATIONS The evaluation team anticipated that it would encounter some recall bias, given the two-year plus period elapse and some instances of turnover in public sector counterparts. Although KIs consistently make a clear distinction between FISH activities and previous non-USAID implementation, there is natural overlap in the cause and effect recall, particularly with actors at the higher levels of central government. Likewise, some inputs offered by key informants may have been affected by response bias. Some key informants, Data Analysis Document Review Start-up, Design & Evaluation Report Field Preparations Field work and Data Analysis Evaluation Work Plan Data Collection Instruments KI Interviews & Site Visits Drafting & Review of preliminary findings Findings, conclusions and recommendations 17 predominantly at the village level, seemed to deliberately avoid direct criticism or controversy regarding project implementation. Conflict or opposing views involving different institutional groups, personalities, and special interests also influenced the responses offered by primary sources. This trend appears to reflect the fairly common concern among stakeholders about sustaining donor funding. While such perspectives could have impacted some of the feedback offered, the evaluation team found that public sector officials were able to point to specific project outcomes that provide a legitimate basis for their assessment of FISH interventions. The field visits also used numerous questioning techniques and opportunities to view interventions, to triangulate and confirm feedback that appeared slanted. Finally, selection bias is an inherent risk when implementers help to facilitate contact with project beneficiaries, as they may select the most active, responsive, or engaged beneficiaries—meaning that the evaluation team may only hear from key informants who report positive experiences. Typically, an evaluation team could observe and speak with a wider range of beneficiaries by attending project activities or using snowball sampling during the fieldwork period—asking key informants to themselves identify other potential key informants in their network. To mitigate selection bias, the evaluation team attempted to filter responses to interview protocol questions to avoid evident slant, as well as, to the extent possible, triangulated responses collected from various data sources. GENDER CONSIDERATIONS To the extent possible, the evaluation team assessed the consideration of gender across the implementation of its planned activities in the field. This was done considering the relevance of gender norms and power relations specific to the contexts (geographic, institutional, communal, etc.) engaged by the FISH activity. Whenever feasible, the evaluation team focused on observing female participation and any visible role in decision-making across different implementation contexts. Based on available annual reporting, the team checked for gender disaggregation in the breakdown of data reported on indicator targets. Additional consultations were conducted with gender experts, familiar with the contexts engaged and challenges faced by the FISH activity to enhance the ability of the evaluation team to factor gender considerations in to the findings. Annex 7 presents a brief summary of the salient points of these consultations. OVERVIEW OF PMEP TARGETS AND DATA QUALITY In assessing FISH’s advancement at its mid-term point, the evaluation team has endeavored to make the most feasible and useful comparisons between progress reported (PMEP-based tracking) and on-the– ground observations where USAID-funded outputs were examined utilizing semi-structured interviews with target beneficiaries and other key stakeholders. Consistent with USAID’s own internally evolving process (ADS 201) performance indicators should assist the measurement of outcomes that are relevant and significant. Accordingly, at the activity level we look for relevance reflected in their direct link to the activities specific logic (theory of change) and conclude on their significance based on their contribution to a chain of results that is consistent with its purpose. Supported by this rationale, the evaluation team choses to highlight a subset of eight indicators that best relate to the findings presented in this report and also appear most useful when examining the contribution story under each of the evaluation questions, as described in the previous section. Despite the data quality challenges faced by FISH and briefly discussed below (trained individuals and use of improved technologies/practices), a quick glance at the PMEP reflects extraordinary progress in core areas, including the areas under improved natural resource management and the adoption of laws, policies and regulations. Gender analysis has been supported by sex disaggregation across all applicable indicators. Accordingly, Table 2 provides a useful summary of the selected indicators, the measured change as reported in the FISH PMEP (updated 2017) and the implications that more readily relate to the Theory of Change (TOC) as presented in Annex 6. In this respect, across the eight indicators the main drivers of change are capacity building and improved information resources, technologies and practices 18 consistent with the four outcomes and associated sub-outcomes of FISH’s Results Framework (RF). Some of the positive differences registered across the PMEP tabulations readily associate to outcomes reported across the midpoint of the activity’s implementation. These outcomes have thus been linked to FISH contributions in the TOC comparisons included in Annex 6. Table 2: Performance Indicators relevant to the TOC Comparisons Indicator 2016 Target 2016 Result Change to PMEP MTE Observations Number of hectares of biologically significant areas under improved natural resource management as a result of USG assistance. 0 248,985 No target NRMs visited in all the four districts and selected field observations of woodlots and regeneration plots support results. Number of institutions with improved capacity to assess or address climate change risks supported by USG assistance. 101 95 Target short by 6% Capacity building described by implementing partners and beneficiary groups (BVCs, NRMs, VSLAs) highlights significant progress. Number of people with improved economic benefits derived from sustainable natural resource management and/or biodiversity conservation as a result of USG assistance. 5,973 9,744 Target exceeded by 63% There is evidence of economic activities established, including value additions to fish processing; bee keeping; village saving loans and generation of fees/fines by fisheries associations. Number of farmers and others who have applied improved technologies or management practices with USG assistance. 9,189 20,153 Target exceeded by 219% Use of drought resistance crops such as improved rice variety was observed being used around Lake Chiuta and Chilwa. Number of users accessing FISH website and databank for information on fisheries best practices in BDC & CCA. 250 704 Target exceeded by 281.5% An online repository with useful search functionality, which is currently available to both students and researchers. 0 Indicator 2016 Target 2016 Result Change to PMEP MTE Observations Number of laws, policies, regulations, or standards addressing climate change adaptation formally proposed, adopted, or implemented as supported by USG assistance. 99 52 Target exceeded by 100% Fisheries and NRM by-laws were made available and their drafting is complete, now awaiting approval by district councils. Number of FISH CBOs (i.e. BVC, FA, etc.) actively participating in co￾management of fisheries. 127 128 Target made Focus group discussions showed commitment, knowledge and a focused vision of targeted achievements and how they will achieve them. Number of people trained in sustainable natural resources management and/or biodiversity conservation as a result of USG assistance. 4,770 7,347 Target exceeded by 154% Focus group discussions with participant institutions evidenced capacity built and an expressed perception of empowerment to achieve their goals in natural resources management. A recent Data Quality Assessment (DQA) laid out recommendations based on assessed deficiencies, which the technical staff are reportedly advancing on the ground. FISH has reported and also openly discussed challenges faced with data collection on the ground under partner oversight and conducted primarily by FISH Technicians at the community level that populate a group register managed by the secretary of a Community Based Organization. Inconsistencies in the translation of the register tools presented important variations from one partner to the other. Gaps in daily data entry tasks and overall staff performance, compounded by poor or insufficient partner oversight significantly impacted data quality. Despite complete and detailed PIRS, showing sufficient details in terms of data collection and applicable sex and age disaggregation, the DQA detected important deficiencies such as double counting (e.g. individuals trained in NRM and/or climate change activities and those applying farming technologies). Moreover, the initial use of data collection software presented significant retrieval challenges and its eventual transition to a more reliable mobile solution to support digital data collection undertaken by FISH partners. Accordingly, the move to iFormbuilder software is expected to correct past deficiencies, enabling systematic and easily verifiable data entry and standardized reporting across partners. Pact Malawi has supported corrective actions with a focused collaboration with partners to ensure consistent data quality up to the present period, including partner contribution to data cleaning and testing to enable the sound updating of the PMEP. Data collection streamlining were underway at the midpoint of implementation have emphasized feedback and guidance provided to partner staff to achieve necessary reporting improvements. At the time of the evaluation team’s review of the PMEP, these intended enhancements to the data collected and reported were highlighted in reference to specific indicators for which the original tabulations have been retraced to source data in collaboration with FISH partners and now reflect verified data for FY17. Corrective actions encompass the addition of dedicated M&E staff at the partner level of operations, which enhances oversight and data testing at the data collection sites and then flows through strengthened partner reporting protocols. Data will be populated into 1 partners’ indicator tracking table as partner M&E Officers concurrently conduct data quality checks and analysis for consolidation into the project PMEP indicator results table. This data subsequently flows into the DevResults platform after review by Pact Malawi M&E staff. As described by Pact Malawi, enhanced data quality assurance is expected as a result of an in-depth review and adjustment of M&E activities, tools (references to a consensus document), and roles that support the revamped operations overseen by a recently recruited FISH M&E Specialist based out of Lilongwe. Nonetheless, central to this purpose is the commitment to consistent mentoring, resources, and feedback for partner M&E Officers. APPROACH TO TOC ANALYSIS In addressing the approved SOW and its stated evaluation purpose, the FISH midterm evaluation enabled a purposeful, albeit limited, snapshot of what the activity had set out to accomplish through the first half of its LOP. Guided by the evaluation questions posed by USAID, the evaluation team set out to collect data from primary and secondary sources that would contribute to the formulation of useful answers. Subsequent to the in-country data collection phase, the team rapidly organized the evidence gathered under each one of the seven questions included in the SOW for initial consideration, both within a discussion of preliminary findings with USAID, which included feedback from the implementing partner, as well as in the more structured format of a draft report that included conclusions and recommendations. Based on the feedback provided and responsive to a subsequent request by reviewers that included contributors from USAID and FISH, IBTCI has incorporated an additional dimension to the final structure of the evaluation report. This added dimension is offered in the form of analytical tables (Annex 9) that examine the implications of the evidence collected to FISH’s theory of change, for each of the evaluation questions. The intention is to further enrich the analytical value of the findings and enhance the utilization potential of the team’s conclusions and recommendations. BRIEF CONTEXT FOR THE CONSIDERATION OF THE FISH TOC FISH addresses USAID/Malawi’s Country Development and Cooperation Strategy (CDCS) goal, “to improve Malawian’s quality of life”, notably the CDCS Objective DO 2: “Sustainable Livelihoods Increased,” and Intermediate Result 2.1, “Resiliency to Climate Change Strengthened”. It targets Sub￾Intermediate Results (SIRs) “institutional capacity improved”, “policy and systems strengthened” and “positive behavior adopted”. Like most implementation contexts, FISH is built around the four primary freshwater ecosystems of the South arm of Lakes Malawi, Malombe, Chilwa, and Chiuta, a context where the accurate determination of attribution is not feasible within the parameters and limitations assumed by the evaluation SOW. When introducing questions of cause and effect, we first address the fundamental determination of attribution, meaning: can we isolate and estimate accurately the particular contribution of an intervention? Attribution refers to ensuring that causality runs from the FISH THEORY OF CHANGE If decisions around fisheries management… (1) are based on shared, evidence-based objectives and learning, (2) are grounded in inclusive and effective ecosystem-scaled governance structures, and (3) strengthen the assets of communities, Then Malawi’s complex and diverse freshwater lake ecosystems can be sustained 2 intervention to the outcome. 1 . Moreover, recognizing the significant context overlap of activities implemented by various development partners, contribution analysis provides a much more credible and useful assessment of cause and effect. In addition, while examining the theory of change adopted by FISH within the limitations of a retrospective desk review, and whenever applicable and feasible, the evaluation team also considered other external factors that could influence the outcomes, based on reasonable evidence about its contribution. APPROACH TO TOC COMPARISONS Developed as a final analysis stage, the evaluation team considered useful (albeit limited) comparisons between observed and/or documented outputs and outcomes, and the causal logic that supports the FISH theory of change. This final consideration was based on the available evidence, which was gathered or reviewed by the team during its in-country data collection activities (including secondary sources provided by the FISH team). Any causality inferred from the available evidence assumes that the key assumptions documented in these secondary sources (reviewed as evidence of implementation) through the first half of its LOP were sound, plausible, and agreed upon by the main stakeholders. Therefore, evidence collected by the evaluation team constitutes a valid (although partial) examination of the expected chain of results. Consistent with the limitations stated in this report, the evaluation team has also assumed that any and all significant contributions associated to outside factors (not directly resulting from FISH interventions) have been recognized by KIs and included in periodic project reporting. Accordingly, these comparisons reflect on the evidence collected by the evaluation team, following the six steps presented in the table below. To the extent possible and applicable, the TOC comparisons offered under six of the evaluation questions (compiled under Annex 6) track FISH outcomes and corresponding sub-outcomes identified in the Results Framework addressed in its Performance Monitoring and Evaluation Plan (PMEP). Given the nature of the seventh evaluation question and its specific probing of the feasibility and consequences of two potential solutions to overfishing (restricted access or exclusion of fishers), its value in rendering causal linkages associated with the evaluation findings was not considered as useful as that of the other six research questions. In addition, the evaluation team recognized a certain degree of redundancy and overlap in some                                                               1 Non IE Guidance on Impact Evaluations, Addressing the Attribution Problem, World Bank Group, 2014.   1. Examine evidence collected: What evidence is collected directly applies to the occurrence of the various results? 2. Determination apparent links: What are the apparent links of the evidence collected to the theory of change that also need to be assessed? 3. Consider assumptions and risks: What evidence was available on the assumptions and risks behind these links? 4. Consider the strongest links: Which links are best represented by the available evidence, strongly adhering to the logic, and seemed to have wide acceptance among stakeholders? 5. Consider the weakest links: Which links appear weaker as per the available evidence, suggest weaker logic, or less convincing agreement among stakeholders? 6. Consider external factors: What evidence was available about other identified influencing factors and the contribution they made? 3 of the evaluation questions and thus has chosen to strategically group some of the findings supporting TOC comparisons following their most useful contribution to render the causal sequence of FISH’s three noted TOC conditions: (1) shared, evidence-based objectives and learning; (2) inclusive and effective ecosystem-scaled governance structures, and, (3) strengthened assets of communities. The initial logic snapshot provided in a table format under each evaluation question (Annex 6) enables a comparison of evidence (primary and secondary sources) available that supports the horizontal association of the three TOC conditions in relation to each one of the four output areas engaged by the FISH activity. Although in some cases contributions noted follow a deliberate overlap between different outputs (e.g. science generated under Output 1 but reflected under Output 3 or 4), this is meant to illustrate complementary or supplementary relationships within FISH that support its theory of change. In this respect, evidence selected for the purpose of developing the tables included in Annex 6 is meant to be illustrative and does not imply that no other FISH contributions may be applicable. It is also important to note that, in the case of the evaluation questions one through five, the horizontal sequence for each output primarily reflects actual results reported and summarized by FISH that are useful in rendering its performance, consistent with the results logic under each of the four outputs considered in the FISH Results Framework. In the case of the evaluation question six, the horizontal sequence in the table also considers planned or anticipated FISH contributions associated with results under each of the four outputs. A subsequent corresponding table constitutes the contribution story that the evaluation team has chosen to highlight under each of the six evaluation questions considered. As such, each contribution story is intended to complement and expand on the analytical summaries that have been included in the main document as a lead into the recommendations offered by the evaluation team. Aligned under the four different outputs, this table also provides observations on both negative factors that may affect FISH contributions and challenges that may impact the achievement of objectives. However, it is important to keep in mind that the relative weight of each story to the overall TOC will vary depending on the end use of the findings and the kinds of decisions that will be based on them. Since the evaluation SOW offered no specific direction in terms of a focus on causality in direct association to the main evaluation questions, the evaluation team chose to address the most representative and practical cross-section of arguments that could be drawn from the evidence collected and reviewed. In practical terms, this means that the assumptions guiding the comparisons to the TOC regarding the nature, extent and difference 4 made by FISH’s contributions may have variable relevance and applicability across each of the six evaluation questions considered for comparison that were also included in the approved SOW. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS This section discusses the findings from the review of primary information gathered from interviews with key informants, meetings, and site visits. To complement findings based on primary sources, a brief synopsis of the performance that tracks the results logic and incorporates secondary sources is included under each of the seven evaluation questions defined in the Scope of Work. Likewise, a synopsis of the contribution story, also tracking each question explores the causal linkages that may be evident between selected outputs and outcomes. In the case of the Evaluation Question 7, the evaluation team opted for building a direct response to the topic-specific scenarios (not specifically aligned with the original RF) posed by the question. Therefore, the noted findings and conclusions relate to FISH outputs and outcomes that, as observed in the field, best provide evidence to substantiate the response sought by USAID. For consistency, Question 7 omits a contribution story since it is not involved in TOC comparisons. EVALUATION QUESTION 1 How effectively are FISH components supporting the project goal of “Increased, social, ecological, and economic resilience of freshwater ecosystems and the people who depend on them.” In particular, given past difficulties with the co-management experience in Malawi, are national and local government officials, local communities and fishers actually working together in a way that contributes to improved management of fisheries resources? TOC SYNOPSIS FOR QUESTION 1 Results Logic: The four project components (tracking the four outputs of the RF) have worked reasonably well together to achieve initial project goals. There are clear associations between the intended increased access and understanding contributed under Output 1(science, analysis and information) and the enabling environment sought under Output 2, which in turn advances change at the district and community levels. Evidence supports both mainstreaming and replication intended under Outputs 3 and 4 in direct association with new knowledge and an enhanced scientific foundation afforded Output 1. Contribution Story: Both the strengthened fishery sections in the District Development Plans and support BVCs and VNMRCs (constitutions, by-laws and management plans) are at the heart of the contribution story rendered under this question. There is also strong evidence that depicts science, analysis and information as instrumental to capacity building of individuals and institutions, contributing to both improved governance structures and community assets. FINDINGS FOR EVALUATION QUESTION 1 1. Evidence of progress includes significant milestones achieved over the first two and half years, including: capacity building in local communities; relevant and timely research findings and; good publications. On the ground, 128 beach village committees have been revamped and made functional and 248,985 hectares of land have been rehabilitated by planting of trees. Most KIs in local communities stressed the need for alternative sources of income, and better marketing of their products and access to capital. 2. Comparisons of allocations for the four program components suggest relatively large amounts of Contribution analysis is useful in situations where the program is not experimental—there is little or no scope for varying how the program is implemented—and the program has been funded on the basis of a theory of change. Mayne, ILAC 2008 5 resources dedicated to research (Component 1) compared to on-the-ground implementation and livelihood development activities (Components 3 and 4). However, activities anticipated by the Pact Malawi team for Years 4 and 5 seem to heavily rely on inputs from Component 1 with direct overlap into Components 2, 3 and 4. 3. As for Component 2, KI respondents recognized strong leadership from FISH staff and implementing partners in creating an enabling environment for conservation and management. Evidence of progress includes revamped and/or created BVCs and FAs and strong political will to support PFM at community, district, and national levels. KIs in all four local government districts recognized FISH efforts to complete socio-economic profiles that were used to develop District Development Plans with strengthened fishery sections. There is evidence of coordinated institutional work supported by FISH including national and subnational level actors. In this respect, KIs mentioned active collaboration with entities such as the Ministry of Agriculture, Irrigation and Water Development and Ministry of Local Government and Rural Development under which where the District Councils are seated. 4. The Fisheries Science and Technology Advisory Panel (FSTAP), an anticipated major contributor to the TOC condition of “ shared, evidence-based objectives and learning” has been slow in fully engaging its role as a “ participatory clearing house mechanism for defining and prioritizing applied science needs” and assumed to be critical to FISH’s exit strategy. In this respect, two fundamental inputs guiding its work, the research and communications strategies are not fully in place. Hosted under the National Commission for Science and Technology (NCST), FSTAP is assumed to effectively service both information management and user needs. Hence, FISH aims to assist these two realms by instilling a participatory approach that guides services as a driver of increased access that hosts shared learning and understanding. These are tasks that may require significant changes in the organizational cultures of participating institutions and thus, need to be advanced early in the remaining LOP. 5. At the local level, FISH technicians and Fisheries/Forestry extension agents are working together in planning and implementing FISH project activities. The vast majority of FISH and Fisheries/Forestry staff interviewed agreed that collaboration was best demonstrated by their shared use of resources and collaborative training. CONCLUSIONS FOR EVALUATION QUESTION 1 1. FISH components are adequate to achieve the project goal and support the theory of change. “Component 1. Utilization of science” takes up a large percentage of program resources and accounts for almost 50% of the total budget. Accordingly, the anticipated leveraging of Component 1 in Years 4 and 5 towards outputs and outcomes that crossover to the remaining three components seems an effective way to assist the ability of FISH to achieve a higher concentration of activities on the ground. It remains unclear whether this shift may be enough to cover the natural demand for follow-up and expansion of the work accomplished through its mid-period, particularly in terms of sustained levels of capacity building (beyond ToT) and the critical mentoring/coaching roles that have surfaced from the organizational/institutional development accrued. 2. There is visible collaboration among implementing partners, Government and local assemblies to implement FISH projects and activities and to achieve program outputs and outcomes. However, important knowledge gaps and hindrances to efficient data and information flows seem to be prevalent in key stakeholders across the local, district and central levels. There are also important lags in achieving the full activation and contribution of FSTAP, as a key GoM partner at the central level, assumed to remain and expand beyond the FISH LOP. These prevalent conditions will need to be intensely addressed, particularly in terms of establishing viable and sustainable feedback loops across the community, district and central government levels. In this respect, the progress made by FISH technicians alongside Fisheries/Forestry extension agents should afford important lessons on building more effective communication and exchange. The successful collaboration of these two different 6 profiles that provide extension services at the community level can be instrumental in building a stronger presence at the district level and more fluid and productive collaboration with both the Fisheries and Forestry Departments at the central level. ANALYTICAL SUMMARY FOR QUESTION 1 The evaluation team concludes that national and local government officials, local communities and fishers work together reasonably well to improve management of fisheries resources. Findings demonstrate co-management support from DoF, local governments, BVCs and Traditional Authorities. National and local government institutions share a realistic vision for Participatory Fishery Management (PFM) and local communities are comfortable with co-management principles; partly based on their experience with Traditional Authorities and village governance. The FISH program addressed some past weaknesses in the PFM model by expanding training and capacity building activities in BVCs and VNRMCs. Moreover, by integrating fishery management and biodiversity conservation with economic development objectives and improved local governance, FISH supports the project goal of increased, social, ecological, and economic resilience of freshwater ecosystems and the people who depend on them. RECOMMENDATIONS TO EVALUATION QUESTION 1 1. Given the need for greater enforcement and more equitable revenue sharing between central and local governments, FISH should develop appropriate advocacy efforts to influence fishery management policy at higher levels of central government to consider sharing revenue generated with the local institutions. 2. FISH should review its research strategy to make sure scientific studies support relevant management decisions. Moreover, a more active role of FSTAP based on a highly visible FISH research plan should complement and influence the research agendas supported by the central government. 3. FISH should consider shifting more resources into on-the-ground implementation with added emphasis on biodiversity conservation and natural resource management. Moreover, selection of implementation sites for project activities should be linked to biodiversity hotspots and climate change vulnerability data that help prioritize best locations. EVALUATION QUESTION 2 How widespread in the target area is knowledge of FISH and its messaging, including on halting use of inappropriate fishing gear such as mosquito nets in fishing, improved methods for fish handling/processing, observance of closed seasons, minimum catch size, mesh size, no-fish areas and sanctuaries, and income-generation activities (e.g., orange-fleshed sweet potatoes, beekeeping etc.). To what extent are the local communities actually using these techniques? TOC SYNOPSIS FOR QUESTION 2 Results Logic: Knowledge about the FISH activity, including its objectives, was clearly articulated by both, key stakeholders and community level beneficiaries. At the local level, the work undertaken by the FISH technicians (in collaboration with Fisheries/Forestry extension agents) seems to have addressed the shared learning (tracking increased access and understanding of Output 1) and constitute the main feedback loop to the target groups. Messages clearly overlap topics associated with Outputs 3 and 4 and their corresponding sub-outputs, where evidence points to their dissemination as instrumental to increased widespread awareness (featuring a proactive connotation, particularly prominent in still limited radio production), including key target groups directly associated with the identified and interlinked livelihood areas: catchment management, habitat/riparian zone conservation and “hotspot” areas. Contribution Story: The outputs that drive the contribution story under this question speak to content (focus on practices and income generation) and outreach (targets of dissemination). No substantial 7 evidence was found on actual changes happening through the midpoint of FISH, affecting the density or patterns of application of improved techniques and practices. Although, for example, strong emphasis has been placed in understanding and correcting the root causes of poor fish handling/processing, there seems to be very little data that examines audience responses and user feedback to link and substantiate message effectiveness. Primary sources addressed by the interviews do provide some evidence of a push/pull effect (knowledge transfer to the user/demand-driven feedback based on user experience and skill) achieved by direct on-the-ground interaction However, effective feedback that guides adaptation and evolution in message crafting and dissemination is not yet a structured and systematic process. There is also a need to better align these critical dissemination actions with the “if” conditions that guide the theory of change. Given the unique capacity of FISH to work across governance levels and district borders, its contribution should facilitate a more integrated view of what, on the ground, is a fractured perception of individual districts and community settings. This seems especially critical to ensure that success in behavioral change supported by dissemination can be scaled up through deliberate and strategic variations (or iterations) shared learning and eco-scaled governance. FINDINGS FOR EVALUATION QUESTION 2 1. Knowledge about the FISH activity, including its objectives, was clearly articulated by primary sources, including key stakeholders and community level beneficiaries, across the different community and institutional settings (four districts) that were visited. While key stakeholders included staff from Department of Fisheries, District Councils, Department of Forestry, MALDECO Fisheries and the FISH activity, Community beneficiaries included BVC members, Village Natural Resources Management Committees (VNRMC), bee-keepers, and Village Savings and Loan (VSL) groups. There is evidence of exchange based on messages (source/user collaboration) among key stakeholders at various levels, including the Departments of Fisheries and Forestry, BVC and Natural resources management committees. These exchanges seem to point to a nascent demand-driven (based on application) focus of future messages as well as an opportunity to identify conveyance hindrances and blind spots where messages may not have a viable user base to build on actual practices. Data that better links messages to modified behavior and application is not available. 2. Across numerous interviews where respondents often used local language, evidence of knowledge introduced by FISH included a clear understanding of core concepts including an integrated approach that links lake and catchment ecosystems to fisheries management. KIs confirmed that messages are disseminated through various channels; but most effectively through direct interaction with FISH and DoF technicians. Based on data collected by FISH, direct interaction was the most effective outreach channel for message dissemination reported (61.4%), followed by radio (21.8%), then leaflets and other means such as traditional songs and dramatization (15.3%). KIs also identified the planned Fisheries Communication Strategy developed by a FISH consultant, as a resource to enhance standardization and quality assurance of messages. Primary sources offered evidence of important differences and contrasts in the skill sets and professional backgrounds among the FISH Technicians recruited by CISER, WESM, and EI. Moreover, DoF staff often support activities in fisheries, forestry, and agriculture and have more fishery training relative to FISH Technicians. A key area of differences in their skill sets and exposure is their previous experience in dissemination and facilitation activities. CONCLUSIONS FOR EVALUATION QUESTION 2 1. FISH messages are adequately and widely disseminated in the project areas and some of these messages have been adopted in practice. However, it is difficult to know whether these messages will lead to widespread adoption of best fishing practices and technologies. 2. Illegal commercial trawlers present threats of overfishing and adverse impacts on the lake ecosystem and aquatic biodiversity. Although DoF licenses the commercial trawler fleet, enforcement remains weak. 8 ANALYTICAL SUMMARY FOR EVALUATION QUESTION 2 The evaluation team found widespread coverage of FISH messages meant to prevent illegal fishing, protect biodiversity and promote income generation activities. Findings from members of BVCs, VNRMCs, bee￾keeping co-operatives, and fish processing groups demonstrate that direct community interaction with technicians provides the most effective public outreach and training methods. The evaluation team concludes that there is good message coverage and reasonable assimilation in all four target districts. Moreover, there is evidence that communities are using some of these techniques; although their overall impact is difficult to know. However, gaps remain in the outreach and communications strategy; especially regarding sustainable fishing messages to the commercial trawl fleet. RECOMMENDATIONS TO EVALUATION QUESTION 2 1. To reduce fishing pressure and to support livelihoods in local communities, FISH should further develop alternative sources of income. In supporting local industries such as honey production, fish processing, and possibly aquaculture, FISH should consider additional capacity building activities and explore public/private support models for marketing ideas, access to seed capital and longer-term financing. 2. The FISH communications strategy should engage the commercial trawl fleet and higher-level government managers in dialogue and provide them with sustainable fishing messages. Deliver good and timely scientific information and encourage government managers to use this in making better and informed decisions. Specifically, results from Usipa and Chambo stock assessments and related management recommendations should be disseminated and discussed among high-level fishery managers. EVALUATION QUESTION 3 How effective have efforts such as campaigns against using mosquito nets, no-fishing zones/time periods, fish sanctuaries, and vessel monitoring systems actually been in countering over-fishing? TOC SYNOPSIS FOR QUESTION 3 Results Logic: Following the logic phrased by the four outputs, effectiveness of FISH efforts would track the relative success of their combined hosted activities against an evidence-based demonstration of positive or negative changes (whether arguing achievement or non-achievement). In relating the combined effect of outputs to the three conditions for change assumed by the FISH TOC, we would want to relate positive or negative changes assessed to specific contributions that can be cross-referenced between any noted output and any of the three TOC conditions. Relevant to the effectiveness of Fish efforts, Annex 6 provides an illustrative distribution of outputs that either support (e.g. content, access, delivery) or complement the FISH campaigns (e.g. increased capacity). The matrix developed for the Evaluation Question 3, illustrates significant intersections between FISH achievements (associated with each of the four outputs) involving or assisted by dissemination and promotional efforts such as campaigns and the causal logic assumed by the FISH TOC. Contribution Story: When related back to the cross-section of TOC with the results logic a good example of this contribution story would be the faster uptake of improved practices detected in enhanced catchment areas (addressed by FISH in tandem with community engagement) has been related a multiplier effect from the use of combined technologies (evidence-based objectives and learning). When linking this uptake across the complete TOC logic, two additional elements have been included that surfaced in the findings from primary sources. One is the successful BVC and VNRMC (ecosystem-scaled governance) implementation and management of protected areas, which promotes habitat protection based on direct community involvement. The second is the revenue proposed by the FISH sharing model for the fishery sector to benefit local institutions such as the BVCs and VNRMCs as a means to reduce illegal fishing. This third element links campaign targets with the means (assets) of the engaged communities. 9 FINDINGS FOR EVALUATION QUESTION 3 4. Most BVC (82%) and fisher’s respondents believe that illegal fishing continues in Malawian lakes, however, most respondents believe it is decreasing. Illegal fishing identified by respondents includes fishing during closed season, mesh size violations (mostly under-meshed mosquito nets), and fishing in restricted areas by commercial trawl fishermen. Secondary sources (DoF Frame Survey) show increasing illegal fishing activity. Out of a total 58,993 gillnets recorded in the Survey, 64.2% were under-meshed nets (Ngongongo) and 5.4% were mono-filament nets, both of which are illegal. Although the total number of gillnets has declined by 13.6% from 66,999 recorded in 2015 survey, the contribution of illegal Ngongongo nets has increased by 5.2% in the 2016 survey from the 2015 survey, where most illegal gear is used by fishers in Lake Malawi. 5. Local communities on all three lakes know about and/or support sanctuaries and no-take zones, recognizing their importance in protecting fish stocks. Several VNRMCs described the importance of tree planting and forest protection to prevent soil erosion in order to help lake fish stocks. BVC members spoke to their collaboration with DoF and Traditional Authorities to patrol waters for illegal activities and confiscate illegal gear. Nonetheless, KIs repeatedly identified the need to strengthen surveillance, monitoring and enforcement activities as a necessary joint effort by BVCs and DoF. Vessel Monitoring Systems (VMS) have been tested and they are required as a condition in the license application form. However, implementation delays have been experienced due to procurement requirements of the FAO-TCP project under which the VMS equipment is planned to be procured. 6. FISH, with CEPA support, proposed a revenue sharing scheme for the fishery sector, but it has not been implemented in the GOM. KIs across the three lake ecosystems demonstrated variable degrees of familiarity with and participation in the FISH program. KIs stressed the importance of promptly improving BVC By-laws as a way to improve enforcement of local fishery management rules and regulations. Interviews with DoF Regional Fishery Officers and District commissioners in all four districts revealed that they have endorsed the new By-Laws, but did not elaborate on an approval timeframe. In one District, approval apparently depends on quarterly meetings of the District Council and in another approval requires a review by the Director of Planning and Development before moving forward to Council review. KIs also refer to the need to equitably share collected revenues among central government and local institutions such as BVCs and VNRMCs as a means to reduce illegal fishing. Fishing communities around Lake Malawi made greater progress implementing BVCs; as measured by progress on the six-step process relative to VNRMCs in agricultural communities around Lake Chilwa and Lake Chiuta. CONCLUSIONS FOR EVALUATION QUESTION 3 1. FISH campaigns to stop illegal fishing practices have reduced some improper activities, but illegal fishing continues. Stronger and greater enforcement is required to stop illegal fishing, and under current laws effective enforcement requires the cooperation of BVCs, DoF, and Traditional Authorities. 2. The FISH project aims to implement an integrated coastal ecosystem conservation strategy that includes the lake and the surrounding catchment extending 10 km from the shore. The community understands the importance of protecting these coastal habitats and some BVCs and VNRMCs are currently implementing and managing protected areas. Based on raised community awareness, the FISH activity now is prepared to implement additional conservation and NRM projects to reduce threats to biodiversity and the coastal ecosystem. ANALYTICAL SUMMARY FOR EVALUATION QUESTION 3 Findings demonstrate good local understanding of fishing rules and practical knowledge of the types of violations that occur. Based on village interviews and fishery data from DoF, the evaluation team concludes that despite FISH campaigns, high levels of illegal fishing continue; especially the use of under meshed nets, 10 fishing in restricted zones, and fishing during closed seasons. However local communities often described decreasing levels of illegal fishing; perhaps because they were more directly involved with monitoring their own areas, or because local knowledge differs from official statistics. Most BVC and VNRMC members understand the ecological links among land uses, water quality, and fish habitats and they support and maintain protected areas. In this context, the evaluation team concludes that FISH campaigns have developed significant community awareness, or intellectual capital, to counter over-fishing and protect biodiversity. This represents an important step toward behavior changes required to stop illegal fishing; even if the current impact is not precisely known. Viewed in this longer-term perspective, FISH campaigns can be considered as an effective first step to counter over-fishing. However, problems remain in maintaining sustainable fish stocks and protecting lake and coastal biodiversity. RECOMMENDATIONS TO EVALUATION QUESTION 3 1. Build BVC capacity for fishery co-management. Advocate for prompt approval and enforcement of BVC By-Laws across the different levels of PFM. 2. Strengthen local enforcement and seek greater widespread clarity and awareness of institutional roles and mandates among BVCs, TAs, and DoF regarding patrolling, monitoring and enforcing regulations and sanctioning violators. 3. Advocate for stronger GOM enforcement at the national levels. 4. To address growing concerns of illegal trawler operations, FISH should:  Engage the commercial trawl fleet in supporting stronger enforcement of fishing regulations. Commercial fleet stakeholders may want to review responsible fishery guidelines developed by FAO and compare trawl operations to international best practices (e.g. Voluntary Guidelines for Securing Sustainable Small-Scale Fisheries). This may help in developing future export markets and provide incentives for better industry management.  Consider strengthening the Fishing Association of Malawi (FISAM) as a trade group of legal trawl fishers working to better manage the commercial fleet.  Advocate for stronger GOM controls on licenses and enforcement of fishing regulations. Decentralize licensing control to District Fishery Offices. EVALUATION QUESTION 4 How sustainable are the Participatory Fisheries Management (PFM) efforts? Are District Council/GOM officials, traditional leaders and local communities likely to have the political will and funding necessary to continue these programs on their own when FISH ends? To what extent have Participatory Fisheries Management Agreements been advanced with BVCs and their subsequent fisheries associations? PFM has been tested and tried in the past and under FISH but the scale and scope of PFM has been variable. What are the key drivers for successful implementation of PFM in Malawi? TOC SYNOPSIS FOR QUESTION 4 Results Logic: To date, the work undertaken by FISH towards better ensuring PFM sustainability pivots around Output 2 and in particular, the Sub-output of Institutional and community capacities strengthened is reflected in FISH efforts towards enabling better structured stakeholder contributions and efforts to increase the operational resources of BVCs, Fisheries Associations and District Councils in support of improved co-management. Although there are important complementary outcomes across the other three outputs (stemming from the corresponding FISH Components), including better informed decision￾making to address sustainability challenges (Output 1) and planned efforts to build on the ecosystem based approach introduced to fisheries management with emphasis on improved adaptive capacity (Output 4). The updated PFM guide produced by FISH (Output 2) specifies ways in which BVCs and FAs can mobilize resources to fund their participatory fisheries management operations and includes fines for illegal fishing, 11 fish landing fees, fishing licenses, beach use fees for fish processors, trawlers, directly relate to parallel hosted outputs to reduce priority threats to freshwater ecosystem biodiversity. These strategies have been incorporated in the BVC’s and FA’s by-laws (Output 2). Contribution Story: Although it is possible to identify significant causal linkages between outputs and outcomes that relate to an increased prospect for sustainability of PFM achievements to date, evidence collected and reviewed does not yet build a complete argument for PFM success. In this respect, the contribution story to date centers on the ability of FISH to improve the implementation context that tracks some of the identified drivers of PFM success. Beyond the obvious contributions to build the capacity and means of key implementers such as BVCs and FAs, FISH has also managed to amplify the basis for PFM success by overlapping its benefits to target groups that previously may have not otherwise been strong stakeholders. For example, identifying key technologies for coordinated use across multiple target groups, involving promotion and dissemination in beach settings. In contrast with historical PFM implementation, FISH strategically leverages the PFM model to address a broader target audience, based on added value for actual or potential users of solar fish drying, fuel-efficient stoves, mobile smoking kilns, climate smart agriculture (intensive rice production) and high yield bee-keeping. FINDINGS FOR EVALUATION QUESTION 4 1. Evidence provided by primary sources at various levels suggested strong political will to support PFM among BVCs, local governments in all four target districts, and in DoF. Specifically, KIs expressed support Participatory Fisheries Management (PFM) as a way to improve fishery management in Malawi and believe that most, but not all, Traditional Authorities support PFM. Respondents believe that PFM progress can be measured by implementation of the six-step process to establish BVCs and management agreements. There is also evidence of the advancement of Fishing Associations in a number of districts, reflected in their success in enlisting the formal membership of multiple BVCs. 2. In terms of factors that could contribute to sustainability KIs noted the legal formalization of BVC By￾Laws and the need for sustainable funding for BVCs, alongside equitable revenue sharing between central and local governments. When asked about possible drivers of PFM success, KIs referred to both the approval and enforcement of BVC By-Laws. However, some respondents further elaborated on the importance of building community ownership when advancing PFM initiatives, highlighting the need to factor contextual differences among the multiple lake settings and how these should guide a sensitive engagement of target communities. This is of particular importance when engaging Traditional Authorities and understanding their variable interests, power base and influence in advancing the BVC mandates. 3. Echoing reflections shared by FISH staff, KIs also spoke to the need to effectively pair the advancement and approval of BVC By-Laws with more productive relationships with Traditional Authorities, possibly better linking them to new emerging spaces for discussion and exchange hosted under the collective BVC mandates promoted by rising Fishing Associations. On a parallel possible course of action, secondary sources suggest that FISH has effectively leveraged new knowledge (e.g. new/improved technologies and productivity) that addresses special interests across multiple stakeholder groups that, in turn, better ground the growth of the PFM model and attract District￾level interest, beyond the immediate lake settings and across districts. CONCLUSIONS FOR EVALUATION QUESTION 4 1. Illegal commercial trawlers present threats of overfishing and adverse impacts on the lake ecosystem. Although DoF licenses the commercial trawler fleet and is responsible for enforcement, enforcement remains weak. FISH, through FSTAP plans to hold meetings with DoF and Fisheries Association of Malawi (FISAM) to make the case for mandatory membership to FISAM for all commercial stern and pair trawls fishers. FISH will assist FISAM and guide them through the 6 PFM steps, if they should so 12 decide to federate, this will be a first step for promulgating rights based management approach to the trawl fishery. Nonetheless, weak policies at higher levels of central government may continue to hinder or weaken critical commitments such as mandatory membership to FISAM to ensure trawlers’ compliance to license conditions. Although co-management can improve enforcement by empowering BVCs to cooperate with DoF in surveillance and monitoring activities, the enforcement mandate belongs to DoF and needs to be substantially improved and increased. 2. PFM has reasonably good political support from central and local governments and community based organizations, responsive to a decisive and readily acknowledged interaction with FISH at all levels. In this regard, FISH evidences a strong grasp of the inherent complexity of the multiple stakeholders at various levels that must work together to successfully implement PFM. Gender is also at the core of this complex collaboration, where cultural and social norms, supported by community-based governance may not be entirely aligned with the economic contribution of women and its critical importance to sustainability. The Fish Value Chain Study undertaken by FISH, for example, revealed that 80% of the total fish production in Malawi goes through this chain with women contributing more that 55% of the value addition through fish processing, distribution and marketing mainly to rural based markets where the majority of the population resides. 3. FISH has advanced PFM by strengthening political will in local governments, building capacity in local communities, and helping to develop BVC By-Laws. However, sustained future building of PFM success depends on equitable revenue sharing among central and local governments and empowered BVCs to be able to co-manage their coastal resources. Longer term PFM sustainability depends on equitable revenue sharing among central and local governments and BVC empowerment based on approved and enforced By-Laws. ANALYTICAL SUMMARY FOR EVALUATION QUESTION 4 Stakeholders from central, local, and community organizations expressed political will to implement PFM. Moreover, these stakeholders understood their respective roles in implementing PFM and their organizations coordinated responsibilities reasonably well. Evidence includes: 1) DoF endorsement of co￾management and community ownership of the PFM process; 2) local government support through strengthened District Development Plans; and 3) BVCs By-Laws to empower community governance. FISH worked successfully to advance PFM agreements as evidenced by the recently developed BVC By￾Laws and expansion of Fishery Associations to represent multiple BVCs. Longer-term BVC sustainability remains uncertain, however, without FISH support. Most respondents agreed that long-term sustainability depends on legally empowered and financially secure BVCs operating in a more equitable revenue sharing environment among central and local governments. Other factors for success include prompt District Council approval and enforcement of BVC By-Laws and more productive relationships with Traditional Authorities. Illegal fishing remains a problem in the commercial trawl sector and successful co￾management will be needed to involve these stakeholders in the process. RELEVANT RECOMMENDATIONS TO EVALUATION QUESTION 4 1. To enhance the sustainability of co-management efforts FISH should consider:  Promoting alternate sources of income (e.g. honey production, fish processing and aquaculture) for BVCs and VNRMCs;  Continuing training activities and placing more emphasis on entrepreneurial skills (e.g. record keeping, marketing, product handling and certification) in local communities;  Support increased advocacy at central government levels to ensure equitable revenue sharing among central, local, and community levels. Specifically, encourage the Director of Fisheries with guidance from the Ministry of Finance, Economic Planning and Development to operationalize a 13 comprehensive Fisheries Fund as provided for in Section 25 of the Fishery Conservation and Management Act (FCMA). Moreover, FISH should focus on implementation of Issue 1 of the FISH Advocacy Strategy: Government ensures increased financing for fisheries management at district and local level in the FISH Project target districts by 2019;  Advancing the research agenda and communication strategy towards leveraging the Fishery Scientific and Technical Advisory Panel (FSTAP) as a viable and influential government partner; and  Designing an exit strategy that builds feasible and strategic transitions to comparable long-term private/public sector initiatives. 2. To strengthen commitments among stakeholders, FISH should consider:  Focusing advocacy at local government levels (e.g. District Councils) on prompt passing and effective enforcement of BVC By-Laws;  Expanding and strengthening networking and learning opportunities (e.g. ceremonies, events, and conferences) that bring together BVCs and co-management partners. Sponsor exchanges and shared learning opportunities. FISH may also want to explore existing community participation models based on iconic or symbolic themes that best mobilize public opinion at the local level (e.g. RARE use of music, storytelling, festivities etc.)  Strengthening stakeholder roles by providing added visibility to and support for community efforts in building a shared sense of purpose. EVALUATION QUESTION 5 How well is FISH integrated with: a) other USAID initiatives (e.g., Feed the Future): and b) other DPs efforts in sustainable fisheries, livelihoods, and climate change adaptation? How successful has FISH been in leveraging assistance from other sources and supporting Malawian efforts to obtain funding from other donors? TOC SYNOPSIS FOR QUESTION 5 Results Logic: In relating the FISH results logic to successful integration with other USAID initiative and efforts by other development partners, the various outcomes under each of the four output areas identified in the FISH Results Framework appear to relate to integration that is either content-driven or based on geographic focus and co-location. While content-driven integration naturally aligns with efforts under Output 1, given FISH’s prominent work on the ground through extension services, geography and co-location track opportunity and is, of course, sensitive to both settings and actors. Therefore, location specific collaboration with other initiatives is mostly driven by the focus of other initiatives in addressing threats (Output 3), and/or supporting adaptation (Output 4). However, the greater advantage of the results logic followed by FISH is opportunities presented by its emphasis on the PFM model as a community-based pivot to comprehensive biodiversity conservation and integrated watershed management. Contribution Story: In considering the sequence and scope of the three TOC conditions, both content development and site interventions are relevant to the contribution story on FISH’s ability to integrate its USAID funded work with other initiatives (whether funded by USAID or other DPs). In this respect, visible synergies in planning and implementation efforts led by FISH can be readily linked to human resource readiness (extension work involving FISH Technicians and DoF agents) that is tasked with a sound contextual understanding of challenges and opportunities that (shared evidence-based objectives). Likewise, FISH is in a position to continue building on knowledge assets that are shared across the portfolio USAID, as well as initiatives of other development partners (shared evidence-based learning). Moreover, 14 its implementation footprint, centered around the PFM model, actively encourages inclusive and interest￾specific collaboration and decision-making (inclusive and effective ecosystem-scaled governance). Hence, FISH contributions in the immediate lake setting readily associate with activities focused on catchment areas, riparian restoration and conservation and the more distant forest conservation. Finally, at the core of any given opportunity for integration, FISH consistently leverages the credibility and legitimacy of its community-based management model, which ultimately strives to translate improved natural resource management into well-grounded human development at the grassroots level (strengthen the assets of communities). FINDINGS FOR EVALUATION QUESTION 5 1. FISH has been able to integrate both its content development and site interventions with either the planning efforts, human resources or knowledge assets developed by multiple USAID initiatives. To this end, there is an obvious and natural overlap between FISH’s goal and the contributions driven by its results logic and those of other donor activities, particularly in the case of PERFORM, ECRP, WALA and TOGETHER. FISH also has leveraged capacity development previously supported by USAID to strengthen its on-the-ground presence. For example, KIs identified capacity previously built in the context of the NJIRA activity by Emmanuel International (EI is a FISH implementer) as critical to advancing the work in target communities now in the context of Christian Aid contributions under FISH. Collaboration was also noted with upper catchment activities implemented by NJIRA and the IDRC-funded Nsomba ndi Chuma Project. 2. KIs from District Councils identified initiatives such as the USG-funded DREAMS initiative, as well as the Local Development Fund Initiatives and the Lake Malawi Basin Project funded by the World Bank as examples of coordinated interventions that targeted complementary outcomes to those pursued by FISH. Likewise, it has tapped into human resources and on-the-ground experience previously developed by the Wellness in Agriculture for Livelihoods Advancements – WALA (also linked by EI) and supported the recruitment of FISH Technicians. WALA’s focus on agronomic practices and grassroots work with Village Savings and loans groups lends added depth to the efforts of technicians recruited with this background. In terms of furthering multi-donor synergies, FISH overlaps with other development partners implementing comparable biodiversity and watershed programs, which include FAO-GEF on Lake Malombe and Lake Malawi, UNDP-GEF on climate resilience support to Mangochi and Machinga, supporting the Lake Chilwa Climate Change Adaptation Project and the Lake Malawi Basin Programme. 3. There is also visible evidence of opportunities opened by overlapping geographic coverage and technical focus. In this respect, FISH has moved forward in better leveraging geographic overlaps with PERFORM’s containment and restoration efforts, where KIs noted a number of strategic and operational advantages. Along the lines of overlapping sites and geographic focus, KIs referred to the Enhancing Community Resilience Program - ECRP with significant coincidences in its focus on food security, reduced vulnerability and strengthened resilience in selected districts that are most prone to natural disasters and climatic hazards. Moreover, CEPA’s role as a technical partner on policy and advocacy as well as knowledge and information, provide ideal grounds for synergies in the mobilization of public sector support aligned with FISH’s work across multiple coincident districts as well as within Central Government. A strategic advantage is also recognized both by KIs and in secondary sources in FISH’s ability to builds on CIP’s work with seed improvement, opening the door for added potential benefits from high-end research (CGIAR), as well as longstanding crop diversification experience that did not specifically target populations associated with lake settings. CONCLUSIONS FOR EVALUATION QUESTION 5 3. Although there is evidence of meaningful content-centered collaboration, much of the integration efforts engaged by FISH are based on geographic focus and co-location to avoid redundancies. 15 However, there is high potential for enhanced integration of FISH with USAID–funded PERFORM, particularly in terms of income generation and broader ecosystem impact. This seems especially timely and appropriate in their overlapping efforts to inform decision-making at the national and subnational levels, as well as in their shared geographic focus for implementation at the local level (Machinga). 4. FISH has successfully engaged selected watershed management improvements (control of erosion, ecosystem protection and restoration), opening an opportunity for knowledge sharing/transfer and coordinated action with initiatives (bilateral or multilateral) that can converge in addressing existing gaps in comprehensive biodiversity conservation. ANALYTICAL SUMMARY FOR EVALUATION QUESTION 5 FISH complements programs within USAID and those of other development partners and NGOs. Since it includes a coastal catchment extending 10 km from lakeshores, FISH activities overlap with comparable watershed management projects related to agriculture, forestry, and adaptation to climate change. Within USAID, this includes the PERFORM program and its focus on forest conservation, green growth, and climate change. And by integrating the DREAMS HIV/AIDS prevention program into FISH activities, USAID expands fishery management to incorporate important social dimensions that can strengthen credibility and boost sustainable resource management. Development partners implementing comparable biodiversity and watershed programs include the FAO-GEF project on Lake Malombe and Lake Malawi, the UNDP-GEF project’s climate resilience support to Mangochi and Machinga, the Lake Chilwa Climate Change Adaptation Project and the Lake Malawi Basin Programme. Moreover, FISH collaborates with upper catchment projects such as those of NJIRA and the IDRC-funded Nsomba ndi Chuma Project. At the community level, FISH overlaps with other local development programs such as Enhancing Community Resilience Program (ECRP) and Wellness in Agriculture for Livelihoods Advancements (WALA). In this context, FISH has an opportunity to support an integrated coastal resource management strategy that advances a comprehensive ecosystem approach to protect biodiversity and maintain fish stocks while promoting sustainable development among local communities. RECOMMENDATIONS TO EVALUATION QUESTION 5 1. Further explore integration with PERFORM focusing on income generation and broader ecosystem impact. In this area, FISH could seek to better leverage its current and future investments with its presence on the ground, providing supplementary inputs and follow-up to direct assistance activities at the community level; 2. FISH efforts to inform decision-making at the national and subnational levels might benefit from PERFORM’s collaboration with and assistance to high-level multi-sector forums such as the RExG; 3. Emphasize knowledge sharing/transfer when considering coordinated action with other initiatives (bilateral or multilateral) that target existing gaps in comprehensive biodiversity conservation and integrated watershed management. 4. Overlaps in geographic implementation at the local level (e.g. Machinga with PERFORM) enable resource sharing and/or multiplier effects that extend coverage and depth of joint interventions. EVALUATION QUESTION 6 What if any adjustment could be made to improve project effectiveness? Are there lessons learned that have broader applicability for USAID Malawi and beyond? 16 TOC SYNOPSIS FOR QUESTION 6 Results Logic: When considering adjustments to improve effectiveness, the FISH results logic clearly supports a desirable shift from the initial emphasis on sharply improving the knowledge base for decision￾making and program action to a more immediate and direct threading of science, analysis and information to the advancement of the PFM model in the new economic and governance context of improved fisheries management (Output 2) and the strengthening of the fisheries value chain. Accordingly, in its subsequent implementation, Output 1 thus shifts to an operational mode that is much more immediate in supporting critical forecasting, emerging habitat protection needs and added efficiency that is setting-specific, thus directly contributing to tangible value chain enhancements (that support resilience and encourage adaptation) and reduced pressure on the freshwater ecosystem biodiversity (Output 3). This is also expected to build on previous work under Output 3 to address a wider range of stakeholders in the private sector (special interest such as developers and commercial investment) and Central Government (land use and physical planning). Following the results logic, FISH operations will continue to be heavily supported by its knowledge building capacity but directly focused on moving “from piloting to scale up of biodiversity conservation and habitat restoration.” This includes scale-out plans and direct technical assistance to partners in the facilitation of the rollout and sustained implementation of scale-out efforts. Contribution Story: The contribution story that emerges from the findings under Question 6, focused on future adjustments, clearly adheres to the logic expressed by the FISH TOC. Accordingly, the previous combined dynamics hosted under the four FISH areas of output will continue to feed into the three TOC conditions but their focus will now be on direct action, seeking the increased agility of the improved governance/management structures that favor an ecosystem-scaled delivery. Evidence suggests that this is expected to translate into District action and resourcing plans supported by strategies for better integrated use and management. Favoring higher degrees of inclusiveness, early warning and contingency planning are also expected to factor into strengthened delivery of improved governance structures, with the added advantage of the multi-level and simultaneous outreach honed by FISH to date. Along these lines, the prime community assets to be strengthened are those resources mobilized at the community level that more effectively empower proactive leadership. Processes that host shared learning should be more effectively linked to both biodiversity conservation and a strengthened value chain. In this, the surfacing focus on the FRAME surveys (including monitoring the post-harvest sector) are a critical resource to inform fisheries management by assessing positive or negative trends in the fisheries sector. FINDINGS FOR EVALUATION QUESTION 6 1. To achieve sustainability, KIs noted that BVCs should be given more time to build their capacity: to incorporate best practices; streamline the management process; and develop new sources of income. In this context, most respondents demonstrated patience in waiting for results, but recognized the realities of a long-term process and the need for continued support. Many pointed out that natural resource management programs take many years to show measurable benefits to fisheries and forests, highlighting the need for continual training based on turnover in BVC membership. Drawing from these activities that are critical to the value chain that supports the PFM model, they have identified problems associated with market access (reliable product transportation) and the need for capital (sustainable operations and growth). In complementary productive realms that support adaptation and increased resilience, respondents recognize the value of FISH-supported bee-keeping activities as effective means to produce commercial honey production and generate alternative sources of income. However, they also point out missing elements impacting profit margins and sustainable operations, including lower transportation costs, better marketing capacity and improved product packaging. 2. Although there is evidence of community awareness of the value of conservation practices, secondary sources point to significant gaps in advancing a coherent and comprehensive biodiversity conservation strategy that includes the lake and catchment area. Although fostering a well-entrenched conservation 17 purpose may be a longer-term endeavor, public perception and peer-to-peer engagement of target populations were noted by primary sources as important and strategic drivers of initial change that yields opportunity. For example, KIs referred to a village dramatization, actors in roles of early FISH skeptics came to support the initiative at the end of the play. When making these references, respondents appeared convinced that added momentum can be gained by influencing perception through increased peer-to-peer dissemination. Along this same rationale, respondents described their needs for low-end equipment to support patrol operations or engage new income generation activities (bee-keeping and fish processing) as opportunities to build more tangible and immediate means to effectively convey and promote prospective longer-term gains. 3. In considering adjustments to improve effectiveness, a central element to consider is the role and scope of activities considered under Output 1, which in turn, may leverage its higher budgetary allocations to priorities identified for Years 4 and 5 of the FISH implementation. Drawing from both primary (FISH senior staff) and secondary (planning documents) sources, these priority tasks cover specific realms across the four program components. Critical capacity building in support of the PFM model, for example, at the Central Government level centers on the implementation and effective use of the FRAME surveys to track pressure on fish biodiversity over time. In this, a major gap in current capacities is the availability and use of integrated spatial plans and maps of critical habitats, sanctuaries, and areas earmarked for development. 4. At its midpoint implementation, FISH is in a position to strategically facilitate both, the effective flow￾down of these enhanced technical capacities to the key actors of co-management at the local and community level (LFMA), as well as in fostering effective and sustainable feedback loops that directly support decision-making. Links that currently adjoin the different levels of intervention engaged by FISH, bridging new knowledge with key actors (particularly at the grassroots level), remain weak or nonexistent. Therefore, there are important and persistent knowledge gaps across the different implementation settings that could critically impact any sustainability/exit strategy adopted by FISH during its final implementation. 5. Also linked to a visible need for sustained and strategic feedback, there is a perceived need (emphasized by primary sources at multiple levels) for brokering opportunities at high levels of authority (Central Government) to promote important policy shifts that are relevant to the success of fisheries co-management and specifically the PFM model implemented by FISH. To this end, primary sources highlight the need to “catalyze the ideal scenario at the highest echelon of government,” seeking to sharpen and expand the work of present advocacy efforts. CONCLUSIONS FOR EVALUATION QUESTION 6 1. Overall, FISH implementation has been successful. Despite early program changes, the PACT consortium of NGOs has coordinated work plans, implemented field projects and reported activities reasonably well; as evidenced by achievement of program milestones and targets. There are some important and persistent challenges that are inherent to its complex implementation and the coordination of numerous partners (and their consortia). This is particularly acute in terms of the required sustained efforts to improve data collection and ensure data quality. However, the multiple layers of this complex implementation model also lend a unique opportunity to build on and expand further integration that secures both FISH results and its exit strategy. 2. Some KIs believe that FISH, now in its third year, might shift emphasis more toward on-the-ground implementation (Outputs 3 & 4) rather than research and planning (Output 1). Moreover, to verify program effectiveness, FISH research findings from Output 1 should be clearly linked to management decisions and/or advocacy campaigns. Moreover, the work undertaken to strengthen feedback and increase capacity public or private sector capacity at various subnational levels (district authorities, 18 Fisheries and Forestry district delegates, LFMA, Traditional Leaders, VSLs) will need to be better integrated with efforts to generate and disseminate a national research agenda. In this respect, FISH needs to more effectively address significant gaps in the advancement of a coherent and comprehensive biodiversity conservation strategy covering the lake and catchment area. Likewise, there a shared perception that, even though harvesting conservation benefits may be a longer-term endeavor, public perception and peer-to-peer engagement of target populations can strategically advance higher levels of community-based commitments in the short-term. 3. Fish processors and honey makers have developed their products and now seek to market and sell them. At this point in the program, FISH may consider a sharper focus on income generation activities, such as improved community financing options and more business management training. Likewise, effective integration with other initiatives and development partners may be instrumental to address challenges associated with market access and reliable product transportation that minimizes losses, as well as added entrepreneurial capacity and access to financing that better assures sustainable (upkeep and expansion) operations and growth. ANALYTICAL SUMMARY FOR EVALUATION QUESTION 6 Under PACT and the consortium of NGO partners, FISH has implemented many and diverse activities reasonably well and the program met important project milestones and targets. Earlier program adjustments re-organized implementing partner responsibilities by region rather than by core expertise to better standardize and coordinate field activities. Other adjustments defined enhanced catchments to better integrate multiple activities. Now in its third year, FISH may consider shifting resources from research and planning to on-the-ground implementation and training. Added integration with other initiatives should be considered to expand capacity development, emphasizing income generation skills and more clearly link economic development with biodiversity conservation and sustainable fishery management. RECOMMENDATIONS TO EVALUATION QUESTION 6 1. FISH should consider shifting additional resources into on-the-ground implementation in biodiversity conservation and natural resource management. Selection of implementation sites for these additional activities should be linked to biodiversity hotspots and vulnerability data that prioritize best locations. Likewise, all additional research must be specifically linked to direct and immediate use by key local actors in specific settings, offering the necessary flexibility to respond to emerging needs. 2. Strengthen research management and communication. FISH should establish a functioning Fisheries Science and Technical Panel (FSTAP) and complete the National Fisheries Research Agenda and a National Fisheries Communication Strategy. Without these elements, securely in place by the close of the project, FISH risks losing scientific information and its impact on decision making. 3. Prioritize habitat protection efforts. Building on Environmental Threats and Opportunities Assessment findings that identify critical habitats, FISH may consider developing methodologies to prioritize sites and patterns for habitat protection and restoration. As best practices, these methods may be considered in district State of Environment Reports (SOER) and/or included in Environmental Action Plans (EAP) that will form the basis for the respective DoF, FA and BVC fisheries management plans.   4. Design an integrated watershed management plan that combines lake and catchment interventions within a coastal ecosystem framework. Define threats and prioritize activities to: 1) mitigate adverse impacts to water quality and quantity from agriculture, forestry, development and other 19 land use activities; 2) prioritize and protect critical habitats for freshwater resources; and 3) adapt to climate change. Collaborate with other development partners involved with watershed management. Review comparable models from USAID and FAO integrated coastal zone management projects as models. Share ideas with developing partners and fill programming gaps. 5. Develop an eco-regional conservation strategy to integrate management activities into a coherent, long-term plan to protect biodiversity. Among other tasks design a network of connected protected areas and prioritize restoration sites. Identify long-term conservation goals and set visions for achieving them over 5, 15, and 50 year periods. Consider conservation methodologies from NGOs such as The Nature Conservancy and the World Wildlife Fund. Integrate the long￾term conservation strategy into GOM planning. 6. Link stock assessments to fishery management decisions. Usipa and Chambo stock assessments present an opportunity to use scientific findings to support management decisions. FISH should disseminate results and use them to engage high-level policy makers. 7. FISH should consider activities driven by community leadership (possibly sponsored by NGOs or VSLs) that lead to sustained provision of low-end equipment to BVC and VNRMCs to support their monitoring efforts. These low-cost inputs (e.g. boots, gloves, life-jackets, hoes, shovels) can boost local community implementation results while contributing to safer and more productive work environments. Fund raising or sponsored inventory can be instrumental in building momentum and reinforcing collective awareness of the benefits of co-management and the gains made through restoration, conservation and protection of natural resources. EVALUATION QUESTION 7 Restriction to access or exclusion of further fishers appears to be one of the workable mechanisms to reduce fishing pressure on the freshwater lake. How feasible would it be for BVCs and District Councils to implement a phased approach of allowing surplus fishers to exit and restricting access for newer fishers? What would be the consequences? FINDINGS FOR EVALUATION QUESTION 7 Findings related to the challenges: 1. Skepticism. District Fishery Officers (DFOs), local government representatives, and some implementing partners expressed skepticism over the ability of BVCs to restrict access to only licensed fishers. 2. Open access fishery. Currently, Malawi maintains an open access fishery and BVCs do not have a strong mandate and enough political power to restrict numbers of fishers operating in their jurisdictions. 3. Legal mandate. Several DFOs pointed out that limiting fishery access would be the responsibility of Traditional Authorities and DoF, not BVCs. Findings related to opportunities: 4. Phased approach. Informant BVCs maintain lists of registered fishers and they request “transfer letters” form migrating fishers. This represents a first step to license the fishery. 5. Alternative livelihoods. Several informants described the lack of alternative livelihoods coupled with rapid population growth as factors to diminish the feasibility of establishing a limited entry fishery. In any case, one of the most important consequences of restricting access to fishing would be the need to create alternative livelihoods in other sectors. 20 CONCLUSIONS RELEVANT TO EVALUATION QUESTION 7 1. Restricting access of small-scale fishers will face resistance both from the fishers affected and from opinion/community leaders, possibly influencing the necessary support from local authorities. The absence of viable alternative livelihood for excluded fishers presents a major hindrance to restricting access. 2. There are weak local controls and monitoring operations of fishing activities. Fishery management must consider the complex social web of fisher communities encompass and understand inherent conflicts with Traditional Authorities and cultural beliefs. Nonetheless, BVCs and District Councils may be able to implement a phased approach to restrict fishers. But it would take continued training, patient advocacy and sustained political will to make it happen. ANALYTICAL SUMMARY FOR EVALUATION QUESTION 7 In the current environment, it is not feasible for BVCs alone to restrict fisher access to the lakes. BVCs lack a legal mandate to control an open access fishery. However, a phased approach to restrict access is one way to reduce fishing pressure and it is a necessary first step on the path toward a more sustainable fishery management. Based on the productive co-management relationships observed among BVCs, District Councils, and DoF, it may be possible to implement a phased approach to license and limit the number of fishers in order to maintain sustainable stocks. Key steps in the phased approach might include: I) implement and enforce BVC By-Laws; 2) register fishers and strengthen fishing user rights in BVC jurisdictions; 3) clarify enforcement roles among BVCs, DoF and Traditional Authorities; 4) license registered fisheries; and 5) enforce the rules and regulations. This phased approach will require continued training, sustained advocacy and growing political will over a long period. RECOMMENDATIONS TO EVALUATION QUESTION 7 1. FISH should advocate for limits to fishing (or restricted access) as a best practice for sustainable fishery management. The need to limit access warrants prompt attention due to the widespread deployment of cheap, illegal, under-meshed mosquito nets over the last ten years and their adverse impacts on fishery resources. 2. Based on a favorable enabling environment, BVCs be able to limit future access after: 1) implementing By-Laws; 2) strengthening fisher user rights in their jurisdictions; 3) and limiting licenses to registered fisheries. Upon meeting these conditions, restricting access within a PFM framework may be feasible through a phased approach wherein FISH provides support to:  Advocate policy reform at high levels of government and in key opinion-making circles.  Develop messages to explain the benefits of a limited entry fishery and describe the adverse consequences of continuing an open access fishery. Use the latest scientific and economic data to demonstrate the current vulnerability of stocks and their economic importance.  Develop stock assessments and prepare fishery management plans for Chambo and Usipa as described in Year 3 Work Plan. Consider introducing fishery limits (or restricted access) as elements of these model fishery management plans. Get feedback and build consensus.  Conduct consultations with decision makers and technical stakeholders to assess possible strategies to introduce limited fisheries and to enhance the sustainability of the co￾management framework. 21  Promote public discussion and new ideas about viable alternative sources of livelihood, intended to host opportunities to encourage fishers to apply their skills and entrepreneurial spirit to other activities. ANNEXES ANNEX I. EVALUATION STATEMENT OF WORK ANNEX II. EVALUATION DESIGN AND METHODS ANNEX III. DATA COLLECTION TOOLS ANNEX IV. EVALUATION QUESTIONS MATRIX ANNEX V. KEY INFORMANTS INTERVIEWED ANNEX VI. THEORY OF CHANGE ANALYSIS ANNEX VII. GENDER CONTEXT FOR FISH ANNEX VIII. CONFLICT OF INTEREST FORMS FOR TEAM MEMBERS ANNEX IX. STATEMENT OF DIFFERENCES 1 ANNEX I: EVALUATION STATEMENT OF WORK Mid-term Performance Evaluation of USAID’s Fisheries Integration of Society and Habitats (FISH) Activity United States Agency for International Development (USAID)/Malawi, Office of Sustainable Economic Growth (SEG) March 2017 POINT OF CONTACT David Chalmers and Bruce Sosola AOR, FISH Project Office of Sustainable Economic Growth USAID/Malawi Tel: 265 (0) 177.2455 Ext. 5128 Email: dchalmers@usaid.gov Abel Nyoni COR, Monitoring, Evaluation, and Learning Support (MELS) Project Office of Sustainable Economic Growth USAID/Malawi Tel: 265 (0) 177-2455, Ext. XX E-mail: anyoni@usaid.gov Project Information Project Name Fisheries Integration of Society and Habitats (FISH) Award Number Cooperative Agreement: AID-612-A-14-00004 Original Project Dates 9 September 2014 – 9 September 2019 Original Funding $8,450,000 Implementing Partner PACT    2 CONTENTS I. Background II. Program Components and Description III. Evaluation Purpose IV. Evaluation Questions V. Evaluation Methodology VI. Existing Sources of Information VII. Deliverables VIII. Evaluation Team Composition IX. Level of Effort X. Scheduling and Logistics XI. Budget    3 I. Background USAID’s fisheries Integration of Society and Habitats (referred to hereafter as FISH) was specifically designed to align with Malawi’s Growth and Development Strategy (MGDS) and to promote sustainable fisheries for improved livelihoods for communities living around Malawi’s main lakes. The project period is 5 years, from 9 September 2014 to 19 September 2019. FISH is one of the key activities under “Development Objective 2 (DO2): Sustainable Livelihoods Increased” of USAID Malawi’s Country Development Cooperative Strategy (CDCS). FISH has completed approximately 2.5 years; therefore, USAID wishes to evaluate the performance of the project at the mid-point of its five-year program. Consistent with USAID’s 2011 Evaluation Policy, the primary goal of this mid-term evaluation is to provide evidence so that USAID/Malawi can determine whether FISH is on track to achieve its desired results or not. Other goals of the evaluation are to inform what course corrections, if any, are needed, identifying specific project interventions that can be scaled up further and to determine best ways to ensure sustainability of the activities, institutions and capacities promoted by the project. Finally, this evaluation will provide strategic options for future USAID/Malawi engagement in fisheries and offer lessons learned with wider applicability to other activities in Malawi or beyond. II. Program Components and Description Malawi is a Least-Developed Country (LDC), whose population is predominantly rural and heavily dependent on smallholder agriculture. Fish are an extremely important source (45%) of protein for Malawi’s population. The fisheries of Lakes Malawi, Malombe, Chilwa, and Chiuta provide employment for 60,000 fishers and another 450,000 individuals involved in fish processing and domestic trade. They also contain 15% of global freshwater fish biodiversity. However, these fisheries are under considerable stress from a growing population, over-fishing, and environmental degradation, complicated by the effects of climate change. FISH seeks to address the drivers of over-fishing and degradation by improving mechanisms for local co-management of fisheries, using the following four main components: Component A1. Utilization of science, analysis, and information for decision making increased. This component focuses on gathering information on Malawi fisheries and making it available on a wider scale through a database; research to understand the current threats faced by the industry and options to improve conservation; and obtaining a better understanding of the effects of climate change on Malawi’s lakes and potential fisheries management options to increase resilience to those climate impacts Component B2. Enabling environment for conservation and management of freshwater ecosystems enhanced. This component focuses on ensuring an enabling legal framework for sustainable fisheries management and biodiversity conservation; greater transparency, representation, and accountability in decision￾making with regard to fisheries; and building institutional and community capacities for shared fisheries management (co-management) between local communities and local/national authorities. Component C3. Priority threats to freshwater ecosystem biodiversity reduced. This component is focused on addressing key threats to fisheries ecosystem biodiversity conservation by implementing best practices in the three interlinked livelihood areas listed below: a. Sustainable natural resources management and agriculture in the catchment. b. Fishery habitat management and riparian zone conservation. c. Sustainable fishing in targeted biodiversity ‘hotspot’ areas. Component D4. Adoption of climate change adaptation measures that support resilience of communities and freshwater ecosystems increased: Component D4 focuses on developing and disseminating strategies, methods, and information that will allow vulnerable individuals and communities to become increasingly resilient in the face of changes likely to be experienced as climate change takes hold around the four target lakes 4 and associated catchment ecosystems. Under this activity, FISH will provide viable, climate smart and more environmentally friendly and diversified livelihood practices by promoting the adoption of best practices in CCA identified and tried during Y1 and Y2 under Output A1. An Intensive ecosystem approach services packages for fisheries management and climate smart agriculture will be established in six catchment areas within the project command areas as outlined in Component H8. FISH’s link to USAID / Malawi’s Country Development Cooperation Strategy. USAID’s CDCS hypothesizes that if its assistance efforts are integrated across sectors, concentrated geographically, and coordinated better with other development partners (DPs), development results will be enhanced, more sustainable, and lead to achievement of its CDCS goal: Malawians’ quality of life improved. The results framework for the Country Development Cooperation Strategy (CDCS) 2013-2018 contains three Development Objectives (DOs) that contribute to USAID/Malawi’s overall Development Goal of “Malawians’ Quality of Life Improved.” FISH contributes to DO2: Sustainable Livelihoods Increased. FISH Results Framework and Development Hypothesis. USAID has identified the development hypothesis under DO2 as: if sustainable livelihoods are increased then Malawian’s Quality of life will be improved. Strengthening resiliency to climate change (IR 2.1), adding value to agricultural production (IR 2.2), better nutrition (IR 2.3), and expanding agricultural trade (IR 2.4) together are means to achieve increased sustainable livelihoods. The FISH goal, objectives and programmatic components are linked through a theory of change that posits that if decisions around fisheries co-management are: a. Based on shared, “evidence-based” objectives and learning; b. Grounded in inclusive user rights and effective ecosystem-scaled governance structure, and; c. Strengthen the assets of communities, then Malawi’s complex and diverse freshwater lake ecosystems can be sustained. The FISH Results Framework is shown below in Figure 1.    5 Figure 1: Results Framework FISH FISH Implementing Partners and Target Areas FISH is implemented by Pact in collaboration with the Coastal Resources Center (CRC) of the University of Rhode Island (URI), Christian Aid (CA), Community Initiative for Self-Reliance (CISER), the Center for Environment Policy and Advocacy (CEPA), Emmanuel International (EI), and the Wildlife and Environment Society of Malawi (WESM). The project works in four key ecological freshwater lake ecosystems of South-East and South-West Arms of Lake Malawi, Lake Malombe, Lake Chiuta and Lake Chilwa. The Map below provides a general picture of where SEG activities are implemented including the four FISH targeted districts: Mangochi, Balaka, Zomba and Machinga. 6 Figure 2. Map of FISH Targeted Districts 7 III. Evaluation Purpose The primary purpose of this mid-term performance evaluation is to assess the effectiveness of the FISH design and implementation processes. This evaluation will also inform management what course corrections, if any, are needed. Additionally, through this evaluation, USAID seeks strategic and programmatic options for future engagement to support fisheries livelihoods and climate change adaptation in Malawi. There are three major objectives of this evaluation. They are as follows: 1) To review, analyze, and evaluate the effectiveness of the FISH activity in achieving program objectives and contributing to USAID/Malawi’s efforts to mitigate climate change and adaptation in Malawi; 2) Evaluate major constraints in achieving expected project results; 3) Provide specific recommendations and lessons learned on strategies and approaches USAID/Malawi should pursue in the remaining period of FISH’s implementation and for future fisheries and climate change adaptation activities. The final performance evaluation of FISH should inter alia, determine whether the findings, recommendations, and lessons learned from the mid-term evaluation were taken into account during the remainder of the implementation period, and if so, what the impact was of any course corrections. IV. Evaluation Questions The contractor’s evaluation of FISH must be comprehensive. Whenever possible, the evaluation team should ensure that data is gender disaggregated and subjected to gender analysis. The contractor shall assess FISH performance and achievements against the performance indicators, targets, reporting requirements, outputs and deliverables described in the approved M&E plan, Annual Work Plans and the agreement. In doing so, the team should pay particular attention to required climate change and biodiversity indicators. While this evaluation is not a data quality audit, the contractor shall assess and describe the quality and use of performance monitoring data and information generated during FISH implementation. Evidence-based conclusions and recommendations on how FISH used performance monitoring data and information in performance management are required when describing FISH effectiveness in component management. The following questions should be addressed: 1. How effectively are FISH components supporting the project goal of “Increased, social, ecological, and economic resilience of freshwater ecosystems and the people who depend on them.” In particular, given past difficulties with the co-management experience in Malawi, are national and local government officials, local communities and fishers actually working together in a way that contributes to improved management of fisheries resources? 2. How widespread in the target area is knowledge of FISH and its messaging, including on halting use of inappropriate fishing gear such as mosquito nets in fishing, improved methods for fish handling/processing, observance of closed seasons, minimum catch size, mesh size, no-fish areas and sanctuaries, and income￾generation activities (e.g., orange-fleshed sweet potatoes, beekeeping etc.). To what extent are the local communities actually using these techniques? 3. How effective have efforts such as campaigns against using mosquito nets, no-fishing zones/time periods, fish sanctuaries, and vessel monitoring systems actually been in countering over-fishing? 4. How sustainable are the Participatory Fisheries Management (PFM) efforts? Are District Council/GOM officials, traditional leaders and local communities likely to have the political will and funding necessary to continue these programs on their own when FISH ends? To what extent have Participatory Fisheries 8 Management Agreements been advanced with BVCs and their subsequent fisheries associations? PFM has been tested and tried in the past and under FISH but the scale and scope of PFM has been variable. What are the key drivers for successful implementation of PFM in Malawi? 5. How well is FISH integrated with: a) other USAID initiatives (e.g., Feed the Future): and b) other DPs efforts in sustainable fisheries, livelihoods, and climate change adaptation? How successful has FISH been in leveraging assistance from other sources and supporting Malawian efforts to obtain funding from other donors? 6. What if any adjustment could be made to improve project effectiveness? Are there lessons learned that have broader applicability for USAID Malawi and beyond? 7. Restriction to access or exclusion of further fishers appears to be one of the workable mechanisms to reduce fishing pressure on the freshwater lake. How feasible would it be for BVCs and District Councils to implement a phased approach of allowing surplus fishers to exit and restricting access for newer fishers? What would be consequences? V. Evaluation Methodology The Contractor must propose a robust evaluation methodology to answer the key evaluation questions identified above. The evaluation methodology must follow a mixed–method approach using both quantitative and qualitative techniques in gathering reliable data and valid evidence of project outcomes or impacts including review of relevant FISH, USAID, GOM, FAO, and other secondary data sources. The contractor must develop a Mixed Methods Evaluation Design Matrix that will detail the data source, data collection methods, data collection instruments and analysis of data to answer each of the evaluation questions. By using a mixed-method approach, the evaluation team is expected to gain insight on the impact of FISH project activities (mostly from quantitative data collected by the project and others) and the processes (mostly qualitative information provided by the project staff and key informants) that lead to those impacts. It should generate gender-disaggregated data and reflect attention to gender relations such as the participation of women in training. Sequential and iterative approaches should be used to integrate the mixture of methods at various stages of the evaluation. The evaluators should utilize several different, yet complementary and inter-related forms of gathering information/data such as those described below. The contractor is expected to utilize its expert judgment and evaluation best practices in selecting which methodological components to include in the evaluation design. Document Review. Evaluation team members will review documents throughout the evaluation process including program reports, relevant studies to ensure that comprehensive and grounded best practices will be identified. Key Informant Interviews. The team will conduct one-on-one interviews with a variety of stakeholders including the most relevant GOM ministries and agencies, bi- and multilateral development partners supporting fisheries, livelihoods, and climate change adaptation activities in Malawi. This will be done administering a semi-structured or structured questionnaire. Self-assessment. The IPs will respond to a self-assessment either through a questionnaire or standard interview checklist put together by the evaluation team and approved by USAID before use. Expert Opinion Survey. Utilizing expert opinion is a technique used increasingly in evaluations. The evaluation team, with approval of USAID, can apply this method as well. Focus Group Discussions (FGDs). FGD (small group of 6 to 10 people) will be used to lead open discussion through a skilled moderator to gather semi-structured qualitative data. The preselected participants will discuss issues and concerns based on a list of key themes drawn up by the moderator. No more than 10 questions will be addressed by a group. These sessions will encourage free flowing discussion about the activity. 9 Mini Survey. This type of survey is small (30-40 participants) and can be performed rapidly in the field without analytical software such as SPSS or a large questionnaire. The sample size is not statistically significant; however, this type of analysis can be used to triangulate with other methods. Because of the small sample size this type of survey can be implemented quickly when time and resources are constrained. All the methodological strengths and weaknesses should be explicitly described in the evaluation report. Prior to the start of the evaluation, the evaluation team shall meet with EG office to refine the evaluation methodology, and address any other concerns the EG office may have. Limitations to Data Collection. There may be some challenges on biophysical markers of ecosystem health for FISH. Also due to the compressed timeline of the evaluation, the evaluation team will not have adequate time to conduct large surveys that are statistically significant. Wherever possible the team will triangulate findings using multiple sources. VI. Deliverables All deliverables are internal to USAID, MELS and the Evaluation Team unless otherwise instructed by USAID. Evaluation deliverables include: Evaluation Team Planning Meeting. Essential in organizing the team’s efforts. During the meeting, the team should review and discuss the SOW in its entirety, clarify team members’ role and responsibilities, draft work plan, develop data collection methods, review and clarify any logistical and administrative procedures for the assignment and instruments and to prepare for the in-brief with USAID/Malawi. In-brief Meeting with USAID/Malawi. Within two working days of international team members’ arrival in Malawi; Inception Report: At the In-brief meeting, the Evaluation team will provide an Inception report that will outline key aspects of the Evaluation, including the Work Plan, Methodology, Evaluation Design Matrix, and proposed Data Collection Instruments (e.g., interview guides). See more detail below.  Work Plan. The Contractor will prepare a detailed work plan that includes task timeline, methodology outlining approach to be used in answering each evaluation question, team responsibility, document review, key informant and stakeholder meetings, site visits, survey implementation, travel time, debriefings (for USAID, implementing partner and, if decided, the GOM), draft and final report writing. The work plan will include a data analysis plan. The work plan will be submitted to the MELS COR and FISH AOR at USAID/Malawi for approval no later than the fifth day after the Evaluation team arrives in Malawi.  Evaluation Design Matrix. A table that lists each evaluation question and the corresponding information sought, information sources, data collection methods, data analysis methods, and limitations and assumptions. The matrix should be finalized and shared with USAID/Malawi before evaluation field work starts. It should also be included as an annex in the evaluation report.  Data Collection Instruments. Development and submission of data collection instruments to USAID/Malawi during the design phase and after the evaluation is completed; Regular Updates with USAID. The Evaluation Team Leader will brief the MELS COR and FISH AOR on progress with the evaluation on at least a weekly basis, in person or by electronic communication. Any delays or complications must be quickly communicated to USAID/Malawi as early as possible to allow quick resolution and to minimize any disruptions to the evaluation. Emerging opportunities to strengthen the evaluation should also be discussed with USAID/Malawi as they arise. Out-briefing with USAID. The Contractor will present the major preliminary evaluation findings to USAID/Malawi through a PowerPoint presentation before the team’s departure from country. The debriefing will include a discussion of project achievements and issues as well as any preliminary recommendations. The team will consider USAID and MELS comments and incorporate them in the Draft Evaluation Report. 10 Stakeholders Workshop. The team will present the major findings from the evaluation to key stakeholders (as appropriate and as defined by USAID) through a PowerPoint presentation prior to the team’s departure from the country. The debriefing will include a discussion of achievements and activities only, with no recommendations for possible modifications to project approaches, results, or activities. The team will consider key stakeholder comments and incorporate them appropriately in drafting the evaluation report. Draft Evaluation Report. A draft report on the findings and recommendations should be submitted to USAID/Malawi and MELS 10 working days after departure of international team members from Malawi. The report must be no more than 50 pages in length (excluding annexes) and comply with the Checklist for Assessing USAID Evaluation Reports (see annexes). The written report should clearly describe findings, conclusions, and recommendations. The draft report must be of high quality with no grammatical errors or typos. A report is high quality when it represents a thoughtful, well-researched and well organized effort to objectively evaluate what worked in the project, what did not and why. The draft report must have well-constructed sentences that are presented in a way that clearly presents findings, conclusions and recommendations. The report should answer all the evaluation questions and the structure of the report should make it clear how the questions were answered. The draft report must meet the criteria set forth under the Final Report section below. USAID will provide comments on the draft report within 10 working days of submission. Final Evaluation Report. The Contractor will submit two versions of the Final Evaluation Report that incorporate Mission comments and suggestions no later than five working days after USAID/Malawi provides written comments on the Draft Evaluation Report. The format of the final reports is provided below. The report will be submitted in English, electronically. The public version of the final report which will be uploaded to the Development Experience Clearinghouse (DEC) will not include the annex on “Strategic Options for Future Programming” – evaluation objective 3. The USAID-only, procurement sensitive version of the final report will include the aforementioned annex on evaluation objective 3. The Contractor must ensure that Appendix I of the USAID Evaluation Policy – Criteria to Ensure the Quality of the Evaluation Report is followed. The final report should at a minimum meet the following criteria to ensure the quality of the report:  The evaluation report should represent a thoughtful, well-researched and well organized effort to objectively evaluate what worked in the project, what did not and why.  Evaluation report shall address all evaluation questions included in the scope of work.  The evaluation report should include the scope of work as an annex. All modifications to the scope of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline need to be agreed upon in writing by the MELS COR and FISH AOR.  The evaluation report must include a separate annex to answer objective 3 “Specific recommendations and lessons learned on strategies and approaches USAID/Malawi should use in the remaining period of FISH’s implementation and for future fisheries and climate change adaptation activities.  Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides will be included in an Annex in the final report.  Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, etc.).  Evaluation findings should be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings should be specific, concise and supported by strong quantitative or qualitative evidence.  Sources of information need to be properly identified and listed in an annex.  Recommendations need to be supported by a specific set of findings.  Recommendations should be action-oriented, practical and specific, with defined responsibility for the action. The format of the final evaluation report should strike a balance between depth and length. The report will include a table of contents, table of figures (as appropriate), acronyms, executive summary, introduction, purpose 11 of the evaluation, research design and methodology, findings, conclusions, lessons learned and recommendations. Where appropriate, the evaluation should utilize tables and graphs to link with data and other relevant information. The report should include, in the annex, any dissenting views by any team member or by USAID on any of the findings or recommendations. The report should not exceed 30 pages, excluding annexes. A second version of this report excluding any potentially procurement-sensitive information will be submitted (also electronically, in English) to Development Experience Clearinghouse (DEC) and for dissemination among implementing partners and other stakeholders. All quantitative data, if gathered, should be (1) provided in an electronic file in easily readable format; (2) organized and fully documented for use by those not fully familiar with the project or the evaluation; (3) owned by USAID and made available to the public barring rare exceptions. A thumb drive with all the data could be provided to the FISH AOR and FISH Chief of Party. The final report will be edited and formatted by the Contractor and provided to USAID/Malawi 5 working days after the Mission has reviewed the content and approved the final revised version of the report. VII. Evaluation Team Composition The team will include two international and three local consultants. The former should include specialists with the following areas of expertise: project evaluations and assessments (ideally natural resource management evaluations), fisheries, climate change adaptation, and co-governance principles. The local consultants should have a background in natural resource management, fisheries or conservation. Team Leader/Evaluation Expert (International). This expert will serve as Team Leader for both the FISH evaluation and the concurrent PERFORM evaluation. The Team Leader/EE will provide overall leadership for the team, and s/he will finalize the evaluation design, coordinate activities, arrange periodic meetings, consolidate individual input from team members, and coordinate the process of assembling the final findings and recommendations into a high quality document. S/he will lead the preparation and presentation of the key evaluation findings and recommendations to the USAID / Malawi team and other major partners. The TL should have the following attributes/qualifications and experience:  S/he should have a postgraduate degree in an appropriate field, with at least 10 years of international experience leading evaluation teams, ideally for natural resources management (NRM) and/or climate change projects. Relevant experience in Malawi or Eastern/Southern Africa preferred.  S/he should have extensive experience in conducting quantitative and qualitative evaluations.  The Team Leader/EE must be familiar with USAID regulations and systems including performance monitoring, gender policies and guidance, project management, budgeting and financial analysis, and reporting.  Experience in international donor development program management and overseeing multiple program areas simultaneously is preferred.  Should be experienced in preparing documents that are objective, evidence-based, and well organized.  Excellent oral and written skills in English are required.  A separate scope of work will for this position will define its role, including tasks and LOE, in the two concurrent evaluations. The LOE shown in the table below for this position is for the combined workload. Senior Fisheries Expert (International). The Senior Fisheries Specialist will be responsible for assessing the effectiveness of project implementation according to the M&E plan, while addressing the evaluation questions above. S/he will provide technical leadership with respect to the natural and man-made challenges facing sustainable fisheries in Malawi, as well as the capacity constraints that impede sustainable fisheries, at both the local and national level. S/he will also provide technical leadership on the evaluation question concerning USAID’s strategic options for future fisheries programming in Malawi. 12 S/he will participate in team meetings, and in the absence of the team leader lead key informant interviews, group meetings, site visits, and draft the sections of the report relevant to his/her expertise and role in the team. S/he will also participate in presenting the report to USAID or other stakeholders and be responsible for addressing pertinent comments provided by USAID/Malawi or other stakeholders. The Senior Fisheries Specialist will have:  A postgraduate degree in fisheries, environment, climate change, natural resource management, aquaculture or a related environmental science.  A minimum of 8 years of international field experience in fisheries work, with an understanding of co￾management and climate change adaptation principles.  M&E experience with at least one similar USAID project, or that of another international donor, and have familiarity with USAID regulations and systems. Evaluation Specialist (Local). The local Evaluation Specialist will participate in team meetings, key informant interviews, group meetings, site visits, and draft the sections of the report relevant to his/her expertise and role in the team. S/he also will participate in presenting the report to USAID or other stakeholders and be responsible for addressing pertinent comments provided by USAID/Malawi or other stakeholders. The local Evaluation Specialist should:  Have a degree in monitoring and evaluation, international development, fisheries, natural resources, environment, or a related field.  Be a Malawi national with at least 2 years of field experience in evaluation, preferably with some sectoral experience with fisheries and/or natural resource conservation. Logistics Specialist (Local). The local Logistics Specialist will be responsible for providing program logistics support, arranging appointments and maintaining the schedule, providing interpretation and translation, and assisting with the preparation of project reports, as needed. S/he will serve both the FISH and PERFORM evaluations, and the LOE shown in the table below is for the combined workload. Conflict of Interest. All evaluation team members will provide a signed statement attesting to a lack of conflict of interest, or describing an existing conflict of interest relative to the project being evaluated. USAID will provide the conflict of interest forms. See Annex 1 for the Template. VIII. Existing Sources of Information The evaluation team should consult a broad range of background documents apart from project documents provided by USAID /Malawi These should include, but are not limited to, documents on freshwater fisheries, co-management efforts, and climate change adaptation, both in and outside of Malawi. The evaluation team also should review relevant GOM national strategies and policies, as well as relevant projects and strategies of other aid agencies active in Malawi. USAID, MELS, and the FISH project will provide the assessment team with a package of briefing materials, including: ● The agreement for FISH activity; ● M&E plan for FISH; ● FISH Baseline Survey; ● Project quarterly and annual reports, work plans and management reviews developed as part of routine monitoring; ● Training reports; ● DQA reports; ● USAID/Malawi Country Development Cooperation Strategy 2014-19 (Public version); ● USAID Malawi DO2 PMP; ● FISH Indicator Tracking Table (PITT); ● M&E tools; 13 ● Training & Beneficiary database ● USAID Evaluation Policy ● Checklist for Assessing USAID Evaluation Policy IX. Level of Effort (LOE) of Study Team by Task Deliverables Below is an estimate of the evaluation level of effort (LOE). Level of Efforts of Team Members by Task Deliverables Task/Deliverable Duration / LOE in Days Team Leader/ EE Sr. Fisheries Expert Evaluation Specialist Logistics Specialist Review background documents and home-based preparation work 5 5 5 2 Pre-departure Mobilization and Logistics Arrangements 0 0 0 5 Travel to Malawi 2 2 0 0 Team planning meeting and meeting with USAID 2 2 2 2 In-country Logistics Arrangements and day-to-day logistical support 0 0 0 7 Development of Evaluation Work Plan (concurrent with document review and initial meetings) 2 2 2 0 Development of data collection instruments 2 2 2 0 Information and data collection. Includes interviews with key informants (stakeholders and USAID staff) and site visits 14 14 14 0 14 Level of Efforts of Team Members by Task Deliverables Task/Deliverable Duration / LOE in Days Team Leader/ EE Sr. Fisheries Expert Evaluation Specialist Logistics Specialist Discussion, analysis, and preliminary draft evaluation report in country including discussion with USAID 5 5 5 2 Debrief meetings with USAID (preliminary draft report due to USAID) 1 1 1 1 Debrief meetings with key stakeholders 1 1 1 1 Team Leader meets with Technical Specialists and USAID to synthesize findings/discussion 1 1 1 0 Depart Malawi/Travel to U.S. 2 2 0 0 Finalization of draft and internal review (IBTCI) remote 4 4 4 0 USAID provides comments on draft report 0 0 0 0 Team revises draft report and submits final to USAID (out of country) 10 4 4 0 USAID completes final review 0 0 0 0 Editing and formatting of report 0 0 0 0 Total Estimated LOE 51 45 41 20 X. Scheduling and Logistics Funding and Logistical Support. USAID/Malawi’s MELS project will be responsible for all off-shore and in￾country administrative and logistical support, including identification and fielding appropriate local staff. It will take care of arranging and scheduling meetings, international and local travel, hotel bookings, working/office spaces, computers, printing, and photocopying. The Logistics Specialist will arrange field visits, local travel, hotel, and appointments with stakeholders and provide translation services. Scheduling (see above table). Include a tentative evaluation schedule showing all the timings for the key tasks to be undertaken as part of this evaluation. Have a sample that I can share with you. Work is to be carried out over a period of approximately 8 weeks (9 weeks for the team Leader because of his dual responsibilities). At this point in time, we anticipate that the evaluation would begin on/about May 1st , with work in Malawi commencing on/about May 10th, field work completed on/about June 10th and final report and close out before the end of June. The anticipated May 10th start date in country provides over one month to finalize, clear the SOW, recruit and approve the consultants, and complete background research prior to arrival. See Annex1 for Malawi Mission Holiday Schedule. A six-day work week (Monday-Saturday) is authorized for the evaluation team while in Malawi, however, no overtime or premium pay is authorized. The evaluation team will submit a work plan with timeline as part of the evaluation methodology proposal and develop a GANTT chart displaying the time periods during which activities occur. Team mobilization will include: travel approval; airline tickets; visa; lodging; work facility and vehicle transport arrangements; dates for meetings with USAID/Malawi EG staff and key contacts; in-country travel agenda; and accommodations.    15 ANNEX II. EVALUATION DESIGN AND METHODS FISH CRITICAL PATH 16 FISH EVALUATION TIMELINE 17 ANNEX III. DATA COLLECTION TOOLS ANNEX 1IIA. CONTACT SHEET IBTCI – JUNE 2017 NUMBER: DATE NAME TITLE / ROLE ORGANISATION PLACE/LOCATION 18 ANNEX IIIB. KII NOTETAKING FORM Meeting: (Date, Time, Name of Individual and Institution) Text Evaluation Question Keyword 19 ANNEX IIIC. FGD & MINI SURVEY GUIDELINES Focus Discussion Group Guidelines 1. Target audience: Beach Village Committee with Participatory Fishery Management   Stakeholders; as appropriate 2. Focus Group Discussion topic:    Working together to improve fishery management.    3. Main Question:    How can national and local government officials, local communities and fishers better work together to improve fishery management?    Sub‐questions: 1.  What is the role of BVCs in managing fishing rules and fish sanctuaries? Facilitator summarizes a brief discussion among all participants and confirms BVC responsibilities to manage the sanctuary; such as marking the boundaries and enforcing no‐take rules.    2.  Does the BVC have enough resources to effectively manage the sanctuary?   Consider: legal authority, financial resources, staff, etc.? 3.  Return to the main question and make recommendations to answer it: How can national and local government officials, local communities and fishers better work together to improve fishery management? 4. Participants: Best to have representation from BVC or FA membership list; but should include representation from different sectors: national and local government, BVC and Fishery Association, fishers, and other local governance institutions; as appropriate.  At the end of the meeting, summarizes conclusions (!) from discussions, write them down and confirm consensus or not.  Offer refreshments and chance for informal reflection and discussion.  Limit to about 6 ‐ 8 people.     Facilitate broad participation and encourage all to speak and share ideas.     Document and systematize feedback.    20 Mini‐Survey Guidelines 1. Create and distribute a Mini‐Survey across five or six BVCs (and/or LFMAs) to gather opinion and perception about the impact of the FISH program at the local level.  These instruments will be applied in printed format, in the context of both selected group and individual interviews. Target respondents will represent a purposeful sample with no statistical significance. The resulting data will seek to support triangulation of findings and added qualitative across the seven evaluation questions 2. Two primary guidelines will apply across the multiple contexts addressed:   2.1. BVCs should be selected to balance the survey audience by geography and gender; among other criteria;    2.2. The survey should focus on a limited number of issues, featuring an easy to comprehend format and a limited amount of text.   3. Additional questions that may be addressed through the mini‐surveys:  What is your major problem?  What is your greatest success?  Do you know about the USAID FISH program?     Did you participate in and USAID FISH training projects?  If yes, how many and what kind?  Do you need more training?  If yes, what kind? What are your fishery management responsibilities?   (Check all that apply) Enforcing fishing rules Removing or preventing use of mosquito nets Managing a fish sanctuary or no‐take zone Training and education Other responsibilities 21 ANNEX IIID. KII MASTER TEMPLATE ANNEX IIIE. EVALUATION CHECKLIST Checklist for FISH FY17 implementation projects Output 1: Utilization of science, analysis and information for decision making increased. Evaluate progress toward establishing: 1. Fisheries knowledge management system (KMS).  Evaluate current status and future sustainability. Who maintains it?  Where?  How? 2. District Development Plans (DDPs).  How does in support fisheries management?  Fisheries chapte integrates ETOA. 3.  FSTAP progress toward establishing:    National fisheries communication strategy; and    Fisheries research agenda. 4.  Research and Information Education and Communication (IEC) materials for:  Early warning system to predict lake recessions and prepare for impacts of drought.  Deep pool refugia and river basin management plans.  Stock assessments for usipa and chambo.  Can FISH set total allowable catch (TAC) for multi‐ species lake fishery?  Brush parks in deep and shallow water to enhance fish production; especially for tilapia.  Value chain assessment to evalutae post‐harvest losses and role of women in broad fishing industry.  Changu‐changu fuel efficient household stoves for fish frying and processing. Output 2: An enabling environment for conservation and management of freshwater ecosystems will be enhanced. Review status of the governance infrastructure for capacity development and advocacy among FA an Consider: 1.  Fishery management plans and byelaws completed for 12 and approved by 4 local governments; 2.  Training for fishery technicians and extension officers to support BVCs and FAs 3.  Training for Malawi College of Fisheries (MCoF) officers; 4.  Capacity building of CBOs, LGAs and TAs to support Participatory Fisheries Management (PFM); 5.  Advocacy strategy aimed to enable LFMAs to self‐advocate for fisheries issues in the DD; and   6.  IEC materials developed to communicate both the economic value and nutritional importance of f resources, including:  Fisheries Extension Guide to the six PFM policy steps  Policy briefs on economic importance of fisheries and financing of the fisheries sector  Good Governance Barometer Malawi Facilitator’s guide  Lake‐wide network maps    Guide to BVC/FA organization and networking  Training guide on gender mainstreaming in FISH project Output 3: To reduce priority threats to freshwater biodiversity:   Respond to threats to fishery biodiversity conservation that emanate from land‐based sources and la water bodies.  Specific activities will: 1. Establish fish sanctuaries to protect key fish breeding habitats around river mouths leading to enhanced fish recruitment and productivity; 2. Enhance brush parks in sanctuaries to deter illegal fishing and increase fishery biodiversity and 23 productivity.   3. Establish FAs and sub‐FAs to implement PFM and fisheries biodiversity conservation best practices to reduce illegal fishing and mitigate land based pollution threats   4.  Expand training activities to focus on:  VNRMC/RVC groups to protect catchments, regenerate forests, and rehabilitate river riparian zones to limit soil erosion impacts in key breeding areas;  farmers in agroforestry and soil and water conservation practices to reduce land‐based erosion impacts on key fish breeding areas Output 4: To encourage adoption of climate change adaptation (CCA) measures that support resilience of communities and freshwater ecosystems: Implement climate smart and diversified livelihood CCA best practices to enhance resilience to climate change impacts. focus on forest regeneration, river bank protection and adoption of climate smart agriculture.  Specific training activities focus on: 1. Households to regenerate forests in critical catchments   2. Beekeeping groups in beekeeping practices and finance management.   3. Shoreline farmers in climate smart agriculture and soil and water conservation; 4. Households in climate smart Integrated Aquaculture Agriculture (IAA)   5. Households from RVCs to protect deep hole refugia and apply climate smart practices to protect fish during lake recession events and drought years. 6. Farmers to adopt adopting water‐efficient rice production and wetland conservation. 7. VSLA groups and households trained to increase literacy, marketing and entrepreneurship skills of the group established using loans from the savings groups. 8. Households to and fish processors to adopt fuel‐efficient changu‐changu domestic stoves and smoking kilns and cookers to reduce firewood consumption; 9. Beach model projects to integrate sanitation, safety at sea, fish handling and processing best practices (solar driers, kilns and cookers) to enable wider adoption in beach communities 10. Fish processing and fish trader groups to reduce post‐harvest losses.    24 Focus Discussion Group Target audience: Beach Village Committee with Participatory Fishery Management   Stakeholders; as appropriate Focus Group Discussion topic:    Working together to improve fishery management.    Main Question:    How can national and local government officials, local communities and fishers better work together to improve fishery management?    Sub‐questions: 1.  What is the role of BVCs in managing fishing rules and fish sanctuaries? Facilitator summarizes a brief discussion among all participants and confirms BVC responsibilities to manage the sanctuary; such as marking the boundaries and enforcing no‐take rules.    2.  Does the BVC have enough resources to effectively manage the sanctuary?   Consider: legal authority, financial resources, staff, etc? 3.  Return to the main question and make recommendations to answer it: How can national and local government officials, local communities and fishers better work together to improve fishery management? Participants: Best to have representation from BVC or FA membership list; but should include representation from different sectors: national and local government, BVC and Fishery Association, fishers, and other local governance institutions; as appropriate.  Limit to about 8 people.  Facilitate broad participation and encourage all to speak and share ideas.  Take good notes.  At the end of the meeting, summarizes conclusions (!) from discussions, write them down and confirm consensus or not.  Then have some cookies.      25 Mini Survey Create and distribute a Mini Survey across five or six BVCs (and/or LFMAs) to gather opinion and perception about the impact of the FISH program at the local level.  BVCs should be selected to balance the survey audience by geography and gender; among other criteria.  The survey should focus on a few issues: What are your fishery management responsibilities:  Check all that apply:  Enforcing fishing rules  Removing or preventing use of mosquito nets  Managing a fish sanctuary or no‐take zone  Training and education  Etc. What is your major problem? What is your greatest success? Do you know about the USAID FISH program?    Did you participate in and USAID FISH training projects? If yes, how many and what kind? Do you need more training?  If yes, what kind? Thank you for your participation and help to improve fishery management. USAID, etc. 26 ANNEX IV. FISH EVALUATION QUESTIONS MATRIX 27 28 29 30 31 ANNEX V. KEY INFORMANTS INTERVIEWED KEY INFORMANT INTERVIEWS (KIIs) DATES ORGANIZATION POSITION NAMES TOTAL NUMBER OF PEOPLE PARTICIPATED 5/7/2017 Department of Fisheries Senior Deputy Director ,Chief Fisheries Officer, Chief Fisheries Research Officer Steve Donda,Brino Chirwa,Geoffrey Kanyerere 3 8/7/2017 Independent Consultants Malawi Gender Experts Habiba Osman, Edfas Mkandawire 2 10/7/2017 Department of Forestry District Officer 1 10/7/2017 PACT FISH DCOP and FISH Governance Specialist Daniel Jame and Dick Kachilonda 2 11/7/2017 UNWomen Malawi Gender Officer ‐ UNWomen Reports Edfas Mkandawire 1 12/7/2017 Madeco Madeco Fishing Company Mr Peter 1 13/7/2017 Mangochi District Assembly District Commissioner and DPD Mr M.O Chimphepo and Mr W Chikuni 2 14/7/2017 Mangochi District Assembly District Fisheries Officer Thom Nyasulu 1 15/7/2018 UNWomen National Coordinator, UNWomen, Malawi Pamela Mkwamba 1 17/7/2017 Machinga District Assembly Director of planning and Development and District Fisheries Officer Mr M. Mwakhwawa and Mr G Mwadzaangati 2 18/7/2017 Balaka District assembly Director of Planning and Development and District Fisheries Officer Violet Kamasumbi and Austin Malizeni 2 20/7/2017 Zomba town assembly DOA and DFO Mlambuzi Hastings and Lapkin Chikoko 2 21/7/2017 Emmanuel International PM Martin Phiri 1 21/7/2017 World Fish Country LEAD Joseph Nagoli 1 24/7/2017 CEPA Program officer&ED Stanley Mvula & William Chadza 1 24/7/2017 Chambo Fisheries Industrial Manager Muhammed Ismail Aboo 1 24/7/2018 LUANAR Head of Dept ‐Fish Genetics Wilson Wesley Lazaro Jere, PhD 1 25/7/2018 Christian Aid Program Manager David Nthakomwa 1 26 32 GROUP INTERVIEWS 10/7/2017 PACT project Staff Implementing Partners Daniel Jamu,Issaquissa Mwanjabe Goodwell Thunga,Yusuf Mtila,Joseph Makwakwa,Essau Chisale 6 12/7/2017 Malawi College of Fisheries Malawi College of Fisheries Staff Dr Y Phiri;E Mogha; N Msusa;Dorris Msukwa;M Daziloni;Kanthenga 6 13/72017 Fisheries Research Unit Fisheries Research Unit James Banda, Harold Sungani, Orton Msiska, Salim M’balaka, Barnett Karphuka 5 17 33 FOCUS GROUP DISCUSSION INTERVIEW 10/7/2017 CISER FISH Techs Damiano Manda,Gift Chigona,Christopher Chisesa,Chikondi Banda 4 13/7/2017 Msaka Fish Processors Fish Traders Justen Banda,Tango Leonard,Lilian Chimphamba,Madalitso Chirwa,Edina Mkandawire 5 14/7/2017 Malembo BVC Members of BVC Fosiko January,Delia Benjamini,Hezekiah Mkaipila,Kelvin Kossam 4 17/7/2017 Mwayiwathu VNRMC & Bee keepers Members of VNRMC & Beekeepers Fred Sita,Suwema Robson, Ethel Chidou,Masautso Chidothi,Christopher Banda,Bonex Suwedi 6 18/7/2017 Mpira Bee Keeping Association Members of Association Musa Kalima,Onex Ibra,Ibrahim Ghana 3 18/7/2017 Mtiule VNRM Members of VNRMC Duncan Killi,Emmanuel Mangaziwa,Patuma Wyson 3 19/7/2017 Aduwa(Lake Chiuta)BVC and Fisheries Association Members of BVC and Fish Association Raphael Muluku,Chrissy Kadangwe,Eneless winesi,Modestar Mtotera,Jaweria daudi,Ntonda saoneka,Kazembe yuda,Jackson Chamasowa,Eliass Mapemba,Witiness Kamwendo 24 17/7/2017 Tisamalire VNRMC/Bee keepers Tisamalire VNRMC/Bee keepers Ruth Kalambule,Jailos Bamus,Tailos Chimombo 15 20/7/2017 Lake Chilwa Fisheries Association Members Of Lake Chilwa Fisheries Association Anderson Thembwa,Maxwell Bandawe,Mary Jailos,Dave Mangungu 10 21/7/2017 Chaone Village Loans and Savings Members of Chaone Village Loans and Savings Zachariah Msamila,Enest Jimu,Dagalasi Somanje 12 21/7/2017 Mpyupyu VNRMC Members of Mpyupyu VNRMC Dalitso Juma,Tuwenji Mitambo,Alex Moffolo 17 123 34 ANNEX VI. THEORY OF CHANGE ANALYSIS 35 36 37 …are based on shared, evidence-based objectives and learning …are grounded in inclusive and effective ecosystem￾scaled governance structures …strengthen the assets of communities EVALUATION QUESTION 3. How effective have efforts such as campaigns against using mosquito nets, no-fishing zones/time periods, fish sanctuaries, and vessel monitoring systems actually been in countering over-fishing? OUTPUT 1: UTILIZATION OF SCIENCE, ANALYSIS, AND INFORMATION FOR DECISION MAKING INCREASED  Access to science, analysis and information;  Understanding threats to biodiversity and options for conservation;  Understanding climate change effects on freshwater ecosystems and adaptation options.  Campaign messages developed in English and three local languages (Chichewa, Yao and Tumbuka) track the resource users in all targeted water bodies  Research and application of the knowledge promoted by FISH is intended to extend from fisheries management to post￾harvest improvements (e.g. reduction of loss of fish)  ETOA and aerial surveys have guided District Councils to target specific areas of ecological importance and BVC’s are better informed to locate sanctuaries near their communities OUTPUT 2: ENABLING ENVIRONMENT FOR CONSERVATION AND MANAGEMENT OF FRESHWATER SYSTEMS ENHANCED  Legal Framework improved  Transparency, representation, and accountability in decision￾making improved  Institutional and community capacities strengthened.  Uptake has been faster in Enhanced Catchment Areas by harnessing the benefit from a multiplier effect of combined technologies at community or catchment levels  Evidence suggests that BVCs and VNRMCs implementing and managing protected areas, understand the importance of protecting coastal habitats, responsive to a community￾supported mandate  FISH has proposed a revenue sharing model for the fishery sector to benefit local institutions such as the BVCs and VNRMCs as means to reduce illegal fishing OUTPUT 3: PRIORITY THREATS TO FRESHWATER ECOSYSTEM BIODIVERSITY REDUCED  Improved natural resource management practices evaluated and promoted;  Habitat restoration and riparian conservation measures stimulated and supported;  Sustainable Fishing Practices evaluated and promoted.  Messages have a balanced emphasis on both threats (use of illegal fishing gear) and advantages (observance of closed season) in the context of catchment management  Based on raised community awareness and engagement, FISH is now moving to implement additional conservation and NRM (threat reduction) activities supported by BVC/VNRMC  Messages on threat reduction target pre-identified traditional leaders where materials distribution (e.g. calendars on illegal use of mosquito nets) supports direct on-the-ground interaction with target audiences (BVC, RVCs, FAs) OUTPUT 4: ADOPTION OF CLIMATE CHANGE ADAPTATION MEASURES THAT SUPPORT RESILIENCE OF COMMUNITIES AND FRESHWATER ECOSYSTEMS INCREASED  Ecosystem-based adaptation solutions identified, evaluated and promoted;  Alternative, scalable and sustainable climate resilient livelihood options stimulated, evaluated and promoted.  Content disseminated introduces research￾supported message threads on conservation and adaptation options  The training process aims to work with committees and the Climate Smart Agricultural groups practicing new technologies.  BVC capacity building leverages effective fish stock management as a community empowerment tool TOC CONDITIONS if decisions around fisheries RESULTS LOGIC Causal linka ges 38 39 40 41 OUTPUT 1 OUTPUT 2 OUTPUT 3 OUTPUT 4 Illustrative Contribution  CEPA’s similar role in ECRP as a technical partner to Christian Aid on policy and advocacy as well as knowledge and information, provide ideal grounds for synergies in the mobilization of public sector support aligned with FISH’s work across multiple coincident districts as well as within Central Government  Supported by significant coordination with other development partners and line ministries at district level, FISH engages the promotion of conservation agriculture, water harvesting techniques to decrease run-off, and the protection of wetlands, critical spawning habitats, and refugia deep holes  Successful engagement of selected watershed management improvements (control of erosion; ecosystem protection and restoration), opens opportunity for knowledge sharing/transfer and coordinated action with a number of coincident initiatives (bilateral and multilateral)  FISH outputs overlap with comparable biodiversity and watershed programs, like FAO-GEF on Lake Malombe and Lake Malawi, UNDP-GEF on climate resilience support to Mangochi and Machinga  FISH supports improvements in post￾harvest value losses of fish products Outcomes Associated with Contribution  Initiatives such as the USG￾funded DREAMS, as well as the Local Development Fund Initiatives and the Lake Malawi Basin Project (World Bank) represent coordinated interventions with complementary to FISH outcomes  FISH tapped into human resources and on-the-ground experience previously developed by the Wellness in Agriculture for Livelihoods Advancements – WALA (also linked by EI) and supporting the recruitment of FISH Technicians  FISH has been able to move forward in formalizing its collaboration with PERFORM to leverage geographic coverage and content delivery overlaps in containment and restoration efforts targeting both, income generation and broader ecosystem impact  There is a clear alignment of FISH outcomes on adaptation and resilience under the results area of ``Sustainable Livelihood Increased`` (CDCS DO2) for the USAID Feed the Future initiative Negative Factors Affecting Contribution  Studies and preparations of evaluating best practices supporting Output1 were delayed in starting due to longer than anticipated local partner contract development  ETOA, APEA, GGB, CPI and socio-economic baseline studies were not complete until 15 months following project inception  Weak emphasis knowledge sharing/lessons when considering coordinated action with overlapping initiatives  Weak targeting of existing gaps in comprehensive biodiversity conservation and integrated watershed mgmt  Location-driven collaboration has not evolved into a formally structured shared learning approach that fosters locally driven knowledge building Challenges affecting FISH Objectives  Extension services that is sensitive to both settings and actors will require more immediate and agile knowledge transfer that is relevant to context and can readily be adapted  FISH will have to effectively build on current advocacy efforts to encourage policy changes promote greater integration and collaboration at the local level  ToT is not an effective way to encourage context sensitive innovation or rapid evolution in knowledge building and thus, FISH may need to rethink how to best support a shared learning approach  Leveraging of current FISH investments in on-the￾ground presence to further integrate parallel initiatives will require supplementary inputs and better follow-up to more effectively link activities at the community level CONTRIBUTION STORY Q5 42 …are based on shared, evidence-based objectives and learning …are grounded in inclusive and effective ecosystem-scaled governance structures …strengthen the assets of communities EVALUATION QUESTION 6. What if any adjustment could be made to improve project effectiveness? Are there lessons learned that have broader applicability for USAID Malawi and beyond? OUTPUT 1: UTILIZATION OF SCIENCE, ANALYSIS, AND INFORMATION FOR DECISION MAKING INCREASED  Access to science, analysis and information;  Understanding threats to biodiversity and options for conservation;  Understanding climate change effects on freshwater ecosystems and adaptation options.  Assess the performance of LFMAs and DoF in Usipa and Chambo management and leadership capacity building needs  Build leader competence and confidence through collaborative research serving diverse interests  Continue tracking pressure on fish biodiversity over time by supporting DoF in conducting FRAME surveys (number of fishermen, boats, and equipment used in the four lakes), including the post￾harvest sector  Continue helping the communities to monitor sanctuaries in Lake Malombe and Lake Malawi by taking a more supportive role to help them refine their skills for independent monitoring OUTPUT 2: ENABLING ENVIRONMENT FOR CONSERVATION AND MANAGEMENT OF FRESHWATER SYSTEMS ENHANCED  Legal Framework improved  Transparency, representation, and accountability in decision￾making improved  Institutional and community capacities strengthened.  Support high profile multi￾stakeholder dialogue sessions convened and led by FSTAP review, define and prioritize fisheries best practices  Develop integrated spatial plans and maps of critical habitats, sanctuaries, and areas earmarked for development  Promote best practices in riparian conservation with front-line workers  Develop a comprehensive strategy to guide the roles and interaction of relevant stakeholders to engage in the implementation of management plans OUTPUT 3: PRIORITY THREATS TO FRESHWATER ECOSYSTEM BIODIVERSITY REDUCED  Improved natural resource management practices evaluated and promoted;  Habitat restoration and riparian conservation measures stimulated and supported;  Sustainable Fishing Practices evaluated and promoted.  Move from piloting to scale up of biodiversity conservation and habitat restoration  Effectively engage local partners and extension personnel to plan and implement efforts to scale up/out  Conduct capacity building of fisheries BDC and CCA best practices  Improve monitoring of illegal gears such as mosquito nets supported by alternative vector control tools (VectorWorks)  Improve biodiversity conservation guidelines and regulations for catchment management, riparian habitats and fishery management  Continue wet and dry season monitoring of deep pool sanctuaries  Assess effectiveness of adaptive management measures to improve the performance of sanctuaries OUTPUT 4: ADOPTION OF CLIMATE CHANGE ADAPTATION MEASURES THAT SUPPORT RESILIENCE OF COMMUNITIES AND FRESHWATER ECOSYSTEMS INCREASED  Ecosystem-based adaptation solutions identified, evaluated and promoted;  Alternative, scalable and sustainable climate resilient livelihood options stimulated, evaluated and promoted.  Develop a scaling framework to assist stakeholders in the rapid scale up of processing technologies (leveraging value chain improvements)  Collaborate with other ongoing climate proofing initiatives to increase capabilities at the district level on drought predictions and forecasting  Support implementation of improvements that address post-harvest value losses of fish products TOC CONDITIONS if decisions around fisheries RESULTS LOGIC Causal linkages supported by 43 ANNEX VII. GENDER CONTEXT FOR FISH Brief Context to Findings Malawi presents a complex context to promote changes in gender dynamics and advance gender mainstreaming. Responses from evaluation interviews with UNWomen outlined three mandates in moving forward in this work: Normative, Coordination; Implementation (policy and legal framework). However, responses point out that, although quotas are the current path to change, Malawi has faced important set back in recent years and has even witnessed the erosion of previous gains (e.g. previous advancement in Parliament has more recently shifted from 15 to 12%). At the village level, the prevalent obstacles are supported by common practices hosted that often prevent addressing the root cause (e.g. gender prejudice in addressing problems such as “sex for fish”) or women lacking access to NRM as direct beneficiaries. Value chains in Malawi also present uneven conditions for women participation, where they lack access to participate in opportunities for higher yields (e.g. the value of fresh fish as opposed to processed/dry fish or high value fish) and lack access to seed capital or critical market connections. In this respect, a study2 that assessed the cost of gender gap made important contributions highlighting how the power dynamics affecting women’s productivity reflect diminished participation based on their ability to make decisions that directly affect productive activities. For example, as the report renders, in Malawi women make decisions on about 26 percent of all agricultural plots; 76 percent of these plots are also actually owned by them, suggesting a strong relationship between ownership and decision-making power, but there is no one-to-one correspondence between plot management and land ownership or household headship. Gender-focused Capacity Building Hosted by FISH through its implementing partner CEPA, a guide for Gender and Youth Analysis was developed in April 2017. This initial guide was subsequently updated closer to the timeframe of the Mid￾term Evaluation and, based on discussions with the external consultant leading this effort, the original terms of reference were enhanced to include a Gender and Youth Strategy that could then be implemented through corresponding training actions. Initial training focused on the skills involved and vantage point of the extension work undertaken by FISH. This initial training hosted by CEPA involved the FISH technicians recruited by all implementing partners, as well as those employed by the Fisheries and Forestry Departments. Subsequently, the rollout of these actions targets both broader community awareness activities, as well as more focused work with Village Savings and Loans and fish producer groups led by women across the different lake and community settings covered by FISH. Interviews with gender experts to discuss the strategy and the implications of the rollout highlighted two aspects: (1) the key role of the technicians and extension agents and (2) the critical follow-up that will be required to ensure that the initial training inputs under this effort could be effectively supported and further extended by appropriate protocols. Ultimately, capacity building should ensure that                                                               2 THE COST OF THE GENDER GAP IN AGRICULTURAL PRODUCTIVITY in Malawi, Tanzania, and Uganda. This report, published in 2015, was a joint product of UN Women, the United Nations Development Programme–United Nations. Environment Programme Poverty-Environment Initiative (UNDP-UNEP PEI) Africa, and the World Bank. The collaboration was led by UN Women, Eastern and Southern Africa Regional Office (ESARO). 44 local actors will be willing and able to continue this work as a fundamental element of their productive efforts within each distinct lake and community setting. Technicians and Extension Agents: Both FISH Technicians and Extension Agents currently constitute the “frontline” of FISH efforts in addressing gender dynamics in each community setting. Evidence from interview responses suggests that while extension agents from both the Fisheries and Forestry Departments appeared progressive in their understanding of gender issues, particularly in terms of context-specific challenges and drivers of change. Although, across the initial training exercise with technicians and extension agents, no perceptible differences in their education, the latter seemed to have access to higher levels of previous training, where most have a diploma from Natural Resources. Moreover, knowledge and use of participatory appraisal and learning tools are mandatory for extensionists and had been included in their previous training. Feedback from the training outcomes also suggests that extension agents also appear to have relevant experiential knowledge, considered key in successfully engaging gender issues. This knowledge was not as present or tangible in the contributions and interaction with technicians recruited by FISH. Interviews highlighted the presence and participation of higher level GoM and District officers during the trainings, which seemed to encourage the active and contributing role of the extension agents. Critical follow-up: The success of the PFM model is based on increased and improved participation in all aspects of co-management. This highlights the need to increase community uptake, which in turn is linked to household dynamics. Responses suggest that initial training provided FISH useful feedback to better adjust the intended Gender and Youth Analysis strategy. Moreover, feedback suggests that follow￾up actions should transcend training (or ToT) and center on a much-needed mentoring/coaching actions as FISH Technicians and Extension agents pursue the achievement of strategy milestones. As gender protocols are implemented and the Gender Mainstreaming Strategy that FISH intends to support in Years 4 and 5 continues to evolve, responsive to emerging and specific challenges and opportunities, “frontline officers” will need to be supported in their ability to create viable spaces for community engagement that best ensures the noted link to household dynamics and its direct impact on decision-making. In addition, congruent with building the capacity of technicians and extensionists, program managers and supervisors may also require capacity building. In this, responses alluded to not only enhancing a congruent understanding of how to engage gender dynamics to strengthen PFA but also, to ensure that reporting nestles and strategically positions critical feedback from the field. These feedback loops are of particular importance in addressing resistance to gender issues. In the case of Fisheries and Forestry extension agents, feedback becomes a way to anticipate sensitivities that have historically emerged in official mid￾level reporting and, by extension, to the Ministry Level Joint Sector review report shared with international development partners; Towards Promoting Changes in the Gender Dynamics in FISH Target Groups CEPA developed five protocols to operationalize the gender mainstreaming, addressing issues such as specific hindrances to the participation of women, child labor, women leadership and empowerment, and NRM and fisheries issues. CEPA’s implementation is based on community mobilization, where the guidelines include a template to facilitate the engagement of target groups. The protocols center on the identification and support of champions (including men and boys) by developing community-owned profiles 45 that speak to their immediate setting. The protocols consider appropriate dissemination, banking on the opportunity and freedom that community-based resources have. Hence, for example, community radios have ample freedom and flexibility in their use of content. Community Theater has significant links to culture and tradition and is thus, a credible and legitimate option for hosting both content distribution and community discussion of desirable change. Respondents emphasized the need for FISH to further focus on household dynamics (where man has resources and women does not) affecting significant participation of women in opportunities opened by improvement in the value chain. Village Savings and Loans (VSL) providing informal credit with a social fund (deductions on loans that benefit community purposes) are effective vehicles for transforming present dynamics (described as high as 90% women membership). 46 ANNEX VIII. Conflict of Interest Form Template 47 48 49 ANNEX IX. STATEMENT OF DIFFERENCES There are no statements of differences from any of the evaluation team members.