USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 Cover Photo: Hilsa Fisherman ENHANCED COASTAL FISHERIES IN BANGLADESH MIDTERM PERFORMANCE EVALUATION REPORT February 2018 This publication was produced at the request of the United States Agency for International Development (USAID). The report was prepared by Ian M. Dutton, M. Shahadat Hossain, and Humayun Kabir for the Accelerating Capacity for Monitoring and Evaluation (ACME) activity held by International Business and Technical Consultants, Inc. (IBTCI). USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 i MIDTERM PERFORMANCE EVALUATION REPORT OF USAID/BANGLADESH ENHANCED COASTAL FISHERIES (ECOFISH) PROJECT February 2018 Contract No. AID-388-C-14-00001 International Business & Technical Consultants, Inc. 8618 Westwood Center Drive Suite 400 Vienna, VA 22182 USA DISCLAIMER The views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Cover Photo: Fisherman displays his catch of hilsa. Photo by ECOFISH USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 ii TABLE OF CONTENTS ACKNOWLEDGEMENTS vi EXECUTIVE SUMMARY 1 1. INTRODUCTION 6 2. DEVELOPMENT PROBLEM AND USAID’S RESPONSE 7 2.1 The Development Problem 7 2.2 USAID Response 8 2.3 ECOFISH Goals and Outcomes 10 3. Evaluation Purpose 12 4. EVALUATION METHODS AND LIMITATIONS 13 4.1 Evaluation Approach 13 4.2 Data Collection Methods 13 4.3 Limitations 15 5. EVALUATION FINDINGS 16 5.1 Question 1- ECOFISH Performance 16 5.1.2 Conclusions 24 Question 1 – ECOFISH Performance: Summary of Recommendations 25 5.2 Question 2 – Opportunities 25 5.2.1 Findings 25 5.2.2 Conclusions 27 Question 2 – Opportunities: Summary of Recommendations 27 5.3 Question 3 – Livelihoods and Resilience 28 5.3.1 Findings 28 5.3.2 Conclusions 30 Question 3 – Livelihoods and Resilience: Summary of Recommendations 30 5.4 Question 4 – Co-management 30 5.4.1 Findings 30 5.4.2 Conclusions 32 Question 4 – Co-management: Summary of Recommendations 33 5.5 Question 5 – Gender Engagement 33 5.5.1 Findings 33 5.5.2 Conclusions 36 Question 5 – Gender Engagement: Summary of Recommendations 36 USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 iii 5.6 Question 6 – Future Strategy 36 5.6.2 Conclusions 38 5.6.3 Question 6 – Future Strategy: Summary of Recommendations 38 Annex I: Evaluation Statement of Work 39 Annex II: Evaluation Design Matrix 52 Annex III: Data Collection Instruments and Limitations 57 Annex IV: Sources of Information 72 Annex V: Performance Indicators Analysis 84 Annex VI: Editorial Quality Control 87 ANNEX VII: Conflict of Interest and Team Composition 89 LIST OF FIGURES AND PICTURES Figure 1. Trends in Total Hilsa Catch in Bangladesh 1991–2017 6 Figure 2. Regional Variability in Marine and Riverine Catches of Hilsa 7 Figure 3. ECOFISH Activity Geographic Focus 7 Figure 4. ECOFISH-Bangladesh Results Framework (v.4) 8 Picture 1. Young Boy with Fish Nets 9 Picture 2. Focus Group Discussion in the Field 12 Picture 3. Goats from Alternative Income Generating Activity 18 Figure 5. Model Co-management Structure 27 Figure 6. Gender Disaggregated Participation in HCG Training 30 Figure 7. ECOFISH Evaluation Fieldwork Itinerary 51 Figure 8. Example of Inadequately Explained ECOFISH Data 80 LIST OF TABLES Table 1. Participation in Evaluation Meetings and Surveys by Gender 12 Table 2. Progress on IR 1 Targets 15 Table 3. Progress on IR 2 Targets 16 Table 4. Co-management Infrastructure Established Under ECOFISH 2015–2017 17 Table 5. Progress on IR 3 Targets 18 Table 6. Progress on IR 4 Targets 19 Table 7. Annual (2016–2017) Income of Fishers From Fishing and Other Sources 24 Table 8. Distribution of ECOFISH Activity Beneficiaries by District 52 Table 9. Reporting of USAID-ECOFISH Activity Indicators 77 USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 iv ACRONYMS AND ABBREVIATIONS ACME Accelerating Capacity for Monitoring and Evaluation AIGA Alternative Income-Generating Activities BD BDT BFRI Bangladesh Bangladesh Taka Bangladesh Fisheries Research Institute BHRRDS Bangladesh Human Rights and Resources Development Society CBFM CBO Community-based Fisheries Management Community-based Organization CDCS Country Development Cooperation Strategy CFG CGIAR CNRS Community Fish Guard Consultative Group for International Agricultural Research Center for Natural Resource Studies COAST Trust Coastal Association for Social Transformation Trust CODEC COP Community Development Centre Chief of Party CPUE CRC/URI Catch per Unit Effort Coastal Resources Center at the University of the Rhode Island CSG Community Savings Group (also referred to as Hilsa Savings Group) DC Deputy Commissioner DD Deputy Director DEC Development Experience Clearinghouse DFO District Fisheries Officer DOF Department of Fisheries EAFM1 EBFM Ecosystem Approach to Fisheries Management Ecosystem-Based Fisheries Management ECOFISH Enhanced Coastal Fisheries in Bangladesh FGD Focus Group Discussion GDP GOB Gross Domestic Product Government of Bangladesh HCG Hilsa Conservation Group HFMAP Hilsa Fisheries Management Action Plan HGG Hilsa Ghat Groups HH HQ Household(s) Headquarters IBTCI International Business & Technical Consultants, Inc. IEM llED Information Exchange Meetings International Institute for Environment and Development IP IR Implementing Partner Intermediate Result IUCN International Union for Conservation of Nature and Natural Resources KII Key Informant Interview LOP M&E Life of Project Monitoring and Evaluation MEL(P) Monitoring, Evaluation, and Learning (Plan) 1 For an explanation of how EAFM and EBFM differ and overlap, see Osgood (2014). USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 v MOFL MPA Ministry of Fisheries and Livestock (GOB) Marine Protected Area MSY Maximum Sustainable Yield NCE No-Cost Extension NGO Nongovernmental Organization NMFS NOAA NRM US National Marine Fisheries Service (part of NOAA) US National Oceanographic and Atmospheric Administration Natural Resources Management PES Payment for Environmental Services SOW Scope of Work SP Police Commissioner SUFO Senior Upazila Fishery Officer TOC Theory of Change UFO Upazila Fishery Officer ULO UNDP Upazila Livestock Officer United Nations Development Program UNO Upazila Nirbahi Officer (Upazila/sub-district-level head of govt. administration) UP USAID Union Parishad (lowest local government administrative unit) United States Agency for International Development USAID/BD United States Agency for International Development/Bangladesh USD United States Dollar USG WB United States Government World Bank WCS Wildlife Conservation Society WFC WorldFish Centre USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 vi ACKNOWLEDGEMENTS The evaluation team would like to acknowledge the support and assistance provided by IBTCI staff based in Dhaka and Washington, D.C. Particular thanks are due to staff who accompanied the evaluation team in the field and who played a critical role with logistics and security coordination. The team would also like to express appreciation to current and former USAID/Bangladesh Mission staff involved with the ECOFISH activity, who provided valuable insights into activity history and evolution as well as guidance during the evaluation process and feedback on initial findings. The team is grateful to ECOFISH-Bangladesh staff, who arranged for a comprehensive briefing to the team, accompanied the team for sections of the fieldwork, and subsequently provided additional information on technical and programmatic issues. The team also acknowledges the assistance of the Director-General and District Officers of the Bangladesh Department of Fisheries who provided the evaluation team with in-depth briefings on Bangladesh fisheries and who facilitated meetings throughout Bangladesh. Finally, we would like to express our sincere appreciation to the hundreds of Bangladesh civil servants, business people, NGO staff, scientists, fishermen and women, community representatives, and activity participants who contributed their experience and stories to the team. We respect and appreciate the trust you placed in us. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 1 EXECUTIVE SUMMARY ACTIVITY PURPOSE AND BACKGROUND The United States Agency for International Development (USAID)/Bangladesh contracted International Business & Technical Consultants, Inc. (IBTCI), through the Accelerating Capacity for Monitoring and Evaluation (ACME) activity contract (AID-388-C-14-00001) to conduct a midterm performance evaluation of the Enhanced Coastal Fisheries in Bangladesh (ECOFISH) Activity. ECOFISH is a $15 million partnership with the Government of Bangladesh (GOB) and WorldFish to (a) enhance ecosystem resilience through the establishment of effective co-management in hilsa shad (Tenualosa ilisha) fish sanctuaries in the Meghna River ecosystem; and (b) strengthen community resilience by improving the savings, livelihoods, and coping strategies of marginalized and extremely poor fisher folk, particularly women, thereby reducing their risk from climate-affected shocks and stresses. ECOFISH-Bangladesh commenced on June 1, 2014, and will end on May 30, 2019. PURPOSE OF THE EVALUATION The primary goals of this midterm performance evaluation are to (a) provide evidence so the United States Agency for International Development/Bangladesh (USAID/BD) can determine whether the ECOFISH activity has achieved its desired results to date; (b) inform what course corrections, if any, are needed in the time remaining of implementation; (c) determine how effectively this activity has incorporated gender in the interventions; and (d) recommend to USAID key strategic and programmatic options for future engagement. USAID outlined six questions for the evaluation to address (see below). EVALUATION METHODOLOGY The evaluation team used a mixed-methods approach to gather both qualitative and quantitative data from primary and secondary sources, with specific attention to gender sensitivity and sex-disaggregated data. The team reviewed relevant documents and data provided by USAID and ECOFISH as well as a significant body of published information on hilsa fisheries and ecosystem-based approaches to fisheries management. The following five (out of nine) activity ECOFISH districts (see Table 8) with the largest number of beneficiaries were selected for data collection: Bhola, Barisal, Patuakhali, Laxmipur, and Chandpur. The team conducted Key Informant Interviews/Information Exchange Meetings (KIIs/IEMs) with 93 individuals, and 12 Focus Group Discussion (FGD) meetings with 108 individuals. Team members also conducted informal interviews and field observations during visits to 12 ECOFISH villages and administered mini-surveys on 546 residents of both ECOFISH and non-ECOFISH villages. An additional mini-survey was also undertaken with 10 scientists involved in hilsa research. FINDINGS AND CONCLUSIONS Question 1. To what extent is the ECOFISH activity on track, in terms of progress and outcomes, to meet its overall goals? What factors (internal/external) affected timely achievement of targets? And what factors impeded performance? While the performance of some strategies under ECOFISH has been variable over the first three years, ECOFISH is on track to achieving activity goals. The large increase in hilsa fish catch in 2016 and 2017 exceeds historical increases in hilsa catch rates. The only major difference in hilsa management since 2014 is the start of ECOFISH, which operates at a sufficient scale to impact the entire hilsa fishery. On the basis of this evidence, the evaluation team believes it is plausible to attribute the increase, at least in part, to specific ECOFISH activity interventions since 2014. Most notable among those interventions are the establishment of increasingly well-enforced seasonal restrictions on hilsa and jatka (juvenile hilsa) catches, expanded community engagement in fisheries co- USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 2 management, targeted public awareness campaigns, and the provision of alternative income-generating activities (AIGAs) which, together with the GOB food (rice) subsidies and increased enforcement activities, have diminished incentives for catching hilsa during ban periods. There is also strong evidence from ECOFISH surveys of increased income for beneficiary fisher communities across the ECOFISH area of operations primarily resulting from interventions such as community savings groups (CSGs), occupational and skills training, and AIGAs (such as goat rearing). Together with a more coordinated approach between GOB agencies and with communities in hilsa fisheries management, community resilience is being improved, although these communities remain vulnerable to climate-affected shocks and stresses. Key contributing factors to the achievement of targets include close alignment and a strong working relationship between ECOFISH implementing partners (IPs) and the Department of Fisheries (DOF) at all levels, a comprehensive public awareness campaign by ECOFISH and the collaborative working relationships established by ECOFISH implementing partners with communities. The CSGs, Hilsa Fisheries Guards, and other co-management forums have created new opportunities to engage communities directly in fisheries management. Factors limiting progress have included insufficient ECOFISH resources devoted to key fisheries research questions and insufficient policy capacity at the national level. Question 2. Given the time remaining, what opportunities exist to enhance and strengthen programmatic effectiveness such as by adding, changing, rescaling, and/or removing activities to meet or surpass targets/objectives, within the stipulations of the ECOFISH agreement? The evaluation team identified eight opportunities to enhance program effectiveness during the remaining life of the activity, including the following: ● A focus on leveraging the proposed $200–500 million World Bank (WB) Bangladesh Fishery project ● Scaling back of planned expansion of field programs in Year 4 to enable focus on the model village program and other policy initiatives at local, district, and national levels ● A more deliberate co-management strategy that incorporates a clear analysis of how co￾management structures will be coordinated and sustained ● Scaling back of current effort on ecosystem approaches to fisheries management (EAFM) ● Further engagement in rice subsidy programs administered by the GOB to ensure they are more complementary to other incentive programs to conserve hilsa fisheries ● Acceleration of efforts on the National Strategic Action Plan for Hilsa Conservation ● Development of payment for ecosystem services (PES) options, especially those that will contribute to the proposed Hilsa Conservation and Development Fund ● Development of a robust exit and transition strategy Question 3. Is there evidence that the activity has led to improved livelihoods (e.g., improved earnings) and greater resilience to shocks associated with climate change or other issues? The activity sought to increase economic benefits for 25,473 household members (15,180 men and 10,293 women) annually by providing livelihood support during the hilsa ban period as well as AIGA support to diversify year-round income sources. An ECOFISH survey of 406 households showed an increase of 57 percent in income from the baseline (see Table 7). ECOFISH has demonstrated that it is feasible to add value to hilsa fisheries via fisheries conservation measures and improve the resilience of fisheries-dependent communities by diversifying their economic opportunities. Activity beneficiaries have USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 3 increased income from fisheries and other economic activities and are more resilient to climate changes that affect hilsa-dependent communities. Question 4. How effectively has ECOFISH advanced the co-management approach with the GOB, local communities, and other stakeholders? What evidence is there that this approach is successfully combating overfishing/improving sustainability of fisheries? ECOFISH has pioneered a diverse co-management infrastructure at the local level, including high-impact community institutions such as CSGs, which enhance community resilience, and Hilsa Fisheries Guards that work with the Coast Guard and the police to increase local fisheries enforcement capacity. ECOFISH is in the process of developing a comprehensive national co-management system, although progress at both district and national levels is lagging. Collectively, ECOFISH co-management interventions have contributed directly and indirectly to the observed increase in hilsa stocks by promoting awareness of ban periods, building community support for compliance with the ban and gear restrictions, and providing alternative livelihoods that incentivize fisher families to pursue alternative income sources during ban periods and possibly to reduce overall fishing efforts. Question 5. How effective have the activities’ gender engagement activities been under Intermediate Result (IR) 2 and IR 3? ECOFISH has had a significant impact in empowering men’s and women’s engagement in both hilsa fisheries conservation and in AIGAs; 4,152 women from 123 villages are now enrolled in savings and loan programs through 133 CSGs, and their total savings amount to BDT 41,50,978 (USD 52,000) as of September 2017. This capital is available for community reinvestment at considerably lower rates than traditional forms of credit from moneylenders and has stimulated new forms of community microenterprise. Question 6. Given the current state of overexploitation of coastal fisheries resources in Bangladesh, how can USAID best direct future resources, both strategically and programmatically, to implement the Government of Bangladesh National Strategy for Marine and Coastal Biodiversity ECOFISH has provided a strong foundation for future USAID strategy and programmatic investment in support of the GOB National Strategy for Marine and Coastal Biodiversity. There are both immediate and longer-term opportunities to strengthen USAID-GOB engagement in this strategy: the near-term opportunities relate to the potential to leverage ECOFISH strategies to shape the proposed $200–500M World Bank (WB) Fishery project. In the longer term, USAID should consider developing a new EAFM program that targets all of Bangladesh coastal waters. RECOMMENDATIONS The evaluation team offers 19 recommendations for consideration: Question 1 1. Theory of Change – ECOFISH should review the theory of change, develop results chains for specific strategies, and use those to guide the programming of Year 4 and Year 5 activities. Technical assistance may be sought from the USAID Forestry and Biodiversity Office’s Measuring Impact program. 2. Editorial Quality Improvement – The poor quality of ECOFISH project documents undermines the activity’s credibility and makes it difficult to obtain useful information from the reports. ECOFISH should employ a scientific editor (ideally with a specialization in fisheries management) to review and USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 4 edit all annual work plans, annual reports, select technical reports, and any external conference and journal publications arising from ECOFISH. 3. Indicators – ECOFISH monitoring and evaluation (M&E) staff should update reports on the standard indicators and review custom indicators as part of a review of the Theory of Change (TOC) to ensure that indicators present an accurate picture of progress toward the ECOFISH objectives. 4. Hilsa Science Strategy – ECOFISH should partner with Bangladesh Fisheries Research Institute to convene a national hilsa science workshop in Q3 2018 under IR 1 that will establish priorities for all disciplines of hilsa research; that agenda should be formally incorporated into the National Hilsa Management Plan. 5. Year 4 Work Plan – USAID should request a revision of the Year 4 work plan identifying immediate priority activities only for interim approval (until May when the Exit Strategy work plan will be adopted). No further village expansion activities should be undertaken in Year 4 or Year 5. 6. Improving Enforcement – ECOFISH should work with partners to test alternative approaches to enforcement, including a boat-registration (and seizure)-based approach, drones, and other surveillance methods. Question 2 7. Exit/Transition Strategy – Develop an activity exit strategy that incorporates scenarios for a transition of ECOFISH activities to GOB, community organizations, and other development projects. 8. Ecosystem Approach to Fisheries Management/Megafauna Conservation – Conservation activities under 4.1.2 of the Year 4 work plan should be scaled back. 9. Model Villages – A specific focus on the current model villages is needed. ECOFISH should realign current efforts to ensure that there are dedicated strategies and leadership for both villages. Question 3 10. Paravet expansion – To address beneficiary concerns about the high rates of livestock mortality in some villages, which affects incomes, the team recommends that ECOFISH strengthen the paravet program with more training, support, and linking with animal medicine companies and the Upazila Livestock Officer’s (ULO) office. Question 4 11. Co-management Strategy – Coastal Resources Center at the University of Rhode Island (CRC/URI) should be reengaged to assist with co-management elements of activity exit/transition strategies under IR 2. ECOFISH also needs to ramp up efforts on establishing district-level co￾management models. 12. Boost Policy Staffing – ECOFISH should employ an additional senior policy advisor, whose primary responsibility will be to support the National Hilsa Management Plan and Conservation Fund processes and to engage ECOFISH actively in the design of the World Bank Fishery project. 13. National Coordinating Office – ECOFISH should establish and maintain a rotating staff presence at the DOF headquarters to ensure closer alignment with DOF national programs. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 5 14. Institutionalization of Community Fish Guard Program – To address program sustainability, ECOFISH should work with the Department of Fisheries to strengthen the Community Fish Guard program as part of the overall co-management infrastructure and financing strategy. Question 5 15. Gender Data Harvest – During Year 4, ECOFISH should review the datasets related to gender and undertake a comprehensive analysis of lessons learned. 16. ECOFISH Publications – Activity publications should specifically address findings related to men’s and women’s roles in hilsa fisheries. Question 6 17. No-Cost Extension (NCE) – ECOFISH should design a Year 5 work plan for an additional six months and submit an NCE request to USAID concurrently with the exit strategy. 18. Strategic Initiatives – As part of its next phase of partnership with the GOB, USAID should consider (1) an EAFM activity that targets coastal and marine waters of Bangladesh (including fisheries and megafauna conservation), (2) supporting creation of a regional hilsa management agreement; (3) riverine and coastal community vulnerability assessments, and (4) strategic science partnerships with US institutions. 19. Support for Secretariat – As part of the activity transition and exit strategy, USAID and ECOFISH should consider the value of a small ($700k/year for three years) continuation of specific IR 4 activities under ECOFISH to leverage World Bank and related investments. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 6 1. INTRODUCTION The United States Agency for International Development (USAID)/Bangladesh contracted International Business & Technical Consultants, Inc. (IBTCI), through the Accelerating Capacity for Monitoring and Evaluation (ACME) activity contract (AID-388-C-14-00001), to conduct a midterm performance evaluation of the Enhanced Coast Fisheries in Bangladesh (ECOFISH) Activity to assess the extent to which it is on track to meet its overall performance goals and to inform management of any challenges or opportunities that warrant adjustments to ensure the achievement of those results. Bangladesh is one of the most densely populated countries in the world, with a population of 162 million that is expected to increase to 220 million by 2050, threatening to outpace food production.2 More than 70 percent of the population lives in rural areas and 87 percent of rural households rely on agriculture for at least part of their income.3 Agriculture is a key driver of economic growth in Bangladesh; it contributes almost 17 percent of the country’s gross domestic product (GDP).4 Bangladesh is also a major fish-producing nation. According to Mohammed et al. (2016), fish production contributed 4.4 percent of the country’s national GDP, 2.5 percent of foreign exchange earnings, 11 percent of employment, and 60 percent of all consumed animal protein. Bangladesh has made substantial progress in reducing poverty from 44.2 percent in 1991 to 18.5 percent in 2010 and it is projected to decrease to 12.9 percent in 2016,5 supported by steady economic growth of 6 percent over the past decade.6 Bangladesh seeks to reach Middle Income Country status by 2021.7 Despite declining poverty, the United Nations Development Program (UNDP) finds that Bangladesh faces continued challenges to sustain its economic growth trajectory, build on the achievements of the past decade, and meet its targets under the United Nations Sustainable Development Goals. These challenges include environmental issues and climate change, political and economic issues, and an increasingly competitive international trade environment, all of which impede higher growth rates.8 Food security, economic development, and community resilience in coastal and riverine regions of Bangladesh are increasingly threatened by climate change. The ECOFISH Concept note (USAID/BD, 2014) made the following observation: Bangladesh is ranked first among countries most vulnerable to tropical cyclones and sixth most vulnerable to flooding. Given projections of climate change, the relative importance of fisheries to the national economy and diets, and limited capacity to adapt, fisheries in Bangladesh rank amongst the most vulnerable globally. (Alison et al., 2009) Immediate risks to seasonal climate variation include an increased frequency and severity of tropical storms, floods, and landslides. 2 Bangladesh FY 2011–2015 Multi-Year Strategy, Feed the Future – A US Government Initiative (February 22, 2011), page 5. 3 World Bank Bangladesh Country at a glance. http://www.worldbank.org/en/country/bangladesh 4 Government of Bangladesh, Agriculture Sector Development Strategy: background paper for preparation of 7th Five Year Plan http://www.plancomm.gov.bd/wp-content/uploads/2015/02/17_-Agriculture-Sector-Development-Strategy.pdf, page 1. 5 The figures are based on international poverty line of $1.90 per person per day. http://www.worldbank.org/en/country/bangladesh/overview 6 Bangladesh Development Update: Breaking Barriers Shows Resilient Economic Growth Despite Headwinds, World Bank (May 14, 2017). http://www.worldbank.org/en/news/feature/2017/05/14/bangladesh-development-update-breaking-barriers 7 Government of Bangladesh, Agriculture Sector Development Strategy, page 1. 8 UNDP – About Bangladesh. http://www.bd.undp.org/content/bangladesh/en/home/countryinfo/ USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 7 These natural hazards cause loss of life, damage to infrastructure, degradation of fisheries and habitats and economic assets, and consequently negatively impact livelihood security for those most reliant upon fisheries, especially the poor. Increased sea-level rise, salinity of croplands and groundwater, and increased incidence of floods and drought will cause additional stress to Bangladesh’s population…. Given the significant gender disparity in both income and human poverty, especially at the lower end of income distribution, women are not only likely to be disproportionally represented in those presently most vulnerable to climate change, but also possess lower levels of capacity to adapt in the future (emphasis added). 2. DEVELOPMENT PROBLEM AND USAID’S RESPONSE 2.1 THE DEVELOPMENT PROBLEM The hilsa9 (Tenualosa ilisha) fishery is the biggest single-species fishery in Bangladesh, with landings accounting for approximately 10 percent of annual fish production by volume; these contribute an estimated one percent to the country’s annual GDP (DOF 2014; Wahab et al., 2013). Hilsa is a keystone species10 important to both the natural and human ecosystems of the Meghna River and more broadly to watersheds through the Bay of Bengal region. Figure 1. Trends in Total Hilsa Catch in Bangladesh 1991–201711 9 Hilsa are also commonly known as ilish (referencing the scientific name given by F. Hamilton in 1822). 10 A keystone species is an organism that helps define an entire ecosystem. Without its keystone species, the ecosystem would be dramatically different or cease to exist altogether (Wagner, 2010). 11 Source – A. Wahab (WorldFish) derived from published DOF catch data, 2017. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 8 Hilsa are anadromous – they spend much of their lives in coastal waters but they migrate upstream to spawn in coastal rivers. In Bangladesh alone, an estimated half a million people directly depend on the fishery as a principal source of protein and a further 2.5 million are indirectly involved in supply-chain activities such as processing, transportation, and marketing (Rahman et al. 2012). Hilsa range from India to Myanmar (and possibly Thailand) and were reportedly abundant throughout the Ganges watershed and most rivers in Bangladesh until the 1970s. Production gradually declined in many river systems over the subsequent 30 years. This decline has been attributed to a combination of factors: the closure of migratory routes, river siltation, overfishing, indiscriminate harvesting of brood stocks and juveniles, use of fishing nets with very small mesh sizes, mechanization of fishing, increasing numbers of fishers, industrial pollution, and climate variability.12 Figure 2. Regional Variability in Marine and Riverine Catches of Hilsa13 In recent years, total hilsa national catches have increased, as shown in Figure 1. Since 2002–2003, hilsa catch has doubled in marine and riverine (inland systems), largely because of the pronounced increase in 2016–2017. Most hilsa are caught in marine waters as indicated in Figures 1 and 2. Figure 2 depicts the significant geographical and temporal variability in hilsa catch throughout Bangladesh over the past 30 years. 2.2 USAID RESPONSE To address opportunities to improve the health of coastal and riverine fisheries and ecosystems and improve the resilience of communities dependent on hilsa fisheries, in 2014 USAID proposed a partnership with the GOB and WorldFish to implement the $15 million Enhanced Coastal Fisheries (ECOFISH) Activity. ECOFISH-Bangladesh contributes to the overall USAID Country Development Cooperation Strategy (CDCS) Development Objective 4, 12 Hilsa have been extirpated in some rivers in eastern Bangladesh (M. Shahadat Hossain, personal communication). 13 Source – M. Shahadat Hossain derived from published catch data DOF, 2016. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 9 “Responsiveness to Climate Change Improved,” which aims to increase the nation’s responsiveness and adaptation to, and mitigation of climate change through three interrelated intermediate results (see Annex 1). ECOFISH aligns directly with two of these Intermediate Results (IRs): (1) improved management of natural resources focused on biodiversity conservation leading to indirect adaptation benefits, and (2) enhanced adaptation capacity and resilience to shocks. Figure 3. ECOFISH Activity Geographic Focus14 The ECOFISH activity was developed within the context of USAID’s broader Global Climate Change (GCC) and Biodiversity Conservation Initiatives. ECOFISH incorporates identifying and maintaining biodiversity and other priority ecosystem services that support fisheries and men and women in poor fishing communities. The ECOFISH activity also seeks to assist Bangladesh in reducing its vulnerability to climate change impacts by decreasing the environmental, economic, and social consequences of climate change. The activity also focuses on concomitant improvements in policies and strategies as well as institutional strengthening of two GOB agencies, the DOF and Bangladesh National 14 Also described as “Theater of Operations” (Figure from ECOFISH Presentation to Evaluation Team Nov. 14, 2017, slide 11). USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 10 Fisheries Research Institute (BFRI), to deliver more effective science inputs, fisheries management, and desired policy outcomes.15 WorldFish, a member of the Consultative Group for International Agricultural Research (CGIAR), is the lead implementing partner (IP) for ECOFISH and works in collaboration with the DOF, which is part of the Ministry of Fisheries and Livestock (MOFL). Other IPs include government institutions, private sector organizations, universities, civil society, nongovernmental organizations (NGOs), and specialist international advisory organizations with a common focus on biodiversity, climate change, livelihood resilience, and management of hilsa shad, other food fish, and aquatic megafauna in the Padma-Meghna River ecosystem. The ECOFISH focal area of operations is outlined in Figure 3. 2.3 ECOFISH GOALS AND OUTCOMES As indicated in Figure 4, the principal goals of ECOFISH-Bangladesh are to (a) enhance ecosystem resilience through the establishment of effective co-management in hilsa shad (Tenualosa ilisha) fish sanctuaries; and (b) strengthen community resilience by improving the savings, livelihoods, and coping strategies of marginalized and extremely poor fisher folk, particularly women, thereby reducing their risk from climate-affected shocks and stresses. 15 ECOFISH Monitoring, Evaluation, and Learning Plan, 2015–2019 USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 11 Figure 4. ECOFISH-Bangladesh Results Framework (v.4) USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 12 3. EVALUATION PURPOSE ECOFISH-Bangladesh commenced on June 1, 2014, and will end on May 30, 2019. The primary goals of this midterm performance evaluation are as follows: ● To provide evidence so USAID/Bangladesh can determine whether the ECOFISH activity has achieved its desired results to date ● To inform what course corrections, if any, are needed in the implementation time remaining ● To determine how effectively this activity has incorporated gender in the interventions ● To recommend to USAID key strategic and programmatic options for future engagement In the evaluation Scope of Work (SOW) detailed in Annex I, USAID/Bangladesh directed the evaluation team to focus on the following questions during the assignment: 1. To what extent is the ECOFISH on track, in terms of progress and outcomes, to meet its overall goals? What factors (internal/external) affected timely achievement of targets? And what factors impeded performance? 2. Given the time remaining, what opportunities exist to enhance and strengthen programmatic effectiveness such as by adding, changing, rescaling and/or removing activities to meet or surpass targets/objectives, within the stipulations of the ECOFISH agreement? 3. Is there evidence that the activity has led to improved livelihoods (e.g., improved earnings) and greater resilience to shocks associated with climate change or other issues? 4. How effectively has ECOFISH advanced the co-management approach with the GOB, local communities, and other stakeholders? What evidence is there that this approach is successfully combating overfishing/improving sustainability of fisheries? 5. How effective have the gender engagement activities been under IR 2 and IR 3? 6. Given the current state of overexploitation of coastal fisheries resources in Bangladesh, how can USAID best direct future resources, both strategically and programmatically, to implement the Government of Bangladesh National Strategy for Marine and Coastal Biodiversity? Picture 1. Young boy with fish nets. These guiding questions and clarifying comments provided by USAID at the evaluation in brief on November 20, 2017, shaped the design of the evaluation process (Annexes I and II) and, together with the results framework for ECOFISH (Figure 4 above), frame the structure for this report. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 13 4. EVALUATION METHODS AND LIMITATIONS 4.1 EVALUATION APPROACH The team designed the evaluation approach in accordance with USAID policy guidance.16 After an initial consultation conference call with USAID/Bangladesh staff during the pre-mobilization period in October 2017 and following inaugural meetings with ECOFISH implementers and partners in Dhaka in early November, the evaluation team developed a detailed evaluation methodology and work plan for review and approval by USAID.17 Three main factors influenced the evaluation approach: 1. Responsiveness to USAID Requirements – The overall approach of this evaluation, like most midterm performance evaluations, emphasized learning and adaptation. The approach employed balanced rigorous, evidence-based independent verification of past performance with a more open-ended inquiry￾based learning18 process, seeking to identify pathways and evaluate alternative strategies/scenarios for improvement as the activity enters its final two years of implementation. 2. Available Resources – Given the duration of fieldwork, evaluation schedule, and data constraints, the evaluation team focused field data collection efforts in the five (of nine total) ECOFISH districts with the largest number of activity beneficiaries. To provide a range of insights into ECOFISH performance, the evaluation team employed an adaptive mixed-methods approach utilizing both quantitative (including nonexperimental design stakeholder mini-surveys) and qualitative analysis (including KIIs, stakeholder interviews, email mini-surveys, FGDs, and ad hoc field observations). The mixed-methods approach was well-suited to the activity being evaluated and enabled the team to work efficiently in the field. For example, this approach provided the flexibility to collect data from several locations concurrently. 3. Appropriate and Ethical Inquiry – Because of the challenges associated with effective cross-cultural communication, especially in rural areas of Bangladesh, and the importance of maintaining confidence in data handling, the evaluation team made every effort to operate in accordance with Ethical Guidelines of International Business and Technical Consultants, Inc. (IBTCI), and the Guiding Principles of the American Evaluators Association.19 4.2 DATA COLLECTION METHODS Annex II includes a copy of the Evaluation Design Matrix, a cross-correlation of key questions (and sub￾questions), data sources/methods, sampling criteria, and available data sources/indicators to be reviewed. As shown in the matrix, the evaluation process employed a mixed-methods approach to data collection, including: ● Document Review: Evaluation team members were provided with an extensive set of activity documents at the time of mobilization. These were supplemented throughout the evaluation process with more detailed program reports, as well as other Bangladesh and global studies 16 See USAID ADS Chapter 201 Program Cycle Operational Policy, (Revision date 10-05-17) https://www.usaid.gov/sites/default/files/documents/1870/201.pdf 17 Draft work plan was presented to USAID/Bangladesh on November 20. The evaluation approach is summarized in Annex II. 18 For a broad overview of inquiry-based learning methods, see https://en.wikipedia.org/wiki/Inquiry-based_learning 19 See American Evaluators Association Guidelines: http://www.eval.org/p/cm/ld/fid=51 USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 14 related to hilsa fisheries and coastal community climate resilience. A full list of activity documents reviewed and additional reference sources cited is provided in Annex IV. ● Key Informant Interviews (KII): The team conducted KIIs with a range of ECOFISH stakeholders including staff of relevant GOB ministries, partner agencies (including ECOFISH IP staff), journalists, community leaders, and staff of other projects related to Bangladesh fisheries, community resilience, and rural development. KIIs were undertaken using a semi-structured interview drawn from the Question Guide (Annex III). ● Focus Group Discussions (FGD): The team conducted FGDs (semi-structured discussions with small groups of six to 10 people) during fieldwork meetings with specific ECOFISH beneficiary groups, including the Community Savings Group (CSG), Hilsa Conservation Group (HCG), and Hilsa Ghat Groups (HGG).20 These FGDs employed facilitated discussion to gather semi￾structured qualitative data. Preselected participants (balanced to the extent feasible by gender, occupation, age, and other relevant factors) discussed issues and concerns based on a list of key themes introduced by the moderator (Annex III contains the core question set for FGDs). All participants were also encouraged to express viewpoints about any aspect of ECOFISH of importance to them. A note taker recorded the discussions. ● Mini-Surveys: In addition to the FGDs, the team conducted two mini surveys: a) Trend Perception Surveys – During visits to the five activity districts, a short trend perception survey (past, present, and future) was completed by a total of 546 randomly selected participants in two preselected beneficiary villages and one non-activity village per district. Annex III describes the process used for identifying participants and the protocols used by primarily independent enumerators to collect data in each village. b) Scientist/Researcher Surveys – The team sent out an email survey (see Annex III) to 26 scientists from universities and research centers throughout Bangladesh to seek broader understanding of the state of knowledge of hilsa fisheries, threats, and priority research needs. Ten responses were received and analyzed. ● Information Exchange Meetings (IEM): These were convened with the WorldFish ECOFISH team and other IPs (International Union for Conservation of Nature and Natural Resources [IUCN], Center for Natural Resource Studies [CNRS], Coast Trust, and CODEC [Community Development Centre]) to obtain additional background on ECOFISH activities. These IEMs also provided opportunities to question activity implementers about ECOFISH strategy, interim results and lessons learned. KIIs were undertaken subsequently with individual ECOFISH IP staff. ● Informal Meetings and Observations: The team conducted short meetings (15 to 60 minutes) in each village visited to observe and verify ECOFISH activities and to ask any questions. These informal interviews provided valuable insights into the different issues affecting communities because they could be spontaneously tailored to issues that were locally or seasonally important such as riverbank erosion, livestock mortality rates, or the effects of fishing gear restrictions on livelihoods. The team planned fieldwork routes (see Annex III) to maximize opportunities to observe and meet with ECOFISH beneficiaries such as small-scale fishermen (interviewed informally during riverboat travel) and other hilsa fishery and community development actors, including fish auctioneers and marketers encountered en route. Table 1 summarizes the numbers of participants in evaluation survey activities and meetings. 20 Hilsa Ghat is the hilsa landing center where trading of hilsa occurs; it is the common meeting place for most of the stakeholders. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 15 Table 1. Participation in Evaluation Meetings and Surveys by Gender Type of Data Gathering Activity # Male Participants # Female Participants # Total Participants KII/IEM 88 5 93 FGD 38 70 108 Perceptions Mini-Survey 437 109 546 Scientists Mini-Survey 10 0 10 Total 573 184 757 4.3 LIMITATIONS Overall, the evaluation field data collection process worked well despite the significant logistical challenges encountered. Key challenges and the team’s responses included the following: ● Security concerns prevented work by the Team Leader in villages after daylight (around 5 p.m.). To counter this concern the team split into sub-groups in the late afternoon of each field day to enable local team members to continue interviews with KIIs into the early evening. ● Field telecommunication challenges made it difficult to assemble teams of independent enumerators 24 hours in advance of a mini-survey in the beneficiary and non-beneficiary villages. Because of these challenges, the evaluation team had to seek assistance from local ECOFISH activity implementing staff to conduct three of the 18 mini-surveys on the first day of fieldwork. Picture 2. Focus Group Discussion in the Field Transportation logistics – As indicated in Annex III, the team covered more than 1,100 km over an 11-day period using boats, vehicles (cars and motorbikes), and by walking (up to 8 km) into remote villages. This required considerable planning and flexibility, and there were inevitable delays in connecting ferry schedules due to rough/impassable roads. The team was fortunate to be able to call upon local IP transport resources to assist in making up time or to reschedule KII meetings to evening sessions at hotels. Further details of the limitations of specific survey methods are discussed in Annex III. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 16 5. EVALUATION FINDINGS 5.1 QUESTION 1: ECOFISH PERFORMANCE To what extent is the ECOFISH on track, in terms of progress and outcomes, to meet its overall goals? What factors (internal/external) affected timely achievement of targets? And what factors impeded performance? 5.1.1 FINDINGS This section addresses the evaluation performance questions, first on an overall activity scale and then by Intermediate Result (which is the main framework for ECOFISH interventions). Overall Progress GOB fisheries catch data indicate that the increase in hilsa fish stocks accelerated in 2016 and 2017, particularly for inland catch (see Figure 1). The surveys undertaken by ECOFISH21 and the evaluation team in both beneficiary and non-beneficiary villages through the ECOFISH area of operations also indicate that fishers are catching more fish and have greater confidence in the future of hilsa fisheries than they had prior to the activity. These recent increases (28 percent from 2015 to 2017) exceed historical changes in hilsa catch rates (5 percent average annual change from 1990/91 to 2016/17). Available catch data for the ECOFISH area of activity is not presently sufficiently granular/detailed to analyze cause-effect trends at the local level; however, the only difference in hilsa management in BD since 2014 has been the commencement of ECOFISH activities. ECOFISH operates at a sufficient scale to impact the entire BD fishery, particularly in affecting juvenile survival and protection of key breeding grounds. On the basis of this evidence, the evaluation team believes it is plausible to attribute the increase, at least in part,22 to specific ECOFISH activity interventions since 2014, most notably the establishment of increasingly well-enforced seasonal restrictions on hilsa and jatka (juvenile hilsa) catches, expanded community engagement in fisheries management via a range of co-management processes, and the provision of alternative income generating activities which, together with the GOB food (rice) subsidies and increased enforcement activities, have diminished incentives for catching hilsa during ban periods. There is also strong survey-based evidence of an increase in the incomes of beneficiary fisher communities across the ECOFISH Theater of Operations, primarily as a consequence of community￾level ECOFISH interventions such as CSGs, occupational and skills training, and AIGAs such as goat rearing (see 5.4 below). Together with strongly increased awareness of the importance of collective efforts to manage Hilsa stocks, and a more coordinated approach between GOB agencies and with communities in Hilsa fisheries management, community resilience is being improved, although these communities remain vulnerable to climate-affected shocks and stresses over the longer term. Theory of Change (TOC) During the briefing with USAID, the evaluation team was asked to assess how the Results Framework/ TOC (Figure 4) has been applied and interpreted. The current version of the framework (v. 4) comprises four “pillars” of IRs, and each pillar contains four interventions, leading to a composite of 16 activity outcomes (although the specific relationship between each IR and those outcomes is not clear). These are the focus of annual progress reports of ECOFISH, but as noted elsewhere in this evaluation, 21 See Islam et al. (2017) survey data from 248 HH that indicates a recent increase in CPUE; Figure 5, p. 12. 22 Other contributing factors are the subsidies provided by the GOB, increased community awareness of hilsa conservation needs, and increased enforcement of ban regulations by GOB and community leaders during ban periods. The relative contributions of specific strategies were not able to be assessed in detail because available data are limited; data on the impacts of specific interventions will be gathered and analyzed during the remainder of the ECOFISH activity. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 17 these sections are often incomplete and inconsistent. It is challenging to compare ECOFISH activities (such as those outlined in Section 3 of the Year 3 Annual Report) with the Results Framework/TOC (Figure 4) and the Activity Monitoring & Evaluation Plan (AMEP) (revised April 2017) for the following reasons: • The structuring of some activity strategies: For example, communications activities, which have been a very successful major investment by ECOFISH and critical to raising awareness of hilsa management needs, are neither described under the TOC nor reported on as specific strategies that have been catalysts for other strategies. This underemphasizes the key role that communications play in ECOFISH’s impact at the community level. A similar concern exists relative to the use of study tours as a strategy for building partner awareness of and buy-in to new concepts such as EAFM. These activities are reported under “Management and Administrative Issues” of annual reports. • The sequencing of activities: For example, under IR 1.1, fish tissues for genetic analysis were sampled in Year 1 of the activity but as of the time of this evaluation, samples were still being processed in Malaysia/USA and were not expected to be available before April 2018. This delay significantly undermines the value of this work to the activity. • The way some activities contribute to the overall activity outcomes and what level of effort is appropriate for specific activities: For example, under IR 1.2 “consultations were held with 575 community members about megafauna conservation.” The relationship between this activity and IR 1.2 (Strengthened community-based monitoring of target fisheries and technologies) was not immediately apparent. • The actual status of specific activities undertaken: For example, under IR 4.1 (Improved economic valuations, policy briefs, and proposals for GOB) the Year 3 Annual Report mentions that “an area of 976 sq km has been delineated as MPA boundary (sic)” but it is not clear what that actually means in terms of marine protected area (MPA) establishment. It is also notable that a systematic analysis of the relationships between ECOFISH targets, threats, contributing factors, opportunities, and strategies that was proposed in the initial ECOFISH Activity Concept document (USAID, 2014) was not undertaken. The concept document detailed initial chain￾based frameworks for each IR consistent with current USAID guidance,23 but was not developed further24 despite a comprehensive stakeholder workshop in November 2014 that refined the TOC (and added the fourth IR). Communications One of the most striking aspects of ECOFISH and a key contributing factor to the success of the activity to date is the strong emphasis on communications. Although not formally a part of the Results Framework, communications is a key cross-cutting strategy of ECOFISH, which has its own brand identity guidelines.25 The ECOFISH brand is now well-established through all districts of operation and has been highly successful in promoting public awareness of ECOFISH core strategies. All KII and FGD participants highlighted the importance of those communications methods in engaging them in the activity. 23 Stem, C. and M. Flores (2016). Using Results Chains to Depict Theories of Change in USAID Biodiversity Programming, Biodiversity How-to Guide 2, USAID, Washington, DC. 24 In the Year 2 Annual Report (p. 65) a “Hypothesized Causal Mechanism” model was outlined that included many elements of a results chain, but this was not further developed. 25 ECOFISH Communications Strategy (2015). USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 18 Communications materials in Bangla are generally well-written and appropriate for the target audience. However, the same is not true for ECOFISH’s English language publications. Across all years of the activity and in all forms of key reports (work plans, annual reports, etc.), there is a consistent trend of poor quality of editorial material with incorrect grammar and data that is either incorrect or incomplete (usually associated with a lack of metadata). On numerous occasions, the evaluation team could not understand the statements made in key reports or interpret data presented without further reference to ECOFISH staff. Annex VII contains examples from the most recent Annual Report (Year 3). As shown in Table 2, performance on IR I has been mixed, with no progress on two of the four indicator targets. This is a combination of misrepresentation of the way IR 1 contributes to the overall activity objectives, the fact that IR I has not yet been implemented for a sufficient period of time to make a substantive contribution in some areas, and underperformance in other areas. The four interventions proposed under IR 1 included a wide range of activities, many of which have been well-implemented by the IR 1 team, including the following: ● Monthly length-frequency (L-F) distribution of hilsa ● Economically important fish catch estimation ● Census of the fishing boats and nets engaged in the Andharmanik River ● Delineation and impact assessment of the sanctuaries26 ● Monthly water quality monitoring along the river courses ● Identification of peak spawning period of hilsa ● Gillnet selectivity study involving fishers ● Hilsa rearing for enhanced fisheries ● Biodiversity and biodiversity threats assessment ● Household socioeconomic surveys ● Community-involved fish landing monitoring [in progress] ● Trial of catch monitoring through boat skippers as citizen scientists [in progress] ● Balancing community fishing needs with the protection of megafauna [in progress] Performance on IR 1: Improved fisheries science for decision-making Table 2. Progress on IR 1 Targets Indicator + Status Unit Base Year LOP Target Target 2016 Actual 2016 2017 Target 2017 Actual 2018 Target 2019 Target #1: Improved decision-making science documents # 2015 7 1 1 2 2 2 2 #2: Increase in hilsa abundance (catch per unit effort [CPUE]) in sanctuary areas supported by USG assistance27 % 2015 10 10 #3: Science clauses in Hilsa Management Plan # 2015 10 10 #4: Fish landings centers with improved monitoring # 2015 40 8 8 12 9 20 26 In the Year 2 Annual Report, BFRI studies affirmed the positive effect of sanctuaries on hilsa production (p. 28). 27 Performance here is rated as poor because of the lack of progress reporting on this indicator, although it should be noted that the survey of 248 households by Islam et al. (2017) indicates that Catch per Unit Effort (CPUE) rates have increased. KEY FOR TABLES 2,3, 5 AND 6 NO REPORTED PROGRESS TO DATE NOT PERFORMING AS PLANNED ON TRACK USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 19 Despite these positive efforts, there have also been some significant delays in the processing of key data obtained under IR 1, the most notable being the extraordinary delay in completing genetic analysis of hilsa otolith and fin tissues samples and integration of water quality data in habitat modeling. There are also key underreported and unresolved questions about the scientific basis for decision-making, including questions posed by activity IPs, GOB partners, and researchers28 such as: ● How does sanctuary design affect hilsa breeding? Do key spawning areas (microhabitats) merit higher levels of protection? Are sanctuaries more effective than blanket bans on fishing?29 ● Do hilsa return to the same locations to spawn (site fidelity)? What happens to juveniles during and after ban periods? ● How important are coastal waters for juvenile rearing? What other life cycle processes are important to the long-term sustainability of hilsa? ● What is the impact of coastal/marine fishing on adult return rates? ● Why do hilsa not return to some rivers in their previous range? Is reintroduction feasible? ● What are the major predators of hilsa at different life stages? ● How do human activities in sanctuaries and upstream from them affect juvenile survival? ● What is the optimal design for community engagement in sanctuary management/enforcement? ● Are there new methods/technologies for improved/lower cost enforcement (e.g., drones)? ● Are there alternative/innovative methods for targeting repeat offender fish poachers?30 ● What are the key water quality thresholds for juvenile (and adult) survival and how does climate change (temperature, river flows, salinity gradients, etc.) affect these? ● What are the barriers to hilsa aquaculture? Can hilsa be produced under different culture regimes? What are the key next steps in hilsa culture research building on the BFRI trials? These issues are not uncommon in any fishery management process.31 The IR 1 team seeks to address many complex scientific questions with relatively limited resources,32 with a limited time frame for research, and within activity constraints of producing science that informs immediate decision-making needs. Given those limits, it is surprising that a major hilsa science synthesis and peer-reviewed research agenda for ECOFISH was not developed at the outset of the activity. Such an agenda is clearly a key contribution to the Revised National Hilsa Management Plan. Researchers at the BFRI interviewed in KIIs emphasized the importance of having a clear national set of hilsa fisheries research priorities so that they can pursue priority national research goals. Performance on IR 2 Strengthened Adaptive Fisheries Co-management Table 3. Progress on IR 2 Targets Indicator + Status Unit Base Year LOP Target 2016 Target 2016 Actual 2017 Target 2017 Actual 2018 Target 2019 Target # 5: Institutions with improved climate capacity # 2015 600 108 n.a. 279 160 n.a. 161 # 6: Hectares under improved management Ha 2015 250,000 2,800 2,800 192,700 186,050 58,500 # 7: Hours of Natural Resources Management (NRM) Training Hrs 2015 230,000 56,252 54,732 83,944 78,519 91,892 # 8: Number of people trained in NRM # 2015 20,000 5,300 5,000 5,200 5,690 5,500 # 9: Percent increase in stakeholder engagement in Sanctuaries % 2015 20 n.a. n.a. n.a. n.a. 20 # 10: Compliance with hilsa rules due to government compensation # 2015 1,400 n.a. n.a. 8,00033 n.a. 6,000 28 See results of scientist mini-survey in Annex V. 29 This question is also posed by ECOFISH in Annual Report Year 2 (p. 66) and in a recent manuscript by van Brakel et al. (2018). 30 For example, instead of a generic fisher registration card (which is reportedly subject to abuse), some DOF staff suggested that a vessel registration program should be introduced. 31 See, for example, the challenges of managing well-known and poorly known fish stocks (Lauck et al., 1998; and Witek, 2017). 32 During a KII with the BFRI, the evaluation team learned that only $149,000 (BDT 12,382,231) had been made available from ECOFISH to study key and complex questions such as sanctuary effectiveness and hilsa aquaculture potential. 33 Note contradiction between MEL plan (this target) and the Year 3 Annual Report, which states “there was no plan to achieve this indicator” (p. 19). USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 20 Improved co-management of fisheries is one of the more ambitious components of the ECOFISH TOC, as reflected in the variable progress on this IR reflected in Table 3. As noted in the activity concept note, there was a limited history of fisheries co-management in Bangladesh prior to the commencement of ECOFISH, despite the well-recognized importance of community engagement (Hossain et al., 2013), and the existence of long-established co-management models in Bangladesh’s forests and protected area management sectors. Unlike many South and Southeast Asian nations where there is a long history of experience in ecosystem-based fisheries management and related fields,34 Bangladesh has only recently begun to pursue these types of biodiversity, coastal resources management, and coastal resilience approaches. To address this gap, USAID commissioned a separate and specific assessment of marine tenure and small-scale fisheries by Pomeroy, Thompson, and Courtney (2016) as a key input to co-management strategy. The study observed that “While community-based fisheries management (CBFM) for inland waters is well established in Bangladesh, CBFM for coastal and marine fisheries is almost non-existent with only a few examples…. Policy statements supporting CBFM… have not been developed into formal instruments or implemented” (page 6). Also, the study identified the challenges facing ECOFISH and made a series of recommendations, including more detailed studies of the hilsa life cycle, transboundary studies, improved monitoring and control systems, and a greater focus on ecosystem-based approaches (e.g., integrating fisheries and agricultural management). Despite the evident challenges of developing workable and formally recognized co-management models, ECOFISH has developed and piloted a wide range of community-level co-management “institutional models” throughout the activity area, as indicated in Table 4. The evaluation team observed that much of the credit for the identification and rapid scaling of community engagement models can be attributed to the following four core activity strategies: 1. Engagement of the Coastal Resources Center of the University of Rhode Island (CRC/URI): In 2015 URI led both a stock assessment training and a co-management training that injected new ideas and provided examples of co-management from other regions and USAID activities garnered over a 20-year partnership between USAID and CRC/URI. URI was subsequently engaged in another workshop in 2016 and assisted with study tours. Despite that early promising start, ECOFISH’s main IP, WorldFish, decided not to engage CRC/URI further for budgetary reasons, which seems to be an odd decision given the lack of capacity within the ECOFISH team in this area and the pressing need for more focused and sustainable co￾management models and strategies as the activity moves forward. 2. Study tours undertaken to other global co-management sites: ECOFISH and close partners have undertaken study tours to the Philippines (2017), Indonesia (2017), Malaysia (2017), and Senegal (2017). Two staff from the BFRI have also been sponsored to undertake hilsa-related postgraduate studies at Malaysian Universities. These international collaborations have apparently been effective in bringing new ideas back to ECOFISH and creating an understanding of new EAFM concepts by senior decision-makers. Although the evaluation team was unable to determine whether additional study tours would be helpful in advancing activity goals at this stage, given the challenges of accessing international experience relevant to ECOFISH from Bangladesh, the team considers that it is important for the activity to reinvigorate links with 34 See, for example, Hotta and Dutton (1995); Olsen et al. (2003); Parks et al. (2003); and Staples et al. (2014). USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 21 existing global IPs such as CRC/URI and the International Institute for Environment and Development (IIED). 3. Partnerships with national IPs, particularly IUCN, CNRS, COAST Trust, and CODEC: All these have long-established relationships in these communities and were able to quickly deploy staff knowledgeable about the communities and leverage existing relationships and infrastructure. For example, in Char Fassion village of Bhola district, CODEC has effectively leveraged its existing women-run radio station programming to engage communities and share ECOFISH news within its 20-km broadcast radius. A member of the CSG group the team met with said, “Hearing news from other women inspires us to trust that information more than other news we get on the radio – and now that we are involved in ECOFISH we know that information is even more important to us.” 4. Effective engagement of community champions via processes such as the CFGs: It was evident during fieldwork KIIs and FGDs that through the village profiling process ECOFISH IPs have thoughtfully engaged local community champions in key activities such as patrolling and surveillance. Interviews with CFGs consistently revealed a strong desire to “self-police” rather than have enforcement activities imposed on them by outsiders. That motivation has been key to catalyzing more consistent enforcement of fishing restrictions during ban periods and has begun to build closer cooperation between villagers and other law enforcement institutions. Table 4. Co-management Infrastructure Established Under ECOFISH 2015-17 Activity Target Cumulative Results Year 1 Year 2 Year 3 Fishing village selection 40 40 42 123 villages in 9 coastal districts Household census and community profiling 3,500 6,500 6,000 15,149 Households (HHs) completed Hilsa Conservation Group (HCG) formation 120 140 160 448 HCGs formed Hilsa Ghat Group (HGG) formation - 28 40 52 HGGs formed Community Fish Guard (CFG) formation - 1 Upazila 3 Upazilas 189 CFG volunteers recruited in 5 Upazilas Community Savings Group (CSG) formation 40 41 42 133 CSGs formed Piloting of co-management of Ghat-based and Union Parishad (UP)-based approaches 12 12 24 Local Co-management Committees (CMCs) formed and piloted Piloting of Upazila-level co-management - 1 Upazila 3 Upazilas 4 CMCs in 3 Upazilas35 35 The most recent Upazila CMC was established on November 20, 2017 (during evaluation process). USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 22 Progress on IR 3: Enhanced Resilience of Hilsa Fisher Communities Table 5. Progress on IR 3 Targets Indicator Unit Base Year LOP Target 2016 Target 2016 Actual 2017 Target 2017 Actual 2018 Target 2019 Target 11: # Women benefiting from access to savings, etc. # 2015 3,600 1,200 1,200 1,200 1,852 1,200 12: # People with increased economic benefits # 2015 100,000 19,260 18,171 26,00036 25,473 27,500 27,500 13: # Training on alternative livelihoods # 2015 20,000 5,300 4,463 5,200 4,228 5,500 14: # Households with improved well￾being # 2015 20,000 4,580 4,297 5,300 5,194 5,200 5,500 ECOFISH activities under IR 3 have been some of the highest profile and impactful of the entire activity and contribute directly to the overall activity objective via three Sub-IRs: IR 3.1: Access to finance targeting women increased As of September 2017, 4,152 women from 123 villages are now enrolled in savings and loan programs through 133 CSGs, and their total savings amount to BDT 4,150,978 (approximately USD 52,000).37 Some 3,291 women have received “soft loans” (of BDT 5,000 each) from their CSGs and have reinvested the loan funds into a range of business enterprises such as net making, tailoring, goat rearing, etc. Loans are repaid over a one-month period at a 5–10 percent service charge, which is considerably lower than traditional forms of credit from moneylenders. One woman who never had the opportunity to contribute financially to her family’s well-being before ECOFISH said, “I am so proud of what I can now do with this funding – it gives me a way I never had before ECOFISH to contribute to my family… through growing my goat herd we now have funds to help us while my husband can’t fish in the ban period.” IR 3.2: Livelihood diversification for poor vulnerable households Picture 3. Goats from Alternative Income-Generating Activity Alternative Income Generating Activities (AIGAs) have proved to be a key “engagement lever” for ECOFISH beneficiaries and provided critical supplementary income support for families during hilsa/jatka ban periods. ECOFISH and its partners have delivered a total of 144 training events on AIGAs to 4,328 people – 2,530 women (58 percent) and 1,798 men (42 percent).38 A total of 117 training events were held on goat rearing practices and homestead vegetable cultivation, and the remaining were on poultry/duck rearing, pebble doll (toy) production, and other skills, including sanitation and maintenance. ECOFISH distributed vegetable seeds to 16,101 households and also initiated a range of activities on aquaculture and other new technologies such as briquette making and fish drying. While participants in these activities appreciated the AIGA opportunities afforded by ECOFISH, as noted throughout the FGDs, the participants expressed concern about the following issues: 36 This is a confirmed typographic error in the report – the target shown in the MEL plan was 2,600, but the actual Year 3 work plan target was 26,000 and actual 25,473. 37 Data from ECOFISH Year 3 Annual Report, p. 23. 38 ECOFISH 2017 Year 3 Annual Report, p. 47. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 23 • The high rates of goat (and other livestock) mortality: CSG, FGDs, and KIIs with vets and DOF staff reported mortality rates of up to 50 percent in the first year and they requested additional veterinary (paravet) and medical support from ECOFISH (the paravet program is well-regarded by activity beneficiaries39). • The limits of current approaches: Participants clearly expressed a desire to accelerate income diversification in activity villages and were somewhat frustrated that they were not able to do so more quickly. One typical comment from a woman beneficiary was, “I’m happy to have a new garden but I really want to have a sewing machine so I can really increase my income.” IR 3.3: Strengthened collective action in support of resilient livelihoods ECOFISH is pursuing new innovations related to seed production, pebble dolls activity, and sanitary installations, but it is unclear how these differ from activities under IR 3.2. Progress on IR 4: Improved Policy, Power, and Incentives Table 6. Progress on IR 4 Targets Indicator Unit Base Year LOP Target 2016 Target 2016 Actual 2017 Target 2017 Actual 2018 Target 2019 Target 15. # Laws, policies, etc. addressing climate change # 2015 0 5 1 1 2 2 The activity has generated products that are counted toward IR 4 targets and thus would indicate that this IR is on track, but there are questions about what should be counted and how. For example, products counted in 2017 included two working papers by IIED, a study on the proposed Nijhum Dwip MPA by IUCN, a marine tenure report (separately supported by USAID), a study on fish value chain power dynamic by Power Participation Research Center (PPRC), and draft guidelines for the Hilsa Conservation and Trust Fund and two model villages. These activities and outputs may ultimately contribute to the activity outcome and objective, but none are at a point where they have made a substantive and lasting difference. Similarly, the work on model villages that was initiated after an ECOFISH study tour to the Philippines, while sound in theory, does not yet have a coherent strategy for implementation. During field visits to one of the proposed model villages, Balaramsura in Bhola district, it was evident that neither the ECOFISH team nor the villagers had a clear vision of what a model village might be. The model village is a broad concept endorsed by the DOF (which also participated in the study tour) and is likely to be a key component of the WB Fishery project.40 In the context of Year 4 activities, it is evident that there is a need to accelerate and focus investments on IR 4. With just two years remaining in the activity, there is an urgent need for a more deliberate and focused policy strategy under IR 4 if the work of ECOFISH is ultimately to be sustained and inform fisheries and community resilience at scale throughout Bangladesh. The Year 4 work plan proposes to advance several key laws and policies but contains inadequately defined steps to accomplish key strategies. For example, the long-awaited update to the National Hilsa Fisheries Management Action Plan lacks details of what role ECOFISH will play. Similarly, under the PES and Hilsa Conservation and 39 The importance of this program was consistently highlighted in FGDs with CSGs and reinforced in KIIs with paravets who observed that they are challenged to keep up with demand for their services. 40 Information provided by World Bank KII. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 24 Development Fund41 activities, there is no discussion of specific financial mechanisms. Additionally, the plan proposes to introduce a Pre-Marine Stewardship Council (MSC) certification plan for hilsa that is not synchronized with any other activities and seems unnecessary at this stage of the activity. 5.1.2 CONCLUSIONS Overall, the evaluation team found that the ECOFISH activity has made strong progress toward, and is on track to achieving, its goals of (a) ecosystem resilience through the establishment of effective co￾management in hilsa shad (Tenualosa ilisha) fish sanctuaries; and (b) strengthening community resilience by improving the savings, livelihoods, and coping strategies of marginalized and extremely poor fisher folks, particularly women, and thereby reducing their risk from climate-affected shocks and stresses. There is also a palpable sense of improved community resilience that has been generated among community members engaged in co-management processes such as CSGs and CFGs, both of which provide new opportunities for individuals to engage in local economic development and governance activities. However, despite many strongly performing activities within the overall activity results framework, the evaluation team found that it is not easy to directly trace links between ECOFISH actions and intended activity outcomes. This finding indicates that the activity team needs to revisit the structure of the overall TOC and determine how it informs/guides activity priorities and allocation of effort. Improving the reporting clarity and accuracy of results is also an important issue for ECOFISH to address, both in the body of work currently in circulation and when producing future activity reports and related publications such as conference presentations and journal papers. The evaluation team observed that WorldFish Headquarters (HQ), a leading global fisheries research institution, has not played the role that was anticipated by USAID in technical support for this activity to date and that this is a contributing factor to the lack of rigorous editorial control of ECOFISH publications (see Annex VII). The evaluation team found that a key limiting factor on progress under IR 4 is insufficient resources dedicated to policy capacity. So far ECOFISH has invested only nine percent of total funding toward IR 4, compared with 21 percent for IR 1, 22 percent for IR 2, and 31 percent for IR 3.42 Despite the exceptional working relationship between the Chief of Party (COP) and the DOF, there is not enough staff capacity within the ECOFISH team to fully shape and pursue policy objectives. This may be partly because IR 4 was added to the design of ECOFISH after the activity commenced, but the evaluation team considers that it is more likely due to the lack of dedicated staff capacity. For example, the assessment of marine tenure and small-scale fisheries by Pomeroy, Thompson, and Courtney (2016) supported ECOFISH’s piloting of a co-management approach in the greater Andarmanik area but expressed concern about the process for MPA establishment at Nijhum Dwip. This concern proved valid; the IUCN team working on that subactivity with little direct support from ECOFISH core staff has encountered similar challenges while developing the MPA proposal. There is little evidence of a formal response or change in direction of ECOFISH activities in response to the assessment’s recommendations, although ECOFISH leads for this IR and IP staff acknowledged the value of the study in broadening their thinking about new approaches to governance, tenure, and co-management. 41 ECOFISH Draft Year 4 Work Plan Sections 4.4.2 and 4.4.3, pp. 41–42. 42 Data provided by WorldFish at request of evaluation team on January 12, 2018. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 25 QUESTION 1 – ECOFISH PERFORMANCE: SUMMARY OF RECOMMENDATIONS Recommendation Responsibility TOC – ECOFISH should review the TOC, develop results chains for specific strategies, and use those to guide the programming of Year 4 and Year 5 activities. Technical assistance may be sought from the USAID E3/FAB Measuring Impact program. ECOFISH with USAID assistance Editorial Quality Improvement – WorldFish should hire a scientific editor (ideally with a specialization in fisheries management) to review and edit all annual work plans and annual reports, select technical reports, and any external conference and journal publications arising from ECOFISH. Priority should be given to recent work and future work, with Year 1 and Year 2 products revised as resources permit. WorldFish HQ and Country Director Indicators – ECOFISH M&E staff should update reports on the standard indicators and review custom indicators as part of a review of the TOC to ensure that indicators present an accurate picture of progress toward the ECOFISH objectives. ECOFISH and sub-IPs Hilsa Science Strategy – ECOFISH, in partnership with BFRI, should convene a National Hilsa Science Workshop in Q3 2018 under IR 1 to establish priorities for all disciplines of hilsa research. This agenda should be formally incorporated into the National Hilsa Management Plan. ECOFISH and BFRI with other University Partners Year 4 Work Plan – USAID should request a revision of the Year 4 work plan to identify immediate priority activities for interim approval only (until May when the Exit Strategy work plan will be adopted). No further village expansion activities should be undertaken in Year 4 or Year 5. WorldFish & USAID Improving Enforcement – ECOFISH should work with partners to develop and test alternative approaches including boat-registration (and seizure)-based approach, drones and other surveillance methods. WorldFish, DOF, Police, and Coast Guard 5.2 QUESTION 2 – OPPORTUNITIES Given the time remaining, what opportunities exist to enhance and strengthen programmatic effectiveness such as by adding, changing, rescaling, and/or removing activities to meet or surpass targets/objectives, within the stipulations of the ECOFISH agreement? 5.2.1 FINDINGS The evaluation team identified eight overarching opportunities that cut across the ECOFISH IRs and have a bearing on how the activity could evolve over the remaining period of implementation. 1. Opportunity to leverage World Bank (WB) Fishery Project – After meeting with WB staff and learning about the significant scale of a proposed bank investment ($200–500M), and after further discussions with DOF staff, the evaluation team finds that there has been insufficient effort to date by ECOFISH to leverage its activities in the final design of the WB project. There are several key components of ECOFISH that the WB project could build upon and sustain such as the model villages, PES, and alternative income generation (including new fishing gear and methods). 2. The field programs of ECOFISH have grown dramatically over the past two years and now include 123 villages. The Year 4 work plan (draft) proposes expanding to 12 new villages, adding another 160 HCGs (to the existing 448), and significantly scaling up several new policy initiatives related to hilsa fisheries management, certification, and marine conservation. Generating these activities while maintaining existing commitments and relationships, all with limited available resources, presents a considerable challenge for the ECOFISH team. There is no clear vision of how those activities should be drawn to a close as the activity winds down in late 2018 and early 2019. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 26 3. Co-management activities under ECOFISH have evolved significantly over the past two years, as indicated in Table 4. There are now literally hundreds of community, government, and industry stakeholders participating in multilevel ECOFISH co-management activities, many of which are very new and evolving (e.g., model villages). The emerging models for integration of these institutions are still under development (see Figure 5). The evaluation team is concerned that too many co-management institutional elements are being piloted and proposed with insufficient analysis of how these will be ultimately coordinated and sustained. A more deliberate co-management strategy is needed. One simple improvement would be to move the Resilient Model Fishing Village Activity (p. 42) from IR 4 to IR 2 in the Year 4 Work Plan. 4. EAFM is increasingly encouraged by USAID and is slowly being recognized by ECOFISH Activity staff and GOB partners as an opportunity to undertake more integrated fisheries management. However, there are significant capacity constraints in developing EAFM approaches and a lack of common understanding within DOF and other ECOFISH IPs of what EAFM entails. This is not uncommon among fisheries managers globally as the approach is a relatively recent and evolving area of fisheries management (Osgood, 2014). Interviews with GOB officials revealed that recent ECOFISH-led study tours to Senegal and the Philippines have opened their eyes to the opportunities associated with EAFM, but they do not yet see how it can be applied in Bangladesh. 5. Rice Subsidy Programs – In KIIs with all GOB staff, journalists, and community leaders, the evaluation team found that one of the most controversial interventions associated with hilsa fishing ban periods is the 40-kg rice subsidy provided by the government during the ban periods ostensibly to compensate poor households for reduced access to fish. The subsidy program, while important in providing food to compensate for restricted access to fisheries, is plagued with political influence. KIIs with DOF district offices advised that up to 50 percent of the rice is appropriated by political leaders to create influence or to cover the costs of rice distribution. ECOFISH has understandably remained disengaged from this program, but FGD participants indicated that they expect it to speak up on their behalf. 6. National Strategic Action Plan for Hilsa Conservation and Management – The ECOFISH Year 4 draft work plan suggests that this plan (which is a key outcome of ECOFISH) will be completed in 2018. The plan, however, does not provide any details on the process or the ECOFISH team’s role and responsibilities. It is critical that ECOFISH should be deeply engaged in this process and its many component activities (such as the proposed national hilsa research agenda-setting workshop). 7. Payment for Ecosystem Services (PES) – There has been some analysis to date of a PES option as part of ECOFISH. Activity staff and partners interviewed, including DOF staff, expressed a range of views on what is required to create funding needed to sustain key ECOFISH activities such as enforcement and AIGAs. A concept paper on a “Hilsa Conservation and Development Fund” has been drafted. The Year 4 work plan proposes that implementation of this activity should commence but it is one of seven major activities under IR 4 and there is no evident increase in resource or staffing allocation. 8. Exit/Transition Strategy – As indicated in the seven opportunities mentioned above, business as usual is not an appropriate way forward for the activity over the remainder of the activity implementation period. ECOFISH does not currently have any vision of next steps after the end of current funding and has no clear plan for closing out, transitioning, and/or sustaining activities as the implementation period of the current activity nears completion. Activity staff and GOB partners appear reluctant to acknowledge that they are now entering the tail end of the activity. Mid- and senior-level staff in ECOFISH and local partner organizations appear to be too busy managing current activities to step back USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 27 and take that longer view of the activity and where it fits within the suite of fisheries and climate initiatives currently being implemented and under development in Bangladesh. 5.2.2 CONCLUSIONS To address the many opportunities identified in this section, the evaluation team recommends that ECOFISH develop a more robust and focused exit and transition strategy that incorporates the following: ● A dedicated effort to shape the design of the proposed WB Fisheries activity such that it builds on programs developed under ECOFISH and provides pathways to the sustainability of ECOFISH initiatives ● A more focused set of strategies in the Year 4 work plan to sustain and leverage key ECOFISH activities such as CFGs, design of model villages, and development of a National Hilsa Action Plan; delivering on these signature ECOFISH initiatives will require a reduction of high-cost transactional activities (such as village expansion programs) and reallocation of staff and budget to policy-level programming ● Additional support to the National Hilsa Management Plan process by providing a secretariat, coordinating meetings, assisting in producing and informing specific strategies included in this document based on ECOFISH experience ● An assessment of whether there is an opportunity for ECOFISH to seek greater transparency in the rice subsidy program and possible alignment with other AIGAs and community investment programs (e.g., would it be possible for ECOFISH to work with the GOB to replace rice with other commodities, using the experience of ECOFISH with AIGAs?) ● Greater resource allocation and effort to the Hilsa Conservation and Development Fund to provide detail on how it will operate at the local and national level, including the level of support that may be provided by the WB Additionally, the evaluation team concluded that it is unrealistic to expect that ECOFISH will be able to develop a comprehensive EAFM program in the remaining period of activity implementation. To meet other activity goals, a narrower focus in ECOFISH EAFM effort is warranted in the near term (see Section 4.5), including scaling back work by IUCN on MPA establishment and by the Wildlife Conservation Society on dolphin and shark bycatch.43 QUESTION 2 – OPPORTUNITIES: SUMMARY OF RECOMMENDATIONS Recommendation Responsibility Exit/Transition Strategy – There is urgent need for an activity exit strategy that incorporates scenarios for a transition of ECOFISH activities to GOB (especially the WB Fishery Project), community organizations, and other development projects. The strategy should be developed as part of the Year 4 work plan process and should be submitted to USAID for approval no later than May 2018. ECOFISH and GOB EAFM/Megafauna Conservation – Given the need to prioritize resources for core fisheries and community resilience programs, the team recommends that conservation activities under 4.1.2 of the Year 4 work plan be scaled back. ECOFISH and Wildlife Conservation Society (WCS) Model Villages – A specific focus on the current model villages is needed. ECOFISH should reallocate current effort to ensure that there is a dedicated strategy and lead for both villages. ECOFISH, DOF District Offices, and local NGO partners 43 See ECOFISH Year 4 Work Plan draft, p. 13. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 28 5.3 QUESTION 3 – LIVELIHOODS AND RESILIENCE Is there evidence that the activity has led to improved livelihoods (e.g., improved earnings) and greater resilience to shocks associated with climate change or other issues? 5.3.1 FINDINGS Hilsa is a fundamental component of the quality of life and well-being of Bangladesh riverine and estuarine communities in a similar manner to cod (Kurlansky, 1997), salmon (Montgomery, 2003) and other iconic species of fish that underpin the economy, food security and cultural identity of long￾established riverine and coastal communities. Based on the many published papers reviewed,44 prior to ECOFISH, there was little formal understanding of the many values of hilsa and the relationships between hilsa fisheries and other livelihoods, community resilience, and vulnerability. One of the most innovative contributions of the ECOFISH activity to date in understanding the relationships between fisheries and resilience was a study undertaken by the International Institute for Environment and Development (IIED) under an ECOFISH contract that provided a first-ever approximation of the non-use values of hilsa (Mohammed et al., 2016). According to the study, “Lower￾income groups are willing to pay proportionately more for hilsa fish restoration than high-income groups will pay. This implies that low-income segments of society are most reliant on the hilsa fishery, and therefore investment in hilsa restoration is pro-poor.” This finding reinforced the conceptual framing and strategic significance of USAID investment in hilsa fisheries and hilsa-dependent communities throughout the Meghna River ecosystem. The importance of hilsa to communities of the Meghna River is well-articulated by a participant in the FGD in the Bhola District (which is the epicenter of hilsa fish catch in Bangladesh): “When we have hilsa everyone is happy. When we don’t have hilsa, people do whatever they can to find food or make money and that has a negative impact on our environment... so aren’t we better to protect our hilsa?” As portrayed in Figure 1 and reported in FGDs and KIIs throughout the activity area, hilsa production has been increasing steadily over the past decade and at a faster rate in the Meghna River (and adjacent areas including coastal waters) since 2015. This has resulted in increased income from hilsa catch in 2016–2017 and is consistent with the activity Monitoring, Evaluation, and Learning (MEL) Plan, which employs a USAID standard indicator (EG.10.2.3) to track “the number of people with increased economic benefits derived from sustainable Natural Resources Management (NRM) and conservation as a result of USG assistance.” ECOFISH measured year-on-year trends in the income of beneficiaries who “successfully adopt climate-resilient improved practices and production systems (e.g., high-value crops, aquaculture, and poultry) or who adopt other alternative livelihoods that reduce the exploitation of targeted natural resources. In the riverine areas, the indicator additionally captures households who achieve higher fish catches and incomes that result from increased fish productivity generated from improved fisheries practices. It is noted that after the intervention of ECOFISH, income from fishing and new employment from alternative income generating activities (AIGA) increased.” 45 As outlined in Table 7, ECOFISH measured baseline income in 412 households in 2016, divided broadly into fish-related income and other types of income. In 2017, the activity again measured household income in those categories. 44 Listed in Annex IV, Part 2 45 ECOFISH Year 3 Annual Report, October 31, 2017, p. 14–15. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 29 Table 7. Annual (2016–2017) Income of Fishers from Fishing and Other Sources46 Source End line (n = 412) Baseline (n = 412) Change (BDT/HH) Change (%) Amount (BDT/HH) % of Income Amount (BDT/HH) % of Income Fishing 114,220 83 68,885 78 45,335 66 Others 23,490 17 18,970 22 4,520 24 Total 137,710 100 87,855 100 49,855 57 Despite the somewhat confusing way in which the data is presented (reflective of earlier comments about editorial quality control), the overall results of this survey affirm the positive impacts of both increased fish production and more diversified household income sources that ECOFISH has supported. The activity has targeted increased economic benefits for 25,473 household members (15,180 men and 10,293 women) by providing livelihood support during the hilsa ban period as well as AIGA support to diversify year-round income sources. These findings were reinforced in the mini-survey results, the FGDs, and a study of 248 households by Islam et al. (2017). In addition to these direct economic benefits, and supplementation of government rice subsidies,47 there are additional noncash benefits like social recognition, enhanced capacities and access to government and nongovernment services, and increased opportunity for local entrepreneurship. Several respondents pointed out in conversations with the team that new fruit plants such as starfruit offered them a comparative advantage in local markets. By attracting new customers to their new products, beneficiaries were also likely to be able to sell more of their existing produce such as beans and mangoes. AIGA support thus not only created new sources of income for these households, it also may have contributed to the growth of existing income streams. Despite these overall positive findings, the evaluation team has unanswered questions regarding trends in hilsa production and community resilience that are of concern to activity beneficiaries. HCG group participants stated that they had enjoyed increasingly higher hilsa catches over the last decade, which has accelerated during the period of ECOFISH. However, some respondents are not sure of the relative contribution of changing biophysical and socioeconomic factors to this outcome. They questioned whether the increases are affected by favorable weather conditions, good river temperatures and flows, and increased fisher mobility. Others KIIs also raised concern about the impact of ongoing climate change on hilsa in relation to trends such as more variable river flows and temperature changes.48 Despite these concerns, both activity beneficiaries and non-beneficiaries were optimistic about the future of hilsa fisheries. 46 ECOFISH Year 3 Annual Report, October 31, 2017, p. 16 (note original errors in presenting end line HH data corrected here). 47 Islam et al. (2017) points out that only 52 percent of surveyed fishers have a “jele card,” which is a precondition for receiving government support during the hilsa/jatka ban periods. 48 Such concerns are increasingly prevalent in global fisheries management as warmer waters change the fundamental parameters for many fish stocks (Hulsenbeck, 2012). USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 30 5.3.2 CONCLUSIONS ECOFISH has demonstrated that it is feasible to add value to hilsa fisheries via fisheries conservation measures and to improve the resilience of fisheries-dependent communities by diversifying their economic opportunities. Activity beneficiaries have increased income from fisheries and other economic activities and are more resilient to climate changes that affect hilsa-dependent communities. Importantly, through engagement in ECOFISH, those communities are now more confident about their future and have greater knowledge and more tools available to continue adapting to future shocks that affect current forms of economic production. ECOFISH communities are also now more aligned with, and directly contribute to, larger-scale GOB fisheries conservation programs that benefit communities well beyond the ECOFISH villages. Non-beneficiary villages have also experienced the wider-scale benefits of increased hilsa production attributed to ECOFISH programs; however, they lack access to the pathways for economic diversification that ECOFISH communities enjoy. QUESTION 3 – LIVELIHOODS AND RESILIENCE: SUMMARY OF RECOMMENDATIONS Recommendation Responsibility Paravet Expansion – To address beneficiary concerns about the high rates of livestock mortality in some villages, ECOFISH should strengthen the paravet program with more training and support and should link with animal medicine companies and ULO office. ECOFISH, local NGOs, and GOB 5.4 QUESTION 4 – CO-MANAGEMENT How effectively has ECOFISH advanced the co-management approach with the GOB, local communities, and other stakeholders? What evidence is there that this approach is successfully combating overfishing/ improving sustainability of fisheries? 5.4.1 FINDINGS ECOFISH has pioneered new fisheries co-management approaches in Bangladesh. While there are long￾established models of co-management in related sectors such as forestry and recent USAID co￾management investments in protected areas and wetlands,49 fisheries management in Bangladesh has largely been the responsibility of government with the engagement of private sector actors and little direct engagement of communities.50 This was identified by NRI (2003) as a key factor in the disenfranchisement of riverine and coastal fisheries communities and targeted in the ECOFISH activity design (USAID 2014, p. 16) as a specific opportunity for improvement. ECOFISH has developed a wide range of co-management institutional infrastructure at the village level and more recently at the ghat and Upazila levels. These have been made possible by the active support at all levels of the Department of Fisheries, particularly at the district level where WorldFish and its partners such as CNRS, CODEC, and COAST Trust maintain field offices inside the DOF offices. During KIIs with district fisheries officers and their staff, it was evident that this close working relationship has been key to building a common vision and local capacity for other co-management infrastructure (such as HGGs, HCGs, and CFGs). ECOFISH activity staff participate in quarterly district-level meetings with 49 The USAID Integrated Protected Area Co-management project (2008–2013) and Climate Resilient Ecosystems and Livelihoods project (2012–2017) have supported the Ministry of Environment and Forests in establishing co-management in more than 25 protected forests and wetlands, including the Sundarbans. 50 Some efforts were made under the Department for International Development (DFID)/WorldFish Community-based Fisheries Management Project Phase II (CBFM-2), which was implemented with 11 NGOs at 120 sites with 23,000 fishing households (2001–2006), but these were reportedly not able to equally engage government and community approaches. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 31 DOF personnel and partner NGO staff to coordinate plans and discuss progress.51 These ECOFISH investments in community capacity building to support fisheries co-management have begun to: • Create increased awareness of the importance of a collaborative approach to hilsa fisheries management. All KII and HCG participants pointed to the importance of ECOFISH in raising awareness of hilsa conservation needs and methods. Similarly, according to the mini-survey of ECOFISH villages, there has been a positive change in perceptions of hilsa availability and increased confidence in the future of hilsa fisheries in beneficiary villages, and that has also spilled over to adjacent (non-beneficiary) villages. • Increase commitment/buy-in to collaborative management approaches, including support for enforcement activities. During FGDs with HCGs, HGGs, and CSGs and in KIIs with the Coast Guard, police, local DOF officials, and journalists, there was unanimous support for the role being played by the CFGs. All KII respondents reported that the CFG program was an effective innovation of ECOFISH and that CFGs had a major impact during ban periods working with local co-management committee members and other enforcement personnel. DOF and CFG KIIs also raised questions about whether there were ways to make their enforcement work more cost-effective (e.g., use of vessel registration systems to identify illegal fishermen). • Increase awareness of the importance of, and respect for, ban periods. During each village visit by the evaluation team, all respondents indicated that hilsa abundance has increased over the past two years. When asked why, the most common response was “compliance” (with the ban period and gear restrictions). This trend was reinforced by data from the police and Coast Guard KIIs who confirmed that arrest rates had declined as more people complied with ban periods. • Invigorate women’s opportunities for engagement in fisheries management and related community development activities. Despite the challenges of changing gender norms, particularly in conservative and traditional rural villages,52 ECOFISH has made dramatic progress over the past two years in engaging and empowering women to expand their roles in fisheries co-management and community development. The most obvious and targeted change has been in women’s engagement in business and microfinance activities under the CSGs. A total of 133 CSGs are now established with some 4,150 members. An equal number of women (4,604) are now active in the 448 HCGs (39 percent of total HCG participation). Beyond these groups and participation levels, it was also evident to the evaluation team that ECOFISH has had a profound impact on the lives of individual women. In each village visited, women were engaged in focus group and casual discussions about their experience with ECOFISH. The following two quotes summarize the rapid change in the circumstances of women in those ECOFISH Activity villages: I could never have imagined that two years after this project began I would be able to come out of my house and talk with you – before ECOFISH we were always told that only our husbands could speak with visitors to this village. 51 Two District Fisheries Officers interviewed reported to the evaluation team that these quarterly meetings were not as effective as they hoped – they felt as if they had been advised of what was happening rather than engaged and empowered in the process, but overall still felt it helpful to have such easy access to ECOFISH IPs. 52 The challenges of working in traditional villages where men control women’s activities was made apparent to the evaluation team during one FGD where a participant was only able to join the meeting after receiving support from ECOFISH IP staff to assure her that her husband would not divorce her (as he threatened) if she attended. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 32 Can you imagine how happy I am to have this new house? I don’t know if this is because of ECOFISH or not really but we have more income now that there is more ilish and we have more ilish since ECOFISH. Figure 5. Model Co-management Structure53 Despite the initial effectiveness of many of these co-management interventions, unresolved issues with co-management strategies under IR 2 still remain at all levels of the ECOFISH activity: t At the local level, there are overlapping co-management efforts (especially between HCGs and HGGs) and a lack of a clear strategy for sustaining these (and related) co￾management efforts once ECOFISH ends. There are also significant unresolved issues regarding the emergence of Union and Upazila co￾management committees, and how these should best be coordinated within the proposed model structure outlined in Figure 5. This lack of adequate coordination is not surprising given the somewhat “organic” approach that ECOFISH has employed in developing co-management infrastructure in different districts and reinforces the need for a more defined and nationally integrated co￾management system. • ECOFISH has been only sporadically engaged in Jatka Task Force Committee (JTFC) meetings (these are mostly a GOB responsibility). ECOFISH’s role in those JTFC meetings needs to be reconsidered. • ECOFISH needs to decide how best to support the establishment of District Co-management Committees. The evaluation team considers this institution likely to be key in sustaining co￾management activities after ECOFISH and so it merits additional investment in the near term. • At the national level, there is currently no operational overarching co-management institution. It is unclear how this will be established and what role ECOFISH will play in this structure.54 Perhaps equally significant, ECOFISH does not have a full-time presence in the national DOF as it does at the district levels of DOF, and that limits opportunities for engagement in key national policy development. 5.4.2 CONCLUSIONS Collectively, ECOFISH co-management interventions have contributed directly and indirectly to the observed increase in hilsa stocks by promoting awareness of ban periods, building community support 53 From ECOFISH Year 3 Annual Report, p. 40. This is an idealized structure, not yet operational. 54 It was reported (ECOFISH Year 3 Annual Report, p. 41) that the DOF/MOFL has published (in Bengali) the co-management structure, but the role of ECOFISH was unclear in that process. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 33 for compliance with the ban and gear restrictions, and providing alternative livelihoods that incentivize fisher families to pursue alternative income sources during ban periods and possibly to reduce overall fishing.55 The limitations of current co-management efforts and previously noted concerns about how ECOFISH local partners will transition/exit from existing co-management activities reinforce the need for an activity exit strategy that draws together the many disparate co-management elements into a more cohesive structure. In developing this strategy, it would be timely to reengage CRC/URI given its familiarity with the activity and global expertise in co-management. QUESTION 4 – CO-MANAGEMENT: SUMMARY OF RECOMMENDATIONS Recommendation Responsibility Co-management Strategy – To develop the activity transition/exit strategies under IR2, the team recommends that CRC/URI be reengaged. Part of that engagement should be to advise on the models that ECOFISH is piloting and to help identify how models should fit within the overall national co￾management structure. ECOFISH also needs to ramp up efforts on establishing district-level co￾management models. WorldFish and URI/CRC Boost Policy Staffing – ECOFISH should employ an additional senior policy advisor whose primary responsibility will be to support the National Hilsa Management Plan and Conservation Fund processes and to engage actively in the design of the WB Fishery project. ECOFISH National Coordinating Office – To ensure closer alignment with DOF national programs and to support the National Hilsa Management Plan (update), ECOFISH should establish and maintain a rotating staff presence at the DOF National Office. ECOFISH, DOF Institutionalization of Community Fish Guard Program – The CFG’s performance is key to the effectiveness of the ban period and so its sustainability needs to be addressed as part of the overall co-management infrastructure and financing strategy. ECOFISH, DOF 5.5 QUESTION 5 – GENDER ENGAGEMENT How effective have the project’s gender engagement activities been under IR 2 and IR 3? 5.5.1 FINDINGS As was noted in Figure 4 and throughout section 4.2, gender has been a prominent and deliberate strategic focus of ECOFISH since activity conception. The activity concept note56 describes three core gender-related strategies: 1. IR 3.1, which focused on access to finance for technology uptake, targeting women 2. IR 3.2, which addresses livelihood diversification for poor and vulnerable households, and increased equality in the value chain 3. IR 3.3, which covers strengthened collective action and accountability in support of resilient livelihoods “Of particular note in the approach taken in this IR is the engagement of men as partners in efforts to empower women. This approach is built on increasing evidence that shows that focusing solely on women can lead to negative outcomes for them in the household. All activities aimed at empowerment of women will seek to establish a joint responsibility of women and men in the households and communities.” The activity MEL plan (ECOFISH 2015 as revised in 2016 and 2017, p. 8) emphasized the key focus of the activity TOC by depicting a cross-cutting strategy to “Promote Gender and Good Governance” as a revision to the activity results framework. The final key activity design element related to gender is the ECOFISH Gender Strategy (ECOFISH, 2015), which was drafted by the ECOFISH Gender Specialist who still works on the activity as one of the few specialist ECOFISH staff with cross-cutting (across IRs) 55 Note that not all these effects have been measured by ECOFISH but they were reported in KIIs. 56 See USAID Project Concept Note 2014, p. 16. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 34 responsibilities. The gender strategy is ambitious. Founded on a “blended learning for action model,” and drawing from CGIAR gender research, the strategy proposed several guidelines for ECOFISH implementation such as equal opportunity employment, gendered budgets, gender-neutral language, and a gender-disaggregated reporting system. The strategy cautions that “The inclusion of gender, particularly in fisheries programs, has a slippery habit of disappearing” (p. 15). The ECOFISH emphasis on gender-sensitive behavior was evident throughout the fieldwork of the evaluation team in activity conduct, with activity staff respectful of behavioral norms and mindful of the emphasis of ECOFISH on both the role of women and the role of men in accomplishing activity outcomes. In all work plans and annual reports, there was clear emphasis on gender-specific engagement strategies and reporting. For example, the Year 1 work plan57 included two specific gender equity strategies and initiated studies of gender relations that have continued throughout the activity.58 ECOFISH also convened two Fisher Women’s Congresses (in 2016 and 2017) to highlight women’s role in the hilsa value chain and reduce gender-discriminatory norms. KIIs with participants in those events suggest that those forums have given women a greater level of confidence about the legitimacy of their involvement in fisheries decision-making. A male participant in those meetings said, “The pledge for parity that we all took there was symbolic, but it caused me to stop and think about how I interact with women in my work at DOF and how I should interact in future.” Beneficiaries underscored the importance of lessons learned about the role of gender and gender diversity, as demonstrated in the following three quotes: 1. “I feel I can talk with her (ECOFISH staffer, name withheld for confidentiality reasons) openly about how hard it is for us to change the way men behave in the ban period, when they get bored and they still don’t help us even when we point out that growing vegetables is good for all of the family.” – Female participant in HCG FGD 2. “We know we can trust the staff of CODEC because they give us training that we need – I never believed I could ever have saved this money and now I have goats and I can only do that because I am involved in the CSG and they gave us good training… and they made it possible for enough women in this village to join that the men can see that its good for all of us...” – Female participant in CSG FGD 3. “Men always think they have the hardest job, but I know my wife is important to keep our family strong and now I see what the ECOFISH has done for her I want to work even harder for my family… with more hilsa, we can do more together!” – Male informally interviewed at landing center As was noted in section 5.1.1, while the strong performance and growing capitalization of CSGs is one of the major highlights of ECOFISH to date, there is also evidence of the following: • Strong engagement of women in HCG activities: As indicated in ECOFISH annual reports, females and males engage actively in HCG activities (depending on seasonal activities and availability). Figure 6 indicates an average participation rate of 39 percent women and 61 percent men in HCG training. In KIIs, ECOFISH local partners suggested that this “higher than expected participation rate by women” was driven by the economic incentives associated with increased hilsa production. The partners also noted that “in some villages, particularly where erosion requires household relocations, it is also motivated by fear of change and women are more vulnerable in those areas.” 57 ECOFISH Year 1 Work Plan 2015, p. 35. 58 The Year 3 Annual report contains an extensive analysis of women and men’s status in fishing communities derived from profiles of 42 villages (p. 35–36). USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 35 • Increased male and female collaboration in AIGAs: In Haimchar village, evaluation team members interviewed three couples informally, all of whom noted that with greater prosperity there was more opportunity to share in activities beyond fishing. This, in turn, had improved their respect for the contributions each could make to the household and relaxed previous limitations on women’s engagement in activities outside the household.59 Expansion of personal economic opportunity in ECOFISH villages was also a strong trend reported in the mini-surveys. • Increased entrepreneurship by both men and women: Analysis of AIGA training activities over the past 2 1/4 years indicates fairly consistent trends in the engagement focus of men and women. Certain AIGA activities (such as tailoring and craft manufacturing) are exclusively of interest to women, some activities are predominantly of interest to men (e.g., pond fish culture, which attracts >75 percent men), but some have had variable ratios of participation. There has not yet been a detailed analysis of those trends, but KIIs conducted with activity staff suggest that both sexes increasingly see the value of income diversification. This trend is portrayed in Table 7. • Improved national dialogue around the role of women in fisheries management as a result of the 2016 and 2017 National Fisherwomen’s conferences and a greater respect for the contributions of women, especially among district-, Upazila-, and Union-level DOF staff. Figure 6. Gender Disaggregated Participation in HCG Training60 The final observation relates to the geographic variability of gender engagement in ECOFISH activities. This is partially evident in Figure 6 and was more obvious during the fieldwork and mini￾surveys of the evaluation team. Some of the ECOFISH Upazilas visited were distinctively more conservative than others. This is reflected in numbers of female participants in HCG meetings, but was most pronounced in CSG FGDs. In a third of villages, up to half the women involved in CSGs chose not to actively participate in the FGDs. They attended and listened but did not offer comments. A similar pattern of lack of willingness to engage was observed in the village mini-surveys, where 80 percent of respondents were men and 20 percent women.61 ECOFISH staff indicated that this is the norm in conservative villages when they interact with visitors. Evaluators had a sense from these conversations that ECOFISH has begun to have a positive effect on engaging women in these more conservative villages but the team was unable to assess that hypothesis further as insufficient 59 There is little information currently available from ECOFISH about those changes in gender interactions and behaviors; this analysis is key to understanding the overall impact of ECOFISH on men and women. 60 ECOFISH Year 3 Annual Report, p. 36. 61 The design proposed 50:50 engagement of men and women, but enumerators could not locate sufficient willing female respondents in some villages, especially in Barisal District. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 36 longitudinal data on differences in gender-related behaviors over the course of the ECOFISH activity in different village types is available. 5.5.2 CONCLUSIONS ECOFISH has provided substantially new opportunities for both men and women to engage in fisheries conservation and AIGAs through the various co-management programs. During the process, ECOFISH has also made a significant commitment to cross-cutting gender research and advancement strategies and has begun to compile a significant knowledge base on gender roles and responsibilities and an improved understanding of the efficacy of different strategies for gender engagement. However, the extensive village profile data and data related to the various community-level programs collected by ECOFISH requires further analysis to inform future gender strategies. For example, it would be useful to collect and synthesize ECOFISH data and participant experience to inform the ECOFISH transition and exit strategy with regard to potential future IAGAs and community resilience projects in Bangladesh.62 There is also a need to ensure that the ECOFISH commitment to gender “learning for action” is made manifest in publications that originate from the ECOFISH activity: a draft manuscript that presents a recent analysis of 248 households in the ECOFISH area of operations makes no specific reference to gender roles and changes.63 QUESTION 5 – GENDER ENGAGEMENT: SUMMARY OF RECOMMENDATIONS Recommendation Responsibility Gender Data Harvest – During Year 4, ECOFISH should harvest the datasets related to gender and undertake a comprehensive analysis of lessons learned to help shape village exits by ECOFISH IPs and inform future activities. ECOFISH Gender Specialist/WorldFish HQ MEL team ECOFISH Publications – All publications that share ECOFISH results with a gender-related component should specifically address and share activity findings related to men’s and women’s roles in hilsa fisheries. ECOFISH and partners (esp. Universities) 5.6 QUESTION 6 – FUTURE STRATEGY Given the current state of overexploitation of coastal fisheries resources in Bangladesh, how can USAID best direct future resources, both strategically and programmatically, to implement the Government of Bangladesh’s National Strategy for conserving marine and coastal biodiversity in Bangladesh? 5.6.1 FINDINGS Based on the interviews with diverse key informants at international, national and district levels, the evaluation team identified five specific areas in which USAID could amplify the impact of ECOFISH in future programming: 1. Enabling ECOFISH transition to leverage WB fisheries investments The activity needs a six-month no-cost extension (through October 2019). In addition, a few members of ECOFISH should be sponsored to work directly with the DOF to consolidate ECOFISH strategies in areas such as co-management, the National Hilsa Management Plan, and PES. Such an investment (estimated at <$700,000/year for up to three years) would enable key ECOFISH staff to directly influence the final design and initial implementation of the WB fisheries project and related bilateral and 62 During KIIs with representatives of recent USAID activities Women’s Empowerment Activity (WEA) and Aquaculture for Income and Nutrition AIN. 63 See Islam et al. (2017). USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 37 multilateral investments, and to shape GOB fisheries policies as they learn from and replicate the innovations developed under ECOFISH. 2. Investing in an EAFM activity that targets coastal and marine waters of Bangladesh Given the importance of marine waters for hilsa and other fisheries production, the critical importance of Bangladesh’s coastal zone for national adaptation planning, and the limited opportunities available under ECOFISH to develop EAFM approaches, it seems timely to develop a new EAFM program that complements the fisheries management strategies pioneered by ECOFISH but with a larger geographic focus (all of Bangladesh coastal waters) and a more comprehensive EAFM underpinning. The activity could test marine spatial planning methods that complement or possibly even supersede sanctuary-based management approaches. 3. Support for creation of a regional agreement on hilsa management To create a more effective framework for management of hilsa stocks (which straddle India, Bangladesh, and Myanmar, and possibly Thailand), a regional fisheries management strategy is needed.64 This is important to secure the gains made under ECOFISH. Discussions during KIIs and FGDs revealed concerns about Bangladesh efforts to conserve hilsa being undermined by lack of regulation of hilsa stocks in other countries. This lack of range-wide management coordination is a disincentive for responsible management of hilsa stocks in Bangladesh. There are many global models for such treaties including the US–Canada Yukon River Salmon Agreement.65 While an important issue, the evaluation team recognizes that this is outside ECOFISH’s and USAID’s manageable interest. 4. Riverine and coastal community vulnerability assessments During fieldwork, the evaluation team visited several villages where accelerated erosion rates have washed away farmland, destroyed infrastructure such as roads and bridges, and caused households and – in some places entire communities – to relocate. Some villages, such as Balaramsura in Bhola District, have responded by installing extensive riverbank revetments, but even these are failing to stop the erosion. These threats add to the challenges of poor communities, create considerable risk and uncertainty, and undermine the efforts of activities such as ECOFISH.66 A climate adaptation strategy for the Meghna river estuary (and likely all other Bangladesh estuaries) and for key fish species such as hilsa is urgently needed to guide both near-term actions (management response options) and longer-term investment and adaptation decisions. At times throughout the evaluation process, the team questioned the value of investing in some villages that will not be there in the next three to five years. 5. Strategic science partnerships with US science institutions The evaluation team observed that there is very limited capacity within Bangladesh to undertake rigorous, data-driven, and state-of-practice research in support of activities such as ECOFISH. There are both well-established and emerging scientists with strong research skills in Bangladesh; however, there are also insufficient funding and limited opportunities for two-way international capacity engagement in the various fields of multidisciplinary research that are critical to implementing activities like ECOFISH. The United States has a wealth of science institutions (such as the US National Oceanographic and 64 This observation is also supported by Jalilur et al. (2017) in their analysis of Hilsa MSY, p. 18. 65 See http://www.yukonriverpanel.com/publications/yukon-river-salmon-agreement/ 66 In a survey of 248 households in the ECOFISH area, 67 percent expressed concern about climate-related events such as loss of dwellings and assets due to riverbank erosion and flooding (Islam et al., 2017). USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 38 Atmospheric Administration [NOAA] and National Marine Fisheries Service) and universities67 that could quickly accelerate research efforts in fields related to fisheries management, climate resilience, community development, behavioral economics, experimental research design, aquaculture of hilsa, data synthesis, etc. USAID investment in strategic science partnerships in support of the above types of initiatives would strengthen Bangladesh capacity to secure and build on the models established under ECOFISH and build stronger bilateral linkages. 5.6.2 CONCLUSIONS ECOFISH provides a strong foundation for future USAID strategy and programmatic investment in support of the GOB National Strategy for Marine and Coastal Biodiversity. There are both short- and long-term opportunities to strengthen USAID-GOB engagement in this strategy. Short-term opportunities include the potential to leverage the $15M investment in ECOFISH to shape the proposed $200–500M WB Fishery project, and long-term opportunities include establishing EAFM capacity that ECOFISH was not able to fully address and contributing to regional-scale biodiversity partnerships in the Bay of Bengal. To fully engage in these opportunities, the evaluation team believes that an investment in Bangladesh biodiversity capacity-building via stronger bilateral science partnerships involving US institutions would be helpful. 5.6.3 QUESTION 6 – FUTURE STRATEGY: SUMMARY OF RECOMMENDATIONS Recommendation Responsibility No-Cost Extension – ECOFISH should design the Year 5 work plan to include an additional six months of implementation and submit an NCE request to USAID concurrently with the exit strategy (by May 2018). ECOFISH and USAID Strategic Initiatives – As part of its next phase of partnership with the GOB, USAID should consider (1) EAFM activities that target coastal and marine waters of Bangladesh (including fisheries and megafauna conservation); (2) supporting work toward a hilsa international management agreement; (3) riverine and coastal community vulnerability assessments; and (4) strategic science partnerships with US institutions. USAID and GOB DOF Secretariat – As part of the activity transition and exit strategy, USAID and WorldFish should evaluate the value of a small ($700k/year for three years) continuation of specific IR 4 activities under ECOFISH to leverage WB and related investments. This proposal would support a small ECOFISH team embedded in DOF to ensure that there are explicit linkages between policy, funding, and co-management processes initiated under ECOFISH, the WB, and other donor investments in hilsa fisheries. USAID and WorldFish 67 See National Association of University Fisheries and Wildlife Programs, http://naufwp.org/about/ USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 39 ANNEX I: EVALUATION STATEMENT OF WORK MIDTERM PERFORMANCE EVALUATION OF ENHANCED COASTAL FISHERIES (ECOFISH) UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID)/BANGLADESH OFFICE OF ECONOMIC GROWTH APRIL 2017 POINT OF CONTACT RENERIO ACOSTA AGREEMENT OFFICER REPRESENTATIVE (AOR) ECOFISH OFFICE OF ECONOMIC GROWTH USAID/BANGLADESH TEL: (+88-02) 5566-2341; CELL: +88 0171 3141186 EMAIL: RACOSTA@USAID.GOV FARZANA YASMEEN CONTRACT OFFICER REPRESENTATIVE (COR) ACCELERATING CAPACITY FOR MONITORING AND EVALUATION (ACME) OFFICE OF ECONOMIC GROWTH USAID/BANGLADESH TEL: (+88-02) 5566-2381; CELL: +88 0175 5629772 EMAIL: FYASMEEN@USAID.GOV Project to Be Evaluated Project Name Enhanced Coastal Fisheries in Bangladesh (ECOFISHBD) Award Number BFS-G-11-00002-00 Original Project Dates June 1, 2014, to May 31, 2019 Original Funding $15,000,000 Implementing Partner WorldFish Center (WFC) Major Counterpart Organizations Department of Fisheries (DOF), Ministry of Fisheries and Livestock, Government of Bangladesh (GOB) Zone of Influence (ZoI) Barisal, Bhola, Patuakhali, Barguna, Jhalokhati, Pirojpur, Chandur, Laxmipur, and Shariatpur BACKGROUND Enhanced Coastal Fisheries in Bangladesh (ECOFISHBD) is a five-year initiative ending September 30, 2019. It is funded by the United States Agency for International Development (USAID) and implemented jointly by WorldFish and the Department of Fisheries (DOF). ECOFISHBD supports coastal fishing communities and other fisheries value chain stakeholders to improve the resilience of the Meghna River ecosystem and communities reliant on coastal fisheries. The primary pathway is through the establishment of effective co-management of hilsa fish sanctuaries, with supporting advancements in USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 40 fisheries science, alternative income-generating activities, biodiversity conservation, and policies and incentives for fisheries conservation.68 ECOFISH contributes to the USAID Country Development Cooperation Strategy Objective 4, “Responsiveness to Climate Change Improved,” which aims to increase the nation’s responsiveness, adaptation to climate change, and mitigation of climate change through three interrelated intermediate results. ECOFISH aligns with two of these: (1) improved management of natural resources focused on biodiversity conservation leading to indirect adaptation benefits, and (2) enhanced adaptation capacity and resilience to shocks. ECOFISH LINKS TO USAID/BANGLADESH’S COUNTRY DEVELOPMENT COOPERATION STRATEGY The ECOFISHBD project was developed within the context of USAID’s broader Global Climate Change (GCC) and Biodiversity Conservation Initiatives. ECOFISHBD will incorporate identifying and maintaining biodiversity and other priority ecosystem services that support fisheries and men and women in poor fishing communities. The ECOFISHBD activity also assists Bangladesh in reducing its vulnerability to climate change impacts by decreasing the environmental, economic, and social consequences of climate change. Focus will also be given to concomitant improvements in policies and strategies as well as institutional strengthening by capacity building of two national organizations, Department of Fisheries (DOF) and Bangladesh National Fisheries Research Institute (BFRI), to deliver more effective science inputs, fisheries management, and desired policy outcomes.69 USAID/BANGLADESH HAS A SPECIFIC COMMITMENT TO IMPROVED CLIMATE CHANGE RESPONSIVENESS (FIGURE 1). ITS 2011–2016 COUNTRY DEVELOPMENT COOPERATION STRATEGY (CDCS) IS ORIENTED AROUND FOUR DEVELOPMENT OBJECTIVES (DOS). DO4 (RESPONSIVENESS TO CLIMATE CHANGE IMPROVED) AIMS TO INCREASE THE NATION’S RESPONSIVENESS, ADAPTATION, AND MITIGATION OF CLIMATE CHANGE THROUGH THREE INTERRELATED INTERMEDIATE RESULTS (IR): (1) IMPROVED MANAGEMENT OF NATURAL RESOURCES; (2) ENHANCED ADAPTATION CAPACITY AND RESILIENCE TO SHOCKS; AND (3) STRENGTHENED CAPACITY FOR LOW-EMISSIONS DEVELOPMENT FOCUSING ON MITIGATION. 68 ECOFISH Annual Report, Year 2. 69 ECOFISH MEL Plan, 2015–2019. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 41 Figure 1 DEVELOPMENT HYPOTHESIS OF ECOFISH USAID has identified the following development hypothesis under its DO4: “Improved management of natural resources alongside livelihood diversification, climate risk management, and enhanced capacity for low-emissions development will address adaptation and mitigation of GCC while providing sustainable economic benefits and clean energy resources for Bangladesh. ECOFISH contributes directly to achieving USAID’s DO4 results. The four intermediate results areas included in the ECOFISH results framework and TOC are causally linked. At the highest level the core links can be expressed as follows: “If decisions around fisheries management are based on shared, evidence-based objectives and learning (Result 1), are grounded in inclusive and effective ecosystem-scaled co-management structures (Result 2), and the assets of communities are strengthened (Result 3), then the sustainability of Bangladesh’s complex and productive coastal and riverine ecosystems will be enhanced.” USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 42 Result Framework70 of ECOFISH PROJECT INTERMEDIATE RESULTS ECOFISH has four intermediate results (IRs)/strategic objectives,71 each of which has sub-IRs72 that are logically interlinked and support one another directly or indirectly. Throughout all activities this project has a particular focus on deep and sustained stakeholder engagement processes and ensuring equitable and sustained outcomes for vulnerable social groups, including women. IR 1: Improved fisheries science for decision-making – Will generate scientific information for IR 2 to formulate and implement sound adaptive co-management approach. - Sub-IR 1.1: Improved application of science in resource conservation, planning, and decision￾making - Sub-IR 1.2: Strengthened community-based monitoring of target fisheries and technologies IR 2: Strengthened fisheries adaptive co-management – Will reduce fishing pressure and improve compliance with fisheries management actions. - Sub-IR 2.1: Increased capability of community-based fisheries management systems 70 Project proposal of ECOFISH. 71 MEL Plan, 2015–2019. 72 Annual Report October 2014–September 2015. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 43 - Sub-IR 2.2: Strengthened collaborative management of riverine and coastal areas - Sub-IR 2.3: Increased stakeholder engagement to protect common-pool resources IR 3: Enhanced resilience of hilsa fisher communities – Will support community engagement in co-management by introducing alternative income-generating activities (AIGAs) and input support that enhances livelihood resilience. It will also introduce community savings and loan schemes for women that reduce fisher households’ dependency on credit providers. - Sub-IR 3.1: Access to finance for technology uptake targeting women increased - Sub-IR 3.2: Livelihood diversification for poor and vulnerable households - Sub-IR 3.3: Strengthened collective action in support of resilient livelihoods IR 4: Improved policy formulation, power streamlining, and incentives – Will assist co￾management implementation by providing multilevel advocacy for formulating appropriate policies, power, and incentives that increase compliance with hilsa fisheries management rules and regulations - Sub-IR 4.1: Improved economic valuations, policy briefs, and proposals for GOB - Sub-IR 4.2: Reduced exploitation of fishers by addressing power dynamics and fishers’ rights - Sub-IR 4.3: Sustainable finance for ecosystem services conservation Because ECOFISH is at the middle of project life, USAID would like to carry out a midterm performance evaluation on all activities performed under each component/IR to assess the progress toward its intended goals. EVALUATION OBJECTIVE Consistent with USAID’s 2011 Evaluation Policy, the primary goal of this midterm performance evaluation is as follows: ● To provide evidence so USAID/BD can determine whether the ECOFISH Activity has achieved its desired results to date ● To inform what course corrections, if any, are needed in the remaining implementation time ● To determine how effectively this project has incorporated gender in the interventions ● To recommend to USAID key strategic and programmatic options for future engagement With the exclusion of procurement-sensitive sections, USAID intends to disseminate the report widely to stakeholders such as USAID/BD implementing partners, GOB agencies, universities, other sector￾specific donors, and the Development Experience Clearinghouse (DEC). EVALUATION QUESTIONS The evaluation team will focus on the following questions during the assignment: 1. To what extent is the ECOFISH on track, in terms of progress and outcomes, to meet its overall goals? What factors (internal/external) affected timely achievement of targets? And what factors impeded performance? 2. Given the time remaining, what opportunities exist to enhance and strengthen programmatic effectiveness such as by adding, changing, rescaling and/or removing activities to meet or surpass targets/objectives, within the stipulations of the ECOFISH agreement? 3. Is there evidence that the activity has led to improved livelihoods (e.g., improved earnings) and greater resilience to shocks associated with climate change or other issues? USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 44 4. How effectively has ECOFISH advanced the co-management approach with the GOB, local communities, and other stakeholders? What evidence is there that this approach is successfully combating overfishing/improving sustainability of fisheries? 5. How effective have the project’s gender engagement activities been under IR 2 and IR 3? 6. Given the current state of over exploitation of coastal fisheries resources in Bangladesh, how can USAID best direct future resources, both strategically and programmatically, to implement the Government of Bangladesh National Strategy for conserving marine and coastal biodiversity in Bangladesh? EVALUATION METHODOLOGY Based on the SOW from USAID, the evaluation team will develop a detailed evaluation methodology and work plan for review and approval by USAID. The detailed evaluation methodology will include a data collection matrix that will explicitly link evaluation questions to particular data collection approaches and data sources. The evaluation team must use a mixed-method approach utilizing both quantitative (including nonexperimental design) and qualitative analysis (such as key informant interviews, stakeholder interviews/mini-surveys, and focus group discussions). The evaluation team should develop the best evaluation design methodology in light of the evaluation questions, time frame, budget, data collection requirements, quality of existing data sources, and potential biases. The evaluation team should utilize several different but complementary and interrelated forms of gathering information/data such as those described below. The evaluation team is expected to utilize its expert judgment and evaluation best practices in selecting which methodological components to include in the evaluation design. Document Review: Evaluation team members will review documents throughout the evaluation process, including program reports and relevant studies to ensure that comprehensive and grounded best practices will be identified (see Section VI). Key Informant Interviews: The team will conduct one-on-one interviews with a variety of stakeholders, including the most relevant GOB ministries, agencies, and other projects supporting conservation of hilsa fishery and improved resilience of coastal fisheries communities. This will be done by administering a semi-structured or structured questionnaire. Self-assessment: The IPs will respond to a self-assessment either through a questionnaire or standard interview checklist compiled by the evaluation team and approved by USAID before use. Expert Opinion Survey: Utilizing expert opinion is a technique used increasingly in the market system sector. The evaluation team, with approval of USAID, can apply this method as well. Focus Group Discussions (FGD): The evaluation team will lead open discussion through a skilled moderator to gather semi-structured qualitative data from small focus groups of 6 to 10 people. The preselected participants (balanced to the extent feasible by gender, age, and any other relevant factors) will discuss issues and concerns based on a list of key themes drawn up by the moderator. No more than 10 questions will be addressed by a group. These sessions will encourage free-flowing discussion about the activity. Mini Survey: This type of survey is small (30–40 participants) and can be performed rapidly in the field without analytical software such as SPSS or a large questionnaire. The sample size is not statistically significant; however, this type of analysis can be used to triangulate with other methods. Because of the USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 45 small sample size this type of survey can be implemented quickly when time and resources are constrained. Social and Economic Analysis: This type of analysis might require a sample survey. Due to time and resource constraints, the survey should be kept to a manageable size since it is neither possible nor desirable to carry out a complete census for social and economic analysis. The team should select a sample that is representative of the beneficiary population and triangulate the findings from this survey with other sources. Regardless of data collection and analysis methods, USAID requires qualitative and quantitative data disaggregated by gender. Methodological limitations and challenges for this evaluation are expected to include the following: • Ensuring that samples of interview sources are sufficient to support evaluation findings • Taking systematic actions to counter any biases in (a) reporting by data collection sources and (b) interpretations of collected data by the evaluation team • Ensuring “actual” results can be measured, which will only be possible if data can be gathered and analyzed beyond respondent perceptions All the methodological strengths and weaknesses should be explicitly described in the evaluation report. The methodology must include a process for sharing draft findings and conclusions with key stakeholders, including beneficiaries and GOB officials, and incorporating a second round of their feedback in the final report, to be agreed upon by USAID. Prior to the start of the evaluation, the evaluation team shall meet with EG office to refine the evaluation methodology and to address any other concerns the EG office may have. EXISTING PERFORMANCE INFORMATION SOURCES The evaluation team should consult a broad range of background documents apart from project documents provided by USAID/BD. These should include, but are not limited to, documents on research and project reports on fisheries science, fisheries adaptive co-management, and resilience of fisheries communities both in and outside of Bangladesh. An important evaluation question is whether the ECOFISH project design adequately incorporated knowledge and lessons learned elsewhere. The evaluation team must also research any other relevant documents pertinent to the sector that ECOFISH project serves from GOB, donors and other international organizations, and universities. USAID, ACME, and the ECOFISH project will provide the assessment team with a package of briefing materials that includes the following: 1. Project description 2. Technical/Project proposal 3. Approved M&E plan 4. Approved work plan for each year 5. Final version of all annual and quarterly reports 6. DQA report(s) 7. Any other relevant event reports (inception, expo, fare, etc.) 8. Case and success story (if any) 9. Any research/study reports such as sector study (if any) USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 46 10. Name of partner(s) with contact details (international and national NGOs) 11. Copy of MOUs signed with GOB agencies and other partner(s)/NGOs (if available) 12. Performance Indicator Tracking Table (PITT) 13. M&E/data collection tools 14. Beneficiary database 15. Training database, modules, and reports (if any) 16. GIS Maps, demonstrated the zone of influence (ZOI) 17. Component and yearly project expenditure 18. One-pager/brochure/pamphlet (if available) [ EVALUATION TEAM COMPOSITION The team will include one international team leader, one international/national socioeconomic expert, and one local fisheries expert. The former should include specialists with the following areas of expertise: project evaluations and assessments, fisheries science, fisheries adaptive co-management, community aquaculture, and/or related scientific fields. The local fisheries expert(s) should have an excellent understanding of fisheries research, policies, and co-management in Bangladesh. Team Leader (Evaluation Specialist – International) The Team Leader should have a post graduate degree in marine biology, fisheries management, agricultural economics, agribusiness management, aquaculture enterprise development, economics or an applicable social science field. The Team Leader will be an international and should have at least 10 years of evaluation and technical work experience. Preferably s/he should have demonstrated experience of leading at least two evaluations of projects with similar scope and complexity within last 10 years. The Team Leader should have the following credentials: • Extensive experience in conducting quantitative and qualitative evaluations and strong familiarity with fisheries management, ecosystem approach to fisheries management, fisheries-related business, and value/supply chain development • Familiarity with USAID regulations and systems, including Biodiversity Conservation performance monitoring guidance, gender policies and guidance, project management, budgeting, and financial analysis and reporting • Experience in international donor development program management and overseeing multiple program areas simultaneously (preferred) • Relevant experience in Bangladesh or South Asia (preferred) • Excellent oral and written skills in English (required) The Team Leader will provide overall leadership for the team and will finalize the evaluation design, coordinate activities, arrange periodic meetings, consolidate individual input from team members, and coordinate the process of assembling the final findings and recommendations into a high-quality document. The Team Leader will lead the preparation and presentation of the key evaluation findings and recommendations to the USAID/Bangladesh team and other major partners. Socioeconomic Expert (International/National) The Socioeconomic Expert will have the following credentials: • Postgraduate degree in economics, sociology, social science, fisheries, or any other relevant subject USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 47 • International/national with 8–10 years of experience in socioeconomic development, social behavior changes, fisheries communities, and research in and outside Bangladesh • Solid understanding about the fisheries science/policy, co-management and social impact. The Socioeconomic Expert will participate in team meetings, key informant interviews, group meetings, site visits, and draft the sections of the report relevant to his/her expertise and role in the team. The Socioeconomic Expert will also participate in presenting the report to USAID or other stakeholders and will be responsible for addressing pertinent comments provided by USAID/BD or other stakeholders. Local Fisheries Expert The Local Fisheries Expert will have the following credentials: • Postgraduate degree in fisheries science or any other relevant subject • Bangladeshi national with 8–10 years of experience in fisheries project development, management, and evaluation • Solid understanding of USAID MEL requirements • Demonstrated knowledge of the latest developments in advancing good/best practices in fish sector development that reaches women, youth, and very poor The Local Fisheries Expert will participate in team meetings, key informant interviews, group meetings, and site visits, and will draft the sections of the report relevant to his/her expertise and role in the team. The Local Fisheries Expert will also participate in presenting the report to USAID or other stakeholders and be responsible for addressing pertinent comments provided by USAID/Bangladesh or other stakeholders. Conflict of Interest All evaluation team members will provide a signed statement attesting to a lack of conflict of interest, or describing an existing conflict of interest relative to the project being evaluated. USAID will provide the conflict of interest forms. SCHEDULE, LEVEL OF EFFORT (LOE), AND LOGISTICS Schedule Work is to be carried out over a period of approximately 10 weeks, beginning in May 2017 with fieldwork completed in the same month and final report and closeout concluding June/July 2017. A six￾day work week (Saturday–Thursday) is authorized for the evaluation team while in Bangladesh. The evaluation team will submit a work plan as part of the evaluation methodology proposal with timeline and develop a Gantt chart displaying the time periods during which activities occur. Pre￾departure arrangements should include travel approval, airline tickets, visa, lodging, work facility and vehicle transport arrangements, dates for meetings with USAID/Bangladesh EG staff and key contacts, in-country travel agenda, and accommodations. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 48 Estimated LOE for the Assignment Level of Efforts of Team Members by Task Deliverables Task/Deliverable Duration / LOE Days Team Leader Socioeconomic Expert Local Fisheries Expert Review background documents and home-based preparation work 5 5 5 Travel to Bangladesh 2 2 0 Team planning meeting and meeting with USAID 2 2 2 Development of Evaluation Work Plan (concurrent with document review and initial meetings) 2 2 2 Development of data collection instruments 2 2 2 Information and data collection – includes interviews with key informants (stakeholders and USAID staff) and site visits 14 14 14 Discussion, analysis, and preliminary draft evaluation report in country, including discussion with USAID 5 5 5 Debrief meetings with USAID (preliminary draft report due to USAID) 1 1 1 Debrief meetings with key stakeholders 1 1 1 Depart Bangladesh/travel to US 2 2 0 Finalization of draft and internal review (IBTCI) remote 10 -5 4 USAID provides comments on draft report 0 0 0 Team revises draft report and submits final to USAID (out of country) 5 4 4 Total Estimated LOE 51 45 40 USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 49 DELIVERABLES All deliverables are internal to USAID and the evaluation team unless otherwise instructed by USAID. Evaluation deliverables are indicated below Evaluation Team Planning Meeting: During the meeting, the team should review and discuss the SOW in its entirety, clarify team member roles and responsibilities, prepare the work plan, develop data collection methods, review and clarify any logistical and administrative procedures for the assignment and instruments, and prepare for the in-brief with USAID/Bangladesh. Work Plan: Prior to initiation of the evaluation activities, the evaluation team will provide a detailed initial work plan to the ACME COR. The ACME COR will provide any necessary feedback or edits to the work plan, after which the evaluation team will have three days to submit a final version of the document. The initial work plan will include a task timeline, a description of the methodology to answer each evaluation question, team responsibilities, document review process, key informant and stakeholder meetings, site visits, survey implementation (if applicable), data analysis, travel time, debriefings (for USAID, implementing partner, and relevant GOB officials, and draft and final report writing. The work plan will be submitted to the ACME COR and ECOFISH AOR at USAID/Bangladesh for approval no later than the fifth day after the evaluation team arrives in Bangladesh. In-brief Meeting: The evaluation team will meet with USAID/Bangladesh within two working days of the international team members’ arrival in Bangladesh. Evaluation Design Matrix: A table that lists each evaluation question and the corresponding information sought information sources, data analysis methods, and limitations. The matrix should be finalized and shared with USAID/Bangladesh before evaluation fieldwork starts. It should also be included as an annex in the evaluation report. Data Collection Instruments: Development and submission of data collection instruments to USAID/Bangladesh during the design phase and after the evaluation is completed. Regular Updates: The Evaluation Team Leader will brief the ACME COR, ACME COP, the ECOFISH AOR, and any other designated evaluation point of contact on progress with the evaluation on at least a weekly basis, in person or by electronic communication. Any delays must be quickly communicated to USAID/Bangladesh as early as possible to allow quick resolution and to minimize any disruptions to the evaluation. Emerging opportunities to strengthen the evaluation should also be discussed with USAID/Bangladesh as they arise. Preliminary Draft Evaluation Report: The evaluation team will submit a Preliminary Draft Evaluation Report to the ACME COR, ECOFISH AOR, and ACME COP five working days before the Mission debriefing. Within three working days after receipt, USAID staff will provide preliminary comments prior to the Mission debriefing. Debriefing with USAID: The evaluation team will present the major evaluation findings to USAID/Bangladesh through a PowerPoint presentation before the team’s departure from Bangladesh. The debriefing will include a discussion of achievements and issues as well as any preliminary findings. The team will consider USAID comments and incorporate them as appropriate in the Draft Evaluation Report. Draft Evaluation Report: A draft report on the findings and recommendations should be submitted to USAID/Bangladesh and ACME within 15 business days after departure of the international team USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 50 members from Bangladesh. The written report should clearly describe findings, conclusions, and recommendations. The draft report must be of high quality with no grammatical errors or typos. USAID will return the report if this requirement is not met. A report is high quality when it represents a thoughtful, well-researched, and well-organized effort to objectively evaluate what worked in the project, what did not work, and why. The draft report must have well-constructed sentences that are presented in a way that clearly presents findings, conclusions, and recommendations. The report should answer all the evaluation questions and the structure of the report should make it clear how the questions were answered. The draft report must meet the criteria set forth under the Final Report section below. Final Evaluation Report: The evaluation team will submit the Final Evaluation Report that incorporates Mission comments and suggestions no later than 10 working days after USAID/Bangladesh provides written comments on the Draft Evaluation report. The format of the final report is provided below. The report will be submitted electronically in English. The final report should meet the following criteria to ensure the quality of the report: • The evaluation report should represent a thoughtful, well-researched, and well-organized effort to objectively evaluate what worked in the project, what did not work, and why. • The evaluation report should address all evaluation questions included in the scope of work. • The evaluation report should include the scope of work as an annex. All modifications to the scope of work – whether in technical requirements, evaluation questions, evaluation team composition, methodology, or timeline – need to be agreed upon in writing by the ACME COR and ECOFISH AOR. • Evaluation methodology should be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists, and discussion guides will be included in an annex in the final report. • Limitations to the evaluation should be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, etc.). • Evaluation findings should be presented as analyzed facts, evidence, and data, and should not be based on anecdotes, hearsay, or the compilation of people’s opinions. Findings should be specific, concise, and supported by strong quantitative or qualitative evidence. • Sources of information need to be properly identified and listed in an annex. • Recommendations need to be supported by a specific set of findings. • Recommendations should be action-oriented, practical, and specific, with defined responsibility for the action. REPORTING REQUIREMENTS The final report should be no more than 30 pages, excluding references and annexes. The following content (and suggested length) should be included in the report: • Table of Contents • List of Acronyms • Executive Summary – Concisely state the project purpose and background, key evaluation questions, methods, most salient findings, and recommendations (2–3 pages). • Introduction – Country context, including a summary of any relevant history, demography, socioeconomic status, etc. (1 page). USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 51 • The Development Problem and USAID’s Response – Brief overview of the development problem and USAID’s strategic response, including design and implementation of ECOFISH and any previous USAID projects implemented in response to the problem (2–3 pages). • Purpose of the Evaluation – Purpose, audience, and synopsis of task (1 page). • Evaluation Methodology – Describe evaluation methods, including strengths, constraints, and gaps (1 page). • Findings and Conclusions – Describe and analyze findings for each evaluation question using graphs, figures, and tables as applicable, and also include data quality and data sources, issues, and outcomes. Conclusions should be credible and should be supported by the findings (12–15 pages). • Recommendations – Prioritize for each evaluation question; should be separate from conclusions and supported by a clearly defined set of findings and conclusions. Include recommendations for future project implementation or relevant program designs, as well as synergies with other USAID projects and other donor interventions as appropriate (3–4 pages). • Lessons Learned – Provide a brief description of key technical and/or administrative lessons on what has worked, what has not worked, and why for future project or program designs (2–3 pages). • Annexes – Include statement of work, documents reviewed, bibliographical documentation, evaluation methods, data generated from the evaluation, tools used, interview lists, meetings, FGDs, surveys, and tables. The Evaluation Design Matrix must be presented as an annex to the report. Annexes should be succinct, pertinent, and readable. Annexes should also include, if necessary, a statement of significant unresolved differences of opinion by funders, implementers, or members of the evaluation team on any of the findings or recommendations. The report format should be restricted to Microsoft products and should use 12-point type font throughout the body of the report, with one-inch page margins at top/bottom and left/right. The final report will be edited and formatted by the evaluation team and provided to USAID/Bangladesh five working days after the Mission has reviewed the content and approved the final revised version of the report. A second, public version of the evaluation report, excluding any potentially procurement-sensitive or host country-sensitive information, will be submitted (also electronically, in English) to the Development Experience Clearinghouse (DEC) and disseminated among implementing partners and other stakeholders within 10 days following approval from USAID. All quantitative data, if gathered, should be (1) provided in an electronic file in easily readable format; (2) organized and fully documented for use by those not fully familiar with the project or the evaluation; (3) owned by USAID and made available to the public barring rare exceptions and excluding procurement￾sensitive information. A thumb drive with all the data could be provided to the ACME COR. LOGISTICS SUPPORT USAID/Bangladesh’s ACME project will be responsible for all off-shore and in-country administrative and logistical support, including identification and fielding appropriate local staff. They will take care of arranging and scheduling meetings, international and local travel, hotel bookings, working/office spaces, computers, printing, and photocopying. A local administrative assistant may be hired to arrange field visits, local travel, hotel, and appointments with stakeholders and to provide translation USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 52 ANNEX II: EVALUATION DESIGN MATRIX Evaluation Question Evaluation Sub-questions73 Data Source and Collection Method Illustrative Indicators Sampling/Selecting Criteria 1. To what extent is the ECOFISH activity on track, in terms of progress and outcomes, to meet its overall goals? What factors (internal/external) affected timely achievement of targets? And what factors impeded performance? 1.1 Timely completion of the project milestones? 1.2 Effective use of resources? 1.3 Reach/capture of pool of beneficiaries? 1.4 Quality of collaboration/partnerships among national, regional, and local governance structures? 1.5 Contribute to enhance marine and coastal biodiversity? 1.6 How has project contributed to the development objectives and goals of Government? 1.7 What strategies, policies, and best practices have been least effective and why? 1.8 How can the constraints to the success of these strategies be overcome? 1.9 Are working relationships effective? 1.10 How have communication activities supported implementation? ● Project documents, including schedule, work plan, field reports, file review, and desk research ● Surveys, interviews/ questionnaire results, and observation ● Government reports and documents ● Mini-surveys (scientists) ● Number of people trained in sustainable Natural Resources Management (NRM) and/or biodiversity conservation by USG assistance ● Number of hectares of biological significance and/or natural resources under improved NRM as a result of USG assistance ● Number of women benefiting from access to finance and savings in community savings and loan programs ● Percent increase in stakeholder engagement in co-management in the Sanctuary area ● Number of people complying with hilsa rules and regulation management actions due to improved delivery of the government compensation package ● Select and review all available project reports and documents ● Select stakeholders from each level for interviews (i.e., community, district, and national) ● Select and review relevant Bangladesh government annual reports/documents ● Select informant from WorldFish, other IPs, and USAID for KII 2. Given the time remaining, what opportunities exist to 2.1 What programs are in place to synergistically address all these issues? 2.2 Are mechanisms/support in place for long-term sustainability? ● Project documents, file review, and desk research ● Small-group ● Number and type of approaches addressing biodiversity, resource management, sustainable ● Review available relevant documents. ● Select focus group Members from fishers 73 Indicative questions to be asked of specific informants as appropriate (see Annex III). USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 53 enhance and strengthen programmatic effectiveness such as by adding, changing, rescaling and/or removing activities to meet or surpass targets/objectives, within the stipulations of the ECOFISH agreement? 2.3 Are there data collection/research programs in place to track long-term effectiveness? 2.4 Are there enough science-based improved hilsa fisheries management programs? If not, what should additional research focus on? 2.5 Are all the groups educated enough to carry out the co-management beyond the project? 2.6 How could DOF mainstream the project result (e.g., co-management, community resilience activities, etc.)? 2.7 Social benefits: how and who defined benefits? 2.8 What approaches have been attempted? 2.9 Are the relevant institutions/persons involved in concerned activities? 2.10 What are key research priorities to scaling-up science-based policy formulation? 2.11 How have the resilience elements/ criteria of fishing community from different locations been prioritized? discussions or focal group sessions ● Key Informant Interviews ● Mini-survey (villager FGDs) fisheries, and overfishing ● Level of engagement of stakeholders/ beneficiaries participating in programs ● Types of mechanisms (e.g., legal, financial, and social) in place to ensure sustainability ● Level of satisfaction of beneficiaries and local associations in four geographic areas. ● Select appropriate individual for KII from different strata. 3. Is there evidence that the activity has led to improved livelihoods (e.g., improved earnings) and greater resilience to shocks associated with climate change or other issues? 3.1 Have processes been established by the project to ensure the local fishing community can take responsibility for improved and sustainable livelihoods? 3.2 Have procedures been established by the project to transfer ownership to country-level agencies to implement the project? 3.3 What types of processes are being used to cope with shocks associated with climate change? 3.4 How has capacity building of women increased for additional earnings and resilience? 3.5 How is linkage made between capacity building and placement of women in ● Project documents, file review and desk research ● Small-group discussions or focal group discussions ● Key Informant Interviews ● Number of institutions with improved capacity to assess or address climate change risks supported by USG assistance ● Number of people with increased economic benefits derived from sustainable NRM and conservation as a result of USG assistance ● Number of households with improved well-being and diversified source of ● Select focus group members (including counterfactual focus group participants) from fishers and local associations ● Select representative supervisors, midlevel, and extension agents/ workers ● Select (and separately engage in FGDs) women leaders within communities and artisanal fisheries within USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 54 leadership and participation? 3.6 Are the livelihood assets identified for different fishing community? 3.7 Have the role of livelihood assets been prioritized for necessary interventions? 3.8 Is the appropriate fishing community selected as beneficiary? 3.9 Have gender, age, race/caste been considered in beneficiary selection and group formation? 3.10 How was the group leader/ representative selected? income (food security, other aspects of livelihood) each of the project areas 4. How effectively has ECOFISH advanced the co-management approach with the GOB, local communities, and other stakeholders? What evidence is there that this approach is successfully combating overfishing/improving sustainability of fisheries? 4.1 What mechanisms/programs are in place to synergistically address all these issues? 4.2 Are mechanisms in place to reduce overfishing for maximum sustainable yield? 4.3 Are there data collection/research programs in place to track long-term effectiveness of co-management? 4.4 What coordination mechanism(s) developed by the project for effective co-management are in place among GOB, fishing community, and other stakeholders? 4.5 How could DOF mainstream the co￾management to combat overfishing/ improve sustainability of marine and coastal fisheries? ● Project documents/file review and desk research ● Small-group discussions or focal group sessions ● Key Informant Interviews ● Number of hectares of biological significance and/or natural resources under improved NRM as a result of USG assistance ● Number of people trained in sustainable Natural Resources Management (NRM) and/or biodiversity conservation by USG assistance ● Number of people with increased economic benefits derived from sustainable NRM and conservation as a result of USG assistance ● Percent increase in stakeholder engagement in co￾management in the Sanctuary area ● Number of people complying with hilsa ● Select appropriate documents/report to review. ● Select focus group members from fishers and local associations in four geographic areas for discussion purposes. ● Select Informants from DOF/ECOFISH for KII. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 55 rules and regulation management actions due to improved delivery of the government compensation package 5. How effective have the project’s gender engagement activities been under IR 2 and IR 3? 5.1 What types of institutional arrangements have been developed for participation of women and sustainability? 5.3 Are additional regulations or policies needed to achieve better outcomes? 5.4 How effectively do women participate in co-management activities and why? 5.5 What is the extent of contribution of women with their additional income to their family and how? 5.6 What are the impacts of training on women for improved livelihoods and how? 5.7 What are the criteria for the selection of women as project beneficiaries? 5.8 How has the training of men impacted women in project sites (and vice versa)? ● Project documents, including schedules, work plans, field reports, and assessments ● Interviews and small-group discussions with stakeholders such as government, partners, and fishers ● Number of vulnerability assessments completed ● Number of women and men from fishing households that have received training on alternative livelihoods ● Number of households with improved well-being and diversified source of income (food security, other aspects of livelihood) ● Number of women benefiting from access to finance and savings in community savings and loan programs ● Number of women leaders in local organizations ● Number of cooperative agreements between key stakeholders ● Identification of Hilsa Savings Group members ● Select gender separated focus group members from fisher’s associations and Hilsa Savings Groups in each of the nine districts USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 56 6. Given the current state of over exploitation of coastal fisheries resources in Bangladesh, how can USAID best direct future resources, both strategically and programmatically, to implement the GOB National Strategy for conserving marine and coastal bio- diversity in Bangladesh? 6.1 What mechanisms are viewed as adequate to reduce overexploitation of coastal and marine fisheries resources? 6.2 Are there sufficiently functional strategies and policies for conserving marine and coastal biodiversity? 6.3 How could USAID contribute to directing future and effective strategies and policies? 6.4 Which fisheries species have bio￾communications for effective EBFM? 6.5 Where are the hotspots of key species breeding/nursing and feeding grounds? ● Review available documents ● Interviews with stakeholders ● Focus group discussions ● Written plans for long-term implementation of approach ● Financial resources allocated to each approach ● Number of laws, policies, regulations, or standards that address biodiversity conservation and/or other environmental themes officially proposed, adopted, or implemented as supported by USG assistance ● Findings from comparable studies supported by USAID and other donors in South and Southeast Asia ● Select individuals for KII using both project documents and recommendations from other KIIs ● Select KIIs and focus group members from opinion leaders and decision makers at USAID, GOB, and think tanks USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 57 ANNEX III: DATA COLLECTION INSTRUMENTS AND LIMITATIONS PART 1: FIELD SAMPLING SCHEDULE AND METHODS Sampling Frame The evaluation team visited six Upazilas (sub-districts) in five of the ECOFISH Activity districts as well as various district offices engaged in ECOFISH-Bangladesh, as indicated in Figure 7 below. Figure 7. ECOFISH Evaluation Fieldwork Itinerary74 These five districts were selected because, as indicated in Table 8 below, they represent the highest portion of project beneficiaries in all nine ECOFISH Districts and also represent a diverse range of project activities in different geographic regions (from coastal communities such as Kalapara to riverine communities such as Haimchar). 74 Drafted by evaluation team member, Dr. M. Shahadat Hossain. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 58 Table 8. Distribution of ECOFISH Activity Beneficiaries by District Beneficiaries by District # District % # Beneficiaries Barisal Division 1 Barguna 4 562 2 Patuakhali 13 1,970 3 Bhola 35 5,386 4 Barisal 12 1,843 5 Pirojpur 3 393 6 Jhalakhati 1 223 Dhaka Division 7 Shariatpur 5 833 Chittagong Division 8 Laxmipur 14 2,134 9 Chandpur 14 2,161 Total 100% 15,505 In each Upazila, a range of data gathering techniques was used, requiring the team to split into sub￾groups. Participants in each village/district office were invited to meet the evaluation team 24 hours in advance and logistical arrangements were coordinated between ECOFISH IPs and IBTCI/ACME staff supporting the evaluation team. Field Data Collection75 Three principal forms of field data gathering were undertaken during fieldwork to six Upazilas commencing on November 21, 2017: 1. Key Informant Interviews – KIIs were planned to be undertaken in ECOFISH Activity villages and with the Upazila- and district-level staff listed below during field visits as time permitted (estimated nUpazilaKIIs = 60); the actual participants in each KII are listed in Annex IV.. Activities Location Dates Day Time KII with Deputy Director (DD) and District Fisheries Officer (DFO), Barisal DD Office 21 Nov 17 Tuesday TBD KII with ECOFISH Barisal Focal Point Hotel 21 Nov 17 Tuesday TBD KII with Partner’s Focal Point Hotel 21 Nov 17 Tuesday TBD KII with Deputy Commissioner (DC) DC Office 22 Nov 17 Wednesday TBD KII with UFO, Hizla, Barisal UFO Office 22 Nov 17 Wednesday TBD KII with UAO/ULO, Hizla, Barisal UAO/ULO Office 22 Nov 17 Wednesday TBD 75 Includes the draft data collection schedule discussed with USAID/Bangladesh before commencing fieldwork. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 59 KII with UNO & USWO, Hizla, Barisal UNO Office 22 Nov 17 Wednesday TBD KII with Upazila Chairman Chairman Office 22 Nov 17 Wednesday TBD KII with Fishers Association Leader UFO Office 22 Nov 17 Wednesday TBD KII with Media Representative Hotel 22 Nov 17 Wednesday TBD KII with DFO, Patuakhali DFO Office, Patuakhali 23 Nov 17 Thursday TBD KII with DC, Patuakhali DC Office 23 Nov 17 Thursday TBD KII with Partner’s Focal Point DFO Office 23 Nov 17 Thursday TBD KII with ECOFISH, Patuakhali Focal Point DFO Office 23 Nov 17 Thursday TBD KII with Coast Guard Coast Guard Office 23 Nov 17 Thursday TBD KII with SUFO Kalapara SUFO Office 23 Nov 17 Thursday TBD KII with Fishers Association Leader SUFO Office 24 Nov 17 Friday TBD KII with Media Representative SUFO Office 24 Nov 17 Friday TBD KII with Aratder Landing Centre 24 Nov 17 Friday TBD KII with Local Moneylenders (Mohajan) Landing Centre 24 Nov 17 Friday TBD KII with DFO, Bhola DFO Office, Bhola 25 Nov 17 Saturday TBD KII with Partner’s Focal Point DFO Office, Bhola 25 Nov 17 Saturday TBD KII with DC/SP, Bhola DC/SP Office 25 Nov 17 Saturday TBD KII with Coast Guard Coast Guard Office 25 Nov 17 Saturday TBD KII with ECOFISH, Bhola Focal Point DFO Office 25 Nov 17 Saturday TBD KII with SUFO, Bhola Sadar DFO Office 25 Nov 17 Saturday TBD KII with Fishers Association Leader DFO Office 25 Nov 17 Saturday TBD KII with Media Representative Hotel 25 Nov 17 Saturday TBD KII with Aratder Landing Centre 26 Nov 17 Sunday TBD KII with Local Moneylenders (Mohajan) Landing Centre 26 Nov 17 Sunday TBD KII with UNO/USWO UNO Office 26 Nov 17 Sunday TBD KII with UAO/ULO, Hizla, Barisal UAO/ULO Office 26 Nov 17 Sunday TBD KII with Upazila Chairman Chairman Office 26 Nov 17 Sunday TBD KII with SUFO, Char Fassion, Bhola SUFO Office 27 Nov 17 Monday TBD KII with Fishers Association Leader DFO Office 27 Nov 17 Monday TBD USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 60 KII with Media Representative Hotel 27 Nov 17 Monday TBD KII with Aratder Landing Centre 27 Nov 17 Monday TBD KII with Local Moneylenders (Mohajan) Landing Centre 27 Nov 17 Monday TBD KII with UNO/USWO UNO Office 27 Nov 17 Monday TBD KII with Upazila Chairman Chairman Office 27 Nov 17 Monday TBD KII with DFO, Laxmipur DFO Office, Laxmipur 28 Nov 17 Monday TBD KII with Partner’s Focal Point DFO Office, Laxmipur 28 Nov 17 Monday TBD KII with ECOFISH, Focal Point DFO Office 28 Nov 17 Monday TBD KII with SUFO, Kamalnagar UFO Office, Kamalnagar 29 Nov 17 Tuesday TBD KII with Fishers Association Leader UFO Office 29 Nov 17 Tuesday TBD KII with Aratder Landing Centre 29 Nov 17 Tuesday TBD KII with Local Moneylenders (Mohajan) Landing Centre 29 Nov 17 Tuesday TBD KII with UNO/USWO UNO Office, Kamalnagar 29 Nov 17 Tuesday TBD KII with UAO/ULO, Kamalnagar UAO/ULO Office 29 Nov 17 Tuesday TBD KII with DFO, Chandpur DFO Office, Chandpur 30 Nov 17 Wednesday TBD KII with Partner’s Focal Point DFO Office, Chandpur 30 Nov 17 Wednesday TBD KII with ECOFISH, Focal Point DFO Office, Chandpur 30 Nov 17 Wednesday TBD KII with CSO & SSO, BFRI, Chandpur BFRI, Chandpur Office 30 Nov 17 Wednesday TBD KII with SUFO, Haimchar UFO Office, Haimchar 01 Dec 17 Thursday TBD KII with Fishers Association Leader UFO Office 01 Dec 17 Thursday TBD KII with Aratder Landing Centre 01 Dec 17 Thursday TBD KII with Local Moneylenders (Mohajan) Landing Centre 01 Dec 17 Thursday TBD KII with UNO/USWO UNO Office, Kamalnagar 01 Dec 17 Thursday TBD KII with River Police Police Station 01 Dec 17 Thursday TBD 2. Focal Group Discussions (FGDs) – The evaluation team planned for at least two FGDs76 in each Upazila rotating between the three project beneficiary groups (HCGs, CSGs, and HGGs), inviting between 6 and10 representatives from villages who participate in ECOFISH (nFGDs = 12; 76 An additional FGD was possible in some villages when travel logistics created additional opportunities. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 61 nFGDParticipants = 72–120); note that all CSG FGD meetings were restricted solely to female participants by virtue of the composition of the CSGs. District Upazila Group Date Time Barisal Hizla HCG 22 Nov 2017 TBD HSG 22 Nov 2017 TBD Patuakhali Kalapara HCG 24 Nov 2017 TBD HGG 24 Nov 2017 TBD Bhola Sadar HGG 26 Nov 2017 TBD HSG 26 Nov 2017 TBD Char Fassion HCG 27 Nov 2017 TBD HSG 27 Nov 2017 TBD Laxmipur Kamalnagar HCG 29 Nov 2017 TBD HSG 29 Nov 2017 TBD Chandpur Haimchar HCG 30 Nov 2017 TBD HGG 30 Nov 2017 TBD A complete list of all FGDs undertaken is provided in Annex IV, but names of participants are not listed to protect confidentiality commitments; a complete (and signed) list of all attendees is available from IBTCI. 3. Mini-Surveys – These were designed to be undertaken of 30 randomly selected participants (50 percent M/50 percent F) in two ECOFISH villages and in one non-ECOFISH village (n = 3) per Upazila (n = 6) in each survey. The proposed number of people to be surveyed was 540 (30 x 18). During fieldwork an additional six respondents were included in one of the villages. In several villages, it was not possible to interview the targeted number of females so enumerators added male respondents from the preselected random name list for that village. Additionally, in some villages there were insufficient respondents available – in those situations enumerators selected new respondents from backup randomly selected villages within the same Upazila. PART 2: KEY INFORMANT INTERVIEW (KII) PROTOCOL AND SURVEY 1. Preparation a. Invitations – Key informants have been pre-identified in the project work plan and interview assignments will be shared by the three evaluation team members. b. Interviews will be scheduled at the convenience of the informant and may be undertaken by phone/Skype or in person. c. Language – Most KIIs will be undertaken in English, but questions (and responses) will be translated into Bangla as needed. 2. Conduct USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 62 a. The interviewers will introduce themselves/the evaluation team and the purpose of the discussion and reassure participants of the confidentiality of the discussion. Notes will be for internal reference only and no individual will be identified in the project report. b. The question list for that discussion will include core questions and optional questions (if time permits). c. The interviewer/note taker will take notes of meeting (for later reference) directly on the questionnaire and will later summarize responses received in bullet point form. When feasible a note taker will also keep more detailed notes of the meeting. It is not uncommon for interviewees (especially senior officials) to seek to direct the conversation – make sure you keep them on track to answer at least the core questions. d. Before the end of the discussion, each participant will be invited to share any final comments about ECOFISH. Interviewers should probe to assess if the interviewee has any concerns about ECOFISH and the manner in which it has been implemented. e. Participants will be thanked and advised on what happens next with the evaluation process. 3. Analysis/Presentation of Findings a. Each KII discussion will be documented as a short meeting summary (1–3 pages, mostly handwritten). Reports will be scanned and compiled in a project resource folder for future reference as needed. Please try and make sure handwritten notes are legible! b. Interviews in Bangla will be recorded in either English or Bangla by the note taker/interviewer – if recorded in Bangla, they will be back-translated and checked by the interviewer to ensure accuracy. c. KII findings will be used both as inputs to the evaluation and as reference points for follow-up. d. Non-attributed quotes from select KIIs may be used in the report to emphasize or underscore key findings/observations. e. A summary of KII interview data will be tabulated by category of response (each project strengths, etc.), by category of respondent (government, IP, etc.), and by gender (m/f). USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 63 KEY INFORMANT INTERVIEW (KII) SURVEY QUESTIONS Introduction Hello, my name is ______________. I am a member of the team that is undertaking an independent midterm evaluation of the USAID ECOFISH-Bangladesh project. The evaluation is a standard process for all USAID projects and our findings will be used by USAID to guide the next two years of implementation of ECOFISH. Your comments to us today will be treated as confidential – no specific individual will be identified in the report we make to USAID. Our interview will take around XX minutes. Are you comfortable proceeding? Do you have any questions before we begin? Core questions 1. What is the nature of your engagement with the ECOFISH-Bangladesh project? 2. How long have you been involved with ECOFISH-Bangladesh? 3. Overall, what do you consider to be the major strengths of the ECOFISH-Bangladesh project? 4. Is there one specific project result or outcome that you consider to be particularly important? Why? 5. What do you consider to be the major weaknesses of the ECOFISH project? 6. Is there any activity under the ECOFISH project which is of particular concern to you? Why? 7. Thinking about the future of hilsa and the communities who depending on them: (a) What should be the priorities of the project over the next two years? (b) What is needed after the current ECOFISH project finishes in 2019? Ask tailored OPTIONAL questions from the Additional Specific question matrix (if time/appropriate) Conclude with 8. Is there anything else that you would like to bring to the attention of the evaluation project team that we have not covered so far? Thank you for participating (and explain evaluation process after this). USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 64 PART 3: FOCUS GROUP DISCUSSION INSTRUCTIONS AND INSTRUMENT 1. Preparation a. Invitations – Representatives from each group (HSG, HCG, and HGG) will be randomly preselected from the project beneficiary lists (not available for HGGs) and invited to the meeting point. To broaden coverage, we will invite representatives from nearby ECOFISH villages within each Upazila so that the groups share perspectives from across that Upazila. b. A venue will be booked for villages to meet and a note taker (Tasmih) and facilitator (Shahadat or Humayun) will lead the meeting; Dutton will observe (and interject questions if appropriate). c. Language – Questions will be translated into Bangla. d. Snacks – Snacks and travel reimbursements will be made available for participants. 2. Conduct a. The facilitator will introduce the team and purpose of the discussion and reassure participants of the confidentiality of the discussion. Notes will be for internal reference only and no individual will be identified in the project report. b. Participants will be invited to introduce themselves. c. The question list for that discussion will include core questions and optional questions (if time permits). We will seek to limit each meeting to around one hour; the observer will assist the facilitator as needed to track this and ensure the FGD moves through questions. d. The note taker will record the meeting (for later reference) and summarize responses received in bullet point form. e. Five minutes before the end of the discussion, each participant will be invited to share any final comments about ECOFISH. f. Participants will be thanked and provided reimbursement or travel costs for attending. 3. Analysis/Presentation of Findings a. Each FGD discussion will be reported as a short meeting summary (2–3 pages). b. Data will be tabulated by Upazila, villages participating, by group (HCG, HSG, HGG) and gender (m/f). c. Non-attributed quotes may be used to highlight specific points in the evaluation. d. Summaries of comments will be prepared as an annex to the evaluation report. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 65 SEMI-STRUCTURED GUIDING QUESTIONNAIRE FOR ECOFISH-BANGLADESH Focus Group Discussions (HCG, HSG, HGG) Instructions: 1. Introduce facilitators/evaluation team, explain purpose of meeting, confirm confidentiality of discussions, and explain process/time frame. 2. Thank participants for meeting us and invite participants to introduce themselves. Core Questions 1. What has been your role(s) in the ECOFISH project? 2. How did you get involved in ECOFISH and why? 3. Overall, what do you consider to be the major benefits of the ECOFISH project? 4. What has changed since ECOFISH began in your village? 5. Is there one specific project result to date that you consider to be particularly important? Why? 6. What do you consider to be the major weaknesses of the ECOFISH project? 7. Is there any activity under the ECOFISH project which is of concern to you? Why? 8. What do you believe the project should focus on in the next two years? 9. What needs to happen to sustain ECOFISH activities after the ECOFISH project finishes in 2019? Ask tailored OPTIONAL questions here (if time) 10. Is there anything else that you would like to share with the evaluation team about ECOFISH? Thank you (and explain next steps as appropriate). USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 66 PART 4: MINI-SURVEY OF PERCEPTIONS OF HILSA TRENDS: INSTRUCTIONS AND INSTRUMENT 1. Preparation a. Evaluation team will preselect two ECOFISH villages and one non-ECOFISH village in each Upazila visited and identify 30 respondents (50 percent male, 50 percent female); additional villages in each Upazila will be identified in the event that we could not locate sufficient respondents and additional respondents to be identified to select as needed. b. Language – Survey is to be translated into Bangla. c. Format – Survey is to be enlarged to A3 size (so there is room for comments). d. Materials – Pens/pencils will be made available for respondents to use in completing the survey. 2. Distribution a. In each Upazila – A meeting will be convened in each village of people who are not involved in the focus groups. b. Villagers will be briefed on why we are doing this survey and up to 30 people will be invited to participate in the survey. c. Survey forms and pens will be randomly distributed to men and women equally and an evaluation team member will be available to answer any questions. d. Each person who receives a survey will be asked to complete it then and we will give each person standard travel reimbursement. e. Villagers will be specifically instructed on completing the form and an evaluation team member will be on standby throughout the process to assist: ● They will be required to identify their name and gender for our reference purposes. Check that this is done. ● They will be asked to make their own judgments about the rating of each question. There is no standard answer for good or poor – they will have to use their own judgments and are free to add any additional comments they wish to make. f. The evaluation team will check all responses received and place them in a file by village. 3. Analysis/Presentation of Findings a. Each response received will be entered into an Excel spreadsheet (or R direct) for analysis. b. Data will be tabulated and classified by based on Upazila, villages (those in and out of project), and gender (m/f). c. Summary tabular and graphical analysis will be prepared based on Upazila, village, and gender d. If feasible, a statistical analysis of differences will be completed (likely using t tests). e. Summaries of comments provided will be translated and prepared as inputs, or an annex, to the survey report. f. A short report of the survey will be prepared and highlights incorporated into the evaluation report. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 67 Survey of Hilsa Trends (complete all three sections of the survey) Upazila/Village Name: __________________ Date: ____/November/2017 Your Name: ___________________________ Male Female (please circle) Before ECOFISH Began in This Area Very Good Good Fair Poo r Very Poor No Opinion Comments 1 How would you rate supply of hilsa before the ECOFISH activity? 2 How would you rate your income from hilsa fisheries before ECOFISH? 3 How would you rate your overall food security before ECOFISH? 4 How would you rate your access to economic opportunities before ECOFISH? 5 How would you rate women’s access to economic opportunities before ECOFISH? 6 How confident were you about the future of hilsa fisheries before ECOFISH? At the Present Time Very Good Good Fair Poo r Very Poor No Opinion Comments 1 How do you rate supply of hilsa now? 2 How do you rate your income from hilsa fisheries now? 3 How do you rate your overall food security now? 4 How do you rate your access to economic opportunities now? 5 How do you rate women’s access to economic opportunities before now? 6 How confident are you about the future of hilsa fisheries now? After ECOFISH Ends Very Good Good Fair Poo r Very Poor No Opinion Comments 1 How do you feel about the supply of hilsa after ECOFISH? 2 How do you feel about your income from hilsa fisheries after ECOFISH? 3 How do you feel about your overall food security after ECOFISH? 4 How do you feel about your access to economic opportunities after ECOFISH? 5 How do you feel about women’s access to economic opportunities after ECOFISH? 6 How confident are you about the future of hilsa fisheries after ECOFISH? Thank you for your comments – your inputs are confidential. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 68 Survey of Hilsa Trends (Bangla) USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 69 PART 4: SURVEY OF HILSA SCIENTISTS: INSTRUCTIONS AND INSTRUMENT ECOFISH-Bangladesh Midterm Evaluation Mini-Survey of Hilsa Scientists 1. Preparation a. Dr. Shahadat will be the focal survey lead and will be responsible for survey administration and compilation of results in collaboration with other evaluation team members. b. Sample selection – National scientists with expertise related to hilsa fisheries/EBFM in Bangladesh will be identified by the evaluation team (we expect to identify up to 30 scientists). c. Language – Most surveys will be transmitted in English, but questions (and responses) will be translated into Bangla as needed. 2. Conduct a. Self-administered email survey (attached). b. Surveys will be sent to the best available email address for each potential respondent with a receipt confirmation and a two-week response deadline. A reminder will be sent after one week if no response has been received and one last reminder will be sent the day the survey is due (allowing a further five days for response, after which the survey will cut off). 3. Analysis/Presentation of Findings a. Data provided from the surveys returned will be entered into an Excel spreadsheet. b. The survey data will be summarized graphically (pie, bar graphs, etc.). c. A summary report of the scientist survey will be prepared (as a possible annex to the evaluation report). d. We may be able to publish the survey findings as a short note in the Bangladesh Fisheries Journal. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 70 Scientist Mini-Survey Questions Introduction This survey is being undertaken as part of an evaluation of the USAID-Funded ECOFISH-Bangladesh Project. Under the ECOFISH-Bangladesh project, there is a specific focus on improved fisheries science for decision￾making and we would like to seek your inputs to that component of the project. Evaluations are a standard process for all USAID projects and our findings will be used by USAID to guide the future implementation of ECOFISH. Your comments will be treated as confidential – no specific individual will be identified in the report we make to USAID. Core questions 1. Which of the following aspects of Hilsa Fisheries Science do you have expertise in? (please circle all that apply) Biology Ecology Genetics Field Research Habitat Water Quality GIS Anthropology Economics Social Science Law Governance Finance Other (please describe): ______________________________ 2. Have you had any engagement with the ECOFISH-Bangladesh project? (Circle) Yes No If yes, please briefly describe and answer questions 3–5. If no, please go to question 6. 3. Overall, what do you consider to be the major strengths of the ECOFISH-Bangladesh project? 4. Are there any specific science-related activities underway via ECOFISH that you consider to be particularly important? Why? 5. What do you consider to be the major weaknesses of the ECOFISH activity? 6. What are the major research needs for the following (please answer questions related to your expertise and be very specific about research needs in these areas and the relative priority of such studies): ▪ Hilsa species biology, including genetics ▪ Hilsa habitat ecology, including habitat types and interaction with other species and trophic levels ▪ Hilsa breeding/aquaculture ▪ Hilsa fisheries economics ▪ Hilsa fisheries community development ▪ Hilsa anthropology USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 71 ▪ Hilsa governance ▪ Hilsa conservation management ▪ Hilsa law ▪ Interdisciplinary hilsa research 7. Who should be engaged in the priority hilsa fisheries research identified above? 8. Do you believe that there is sufficient expertise to undertake priority hilsa research in Bangladesh? Yes No (Circle) If you answered no, why do you have that view and who do you believe Bangladesh researchers should collaborate more closely with among the global fisheries research community? (Please be specific – name research/institutions.) 9. Do you have any suggestions for ways to improve coordination of hilsa fisheries research? 10. Do you have suggestions for how to improve the way research is communicated with key hilsa stakeholders? 11. Is there anything else about hilsa research that you would like to bring to the attention of the evaluation project team that we have not covered so far? 12. What is the name of your main employer/host institution? _____________________________________________________________ Thank you for participating in this survey – we will share a summary of the survey results with you. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 72 ANNEX IV: SOURCES OF INFORMATION Part 1: Evaluation Meeting and Interview Schedule Date Location Format77 Participant(s)/Affiliation 12-11-17 ECOFISH Office, Dhaka + Skype IEM 1. M. Millat-e-Mustafa, ACME 14-11-17 ECOFISH Office, Dhaka IEM 1. Safina Naznin, ECOFISH 2. Md. Jalilur Rahman, ECOFISH 3. Dr. Martin L. Van Brakel, ECOFISH 4. A.S.M Rashedul Haque, DOF 5. Md. Nahiduzzaman, ECOFISH 6. Dr. Md. Golam Mustafa, ECOFISH 16-11-17 Dept. of Fisheries, Dhaka KII 1. Dr. Syed Arif Azad, DG, DOF 16-11-17 BFRI, Dhaka KII 1. Dr. Masud H. Khan, Chief Scientist Officer, BFRI 18-11-17 ACME Office, Dhaka KII 1. Masood Siddique, CNRS 19-11-17 IUCN, Mohakhali DOHS, Dhaka IEM 1. Md. Shahad Mahbub Chowdhury, IUCN 2. Md. Fazlay Arafat, IUCN 3. Haseeb Irfanullah, IUCN 19-11-17 ACME Office, COAST Trust IEM 1. Sanat Bhowmik, Director, COAST Trust 20-11-17 USAID PRE 1. USAID staff 2. IBTCI-ACME staff 20-11-17 World Bank, Dhaka KII 3. Dr. Md. Istiak Sobhan, World Bank Group 21-11-17 DOF, Barisal KII 1. Md. Nahiduzzaman, DD, DOF 21-11-17 DOF, Barisal KII 1. Md. Sazder Rahman, DFO, DOF 21-11-17 DOF, Barisal KII 1. Balaram Mahalder, WorldFish 2. Tanzina Nazia, Research Assistant, WorldFish 21-11-17 ECOFISH Rest House, Barisal KII 1. Rafiqul Islam, Journalist, Kaler Kantha 21-11-17 ECOFISH Rest House, Barisal IEM 1. Al Mamun, CODEC 2. Shital Kumar Nath, CODEC 22-11-17 UP Office, Barjalia, Barisal KII 1. Pondeet Shahabuddin, Barjalia 2. Mohammad Ali, Barjalia 3. Al Mamun Talukder, CODEC 77 KII = Key Informant Interview, FGD = Focus Group Discussion (Note: individual attendees not identified to respect confidentiality commitments), IEM = Information Exchange Meeting, PRE = Presentation and Discussion of Evaluation Process/findings USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 73 22-11-17 Hijla, Barisal KII 1. Md. Altaf Mahmud, ULO Office, Hijla 2. Md. A. Salam, Upazila Livestock Office 22-11-17 DOF, Hijla, Barisal KII 1. Md. Abdul Bashar, SUFO, DOF 22-11-17 Hijla, Barisal FGD 8 people attended from HCG, Hijla. 22-11-17 Hijla, Barisal FGD 9 people attended from HSG, Hijla. 23-11-17 DOF, Patuakhali KII 1. Dr. Md. Abul Hasanat, DFO, DOF 23-11-17 DOF, Patuakhali KII 1. Md. Jimi Reza, WorldFish 2. Md. Sarwardi, WorldFish 23-11-17 DOF, Kalapara, Patuakhali KII 1. Md. Kamrul Islam, DOF 23-11-17 DOF, Kalapara, Patuakhali KII 1. Nasir Uddin Ahmed, Daily Prothom Alo 2. Jasim Parvez, Daily Kaler Kantha 3. Amal Mukherjee, Daily Jugantor 4. Mejbahuddin Mannu, Daily Janakantha 23-11-17 DOF, Kalapara, Patuakhali KII 1. Abdul Motaleb, Chairman 24-11-17 Poshchim Kuakata, Patuakhali FGD 9 people attended from HSG, Kuakata, Patuakhali 24-11-17 Pashurbunia, Kalapara, Patuakhali FGD 9 people attended from HCG, Pashurbunia, Patuakhali 25-11-17 Kalapara, Patuakhali KII 1. Md. Nasiruddin Mahmud, UP, Nilgonj 25-11-17 ECOFISH Office, Bhola KII 1. Angkur Md. Imteaz, WorldFish 2. Md. Aminul Islam, WorldFish 3. Md. Iftakharul Islam, WorldFish 25-11-17 DOF, Bhola KII 1. Md. Asaduzzaman, DOF 25-11-17 DOF, Bhola KII 1. Md. Nurul Islam, B.F.C.S.L 25-11-17 Boloram Shura, Bhola KII 1. M. Arsad, HGG 25-11-17 DOF, Bhola KII 1. Md. Rezaul Karim 25-11-17 The Papillon Hotel, Bhola IEM 1. Md. Billal Hossain, COAST Trust 25-11-17 The Papillon Hotel, Bhola IEM 1. Khokhan, COAST Trust 2. Jahirul Islam, COAST Trust 25-11-17 Nasir Majhi Ghat, Bhola KII 1. Md. Jakir Hossain, Dhonia UP 25-11-17 Nasir Majhi Ghat, Bhola KII 1. Md. Abdul Hai, Dhonia UP 25-11-17 The Papillon Hotel, Bhola KII 1. Hamidur Rahman, Janakantha and Maasranga TV 25-11-17 Boloram Shura, Bhola FGD 9 people attended from HSG, Boloram Shura 26-11-17 DC Office, Bhola KII 1. Md. Salim Uddin, DC, Govt. Bhola 26-11-17 Coast Guard, Bhola KII 1. LT Devayan, Coast Guard 2. Captain Mosayed Hossain, Coast Guard 26-11-17 Upazila Parishad, Bhola KII 1. Md. Eunus, Bhola Sadar Upazila USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 74 26-11-17 Dakkhin Digoldi, Bhola FGD 8 people attended from HCG, Dakkhin Digoldi, Bhola 27-11-17 Gachir Khal, Char Fassion, Bhola FGD 10 people attended from HSG, Gachir Khal, Nil Komol, Char Fassion 27-11-17 Char Fakira, Char Fassion, Bhola FGD 10 people attended from HCG, Char Fakira, Char Fassion 27-11-17 Khejur Gachia Ghat, Char Fassion, Bhola KII 1. Lokman, Secretary, Khejur Gachia Fish Arotdar Ghat 27-11-17 COAST Trust, Char Fassion, Bhola KII 1. Mozammel, Chairman 27-11-17 COAST Trust, Char Fassion, Bhola KII 1. Md. Kawsar, Aratdar+ Money Lender, Bangla Bazar Ghat, Char Fassion 27-11-17 UNO Office, Upazila Administrative Office, Char Fassion, Bhola KII 1. Mohammad Monowar Hossain, UNO, Upazila Administrative Office 27-11-17 Char Fassion, Bhola KII 1. Nahiduzzaman, Project Manager, WorldFish 27-11-17 SUFO Office, Char Fassion, Bhola KII 1. Ali Ahmed Akhond, SUFO, DOF 28-11-17 DOF, Lakshmipur PRE 1. S.M. Mohibullah, DFO, DOF 2. Mir Mohammad Ali, WorldFish 28-11-17 DOF, Lakshmipur KII 1. Shankar Bisshas, FC, CNRS 28-11-17 Motirhut, Mach Ghat, Kamalnogor, Lakshmipur FGD 10 people attended from HCG, Motirhut, Mach Ghat 29-11-17 SUFO Office, Kamal Nogor, Lakshmipur KII 1. Abdul Quddus, SUFO, DOF 29-11-17 Uttor Chor Jogobondhu, Komol Nogor, Lakshmipur FGD 8 people attended from HCG, Uttor Chor Jogobondhu, Komol Nogor. 30-11-17 Uttar Bagula, Haimchar, Chandpur KII 1. Md. Manik, Chor Bhairobi, Bajar Matshyajibi Committee 30-11-17 Uttar Bagula, Haimchar, Chandpur KII 1. Md. Jahangir Hossain, Paravet, CNRS 30-11-17 Neil Komol River Police, Haim Chor, Chandpur KII 1. Md. Shikdar Hasanuzzaman, SI, River police 30-11-17 SUFO Office, DOF, Haimchar, Chandpur KII 1. Md. Habibur Rahman Talukder, SUFO, DOF 30-11-17 DOF, Chandpur KII 1. Md. Shafiqur Rahman, DFO, DOF, Chandpur 30-11-17 BFRI, Chandpur KII 1. Md. Harunor Rashid, SSO, BFRI 2. Dr. Md. Robiul Awal Hossain, SSO, BFRI 3. Mohammad Ashraful Alam, SSO, BFRI 4. Md. Mehedi Hasan Pramanik, SO, BFRI 5. Md. Monjurul Hasan, SO, BFRI 6. Dr. Masud Hossain Khan, CSO, BFRI 30-11-17 Grand Hilsha, Chandpur KII 1. Kingkar Chandra Saha, Research Associate, WorldFish 30-11-17 Uttar Bagula, Haimchar, Chandpur FGD 9 people attended from CSG, Uttar Bagula, Haimchar, Chandpur 30-11-17 Kata Khal Mach Ghat, Chandpur FGD 10 people attended from HCG, Kata Khal Mach Ghat, Chandpur USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 75 2-12-17 WorldFish, Dhaka IEM 1. Md. Jalilur Rahman, Scientist, WorldFish 2. M.G. Mustafa, Scientist, WorldFish 3. Abdul Wahab, TL, WorldFish 4. Safina Naznin, Gender & Environmental Specialist, WorldFish 5. Israt Zahura, Training Specialist, WorldFish 6. Martin Van Brakel, Scientist, WorldFish 7. A.B.M. Mahfujul Hauque, M&E Specialist, WorldFish 8. D. Shahidul Islam, Training Associate, WorldFish 9. Md. Nahiduzzaman, Project Manager, WorldFish 10. Tilon Chandra Sarker, Messenger 11. Md. Mursed, 12. Dr. Malcom Dickson, CD, WorldFish 13. Md. Motaleb Hossian 14. Md. Hafijur Rahman, Finance Assistant, WorldFish 2-12-17 WorldFish, Dhaka KII 1. Md. Nahiduzzaman, WorldFish 2-12-17 WorldFish, Dhaka KII 2. Dr. Martin Van Brakel, WorldFish 2-12-17 WorldFish, Dhaka KII 1. Israt Zahura, WorldFish 2-12-17 WorldFish, Dhaka KII 1. M.G. Mustafa, WorldFish 2. Safina Naznin, WorldFish 2-12-17 WorldFish, Dhaka KII 1. Md. Jalilur Rahman, WorldFish 2-12-17 USAID/BD PRE 1. Renerio Acosta, USAID 2. Farzana Yasmeen, USAID 3. Tom Carson, IBTCI ACME 5-12-17 USA (phone) KII 4. Dr. Jim Tobey, ECOFISH Advisor, CRC/URI 6-12-17 + 12-12-17 USA (phone) KII 1. Mr. Nathan Sage, USAID 6-12-17 ACME Office, Dhaka (Skype) KII 1. Naseem Ahmed Aleem, DCoP, AIN, WorldFish 2. Masudur Rahman, TC, AIN, WorldFish 3. Dr. Rayhan Sarwer, MES, AIN, WorldFish 4. Mahbubur Rahman, PC, AIN, WorldFish 6-12-17 WEA, WINROCK Office, Dhaka KII 1. Zainab Akhter, CoP, WINROCK 2. Nilufar Sultana, DCoP, WINROCK 10-12-17 DOF, Dhaka KII 1. A.S.M. Rashedul Haque, Project Director, DOF 11-12-17 WorldFish, Dhaka KII 1. Dr. Malcom Dickson, CD, WorldFish 11-12-17 DOF, Chittagong (From ACME Office, Dhaka) KII (Phone) 1. Dr. M I Golder, Director, Marine & X-PD, ECOFISH 11-12-17 DOF, Chittagong KII (Phone) 1. Mr. Bazlur Rashid, DD, Chittagong & Former DD Barisal USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 76 (From ACME Office, Dhaka) USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 77 Part 2: Documents Reviewed The Evaluation Team reviewed the following documents to understand the activities undertaken by the ECOFISH Team and the broader context for Hilsa Conservation and Fisheries Management in Bangladesh generally: a. Government of Bangladesh Laws and Policies 1. National Fisheries Policy 1998 2. National Marine Fisheries Act 1983 3. Jalmahal Management Policy 2009 4. Hilsa Fisheries Management Action Plan (HFMAP) b. USAID Documents 1. USAID/Bangladesh Country Development Cooperation Strategy (2011–2016) 2. USAID/Bangladesh Enhanced Coastal Fisheries (ECOFISH) Activity. 3. Concept Paper (v 4.1 Apr 2014) c. Documents Authored by ECOFISH 1. ECOFISH (2014). USAID/Bangladesh Enhanced Coastal Fisheries (ECOFISH) Activity Concept Paper (v 4.1 Apr 2014), 20 pp, Dhaka. 2. ECOFISH (2015). Enhanced Coastal Fisheries (ECOFISHBD) Annual Work Plan Year 1 (January–December 2015), Submission February 28, 2015, 58 pp, Dhaka. 3. ECOFISH (2015). Enhanced Coastal Fisheries in Bangladesh (ECOFISHBD) Annual Work Plan Year 2 (October 2015–September 2016), Submission October 30, 2015, 55 pp, Dhaka. 4. ECOFISH (2016). Enhanced Coastal Fisheries in Bangladesh (ECOFISHBD) Annual Work Plan Year 3 (October 2016–September 2017), Submission September 30, 2016, 59 pp, Dhaka. 5. ECOFISH (2017). Enhanced Coastal Fisheries in Bangladesh (ECOFISH-Bangladesh) Monitoring, Evaluation and Learning Plan 2015–2019. 2nd Revised draft, 54 pp, Dhaka. 6. ECOFISH (2015). Enhanced Coastal Fisheries in Bangladesh (ECOFISHBD) Annual Report 1 (October 1, 2014–September 30, 2015), Submission October 31, 2015, 43 pp, Dhaka. 7. ECOFISH (2016). Enhanced Coastal Fisheries in Bangladesh (ECOFISHBD) Annual Report Year 2 (October 2015–September 2016), Submission October 30, 2016, 84 pp, Dhaka. 8. ECOFISH (2015). ECOFISHBD Communications Strategy 2015, 16 pp, Dhaka. 9. ECOFISH (2016). Pledge for Parity in the Fishing Households: Fishers Women Congress 2016. Conservation of Hilsa through Women Empowerment, 10 pp, Dhaka. 10. ECOFISH (2017). Enhanced Coastal Fisheries in Bangladesh (ECOFISHBD) Annual Report Year 3 (October 2016–September 2017), Submission October 31, 2017, 73 pp, Dhaka. 11. ECOFISH (2015). Enhanced Coastal Fisheries in Bangladesh (ECOFISHBD) Gender Strategy (Submission October 30, 2015, revised version), 23 pp, Dhaka. 12. ECOFISH (2017). Enhanced Coastal Fisheries in Bangladesh (ECOFISHBD) Fishers Women Savings Fund Management Guidelines. Published March 2017, 28 pp, Dhaka. 13. ECOFISH (2016). Enhanced Coastal Fisheries in Bangladesh (ECOFISHBD) Environmental Monitoring and Mitigation Plan (EMMP), Submission July 31, 2016, 49 pp, Dhaka. 14. ECOFISH (2015). Draft Report Last updated: 16 March 2015 Rapid Assessment of the Fishing Villages around Five Hilsa Sanctuaries in Bangladesh for USAID-DOF Supported ECOFISHBD Project. Submission March 2015, 54 pp, Dhaka. 15. iDE Bangladesh (2016). ECOFISHBD Hilsa Market Systems and Governance Strategy Report. December 23, 2015 (Revised January 13, 2016). 76 pp, iDE Bangladesh, Dhaka. 16. Islam, M.M. (2016). Background paper: Institutionalization of co-management arrangements in the Andharmanik River Sanctuary. 13 pp, Dhaka. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 78 17. Islam, Mohammad Mahmudul, Rumana Aktar, Md. Nahiduzzaman, Benoy Kumar Barman, Md. Abdul Wahab. (2017). Socio-economic and ecological perspectives around Hilsa shad sanctuaries in Bangladesh: Insights for co-management, draft MS in preparation. 18. IUCN (2017). Ecological and Socio-economic Assessment of Nijhum Dwip Seascape: A Framework for Designating and Managing a Potential Site for Marine Protected Area in Bangladesh. 309 pp, Dhaka. 19. Mohammed, E.Y., Ali, L., Ali, S., Hussein, B., Wahab, M.A. and Sage, N. (2016). Hilsa’s non￾consumptive value in Bangladesh Estimating the non-consumptive value of the Hilsa fishery in Bangladesh using the contingent valuation method. Working Paper November 2016, IIED-BCAS￾WorldFish, 20 pp, Dhaka. 20. PPRC (2017). Power and Incentives in Hilsa Fishery: A Power and Participation Research Centre Study. 58 pp, Dhaka. d. Additional ECOFISH Reference Material (not authored by ECOFISH IPs) 1. Pomeroy, R., P. Thompson, and C. Courtney (2016). Marine Tenure and Small-scale Fisheries: A Summary of the Bangladesh Experience and Recommendations for the Hilsa Fishery, Tetra Tech Report to USAID, Tetra Tech VT. e. Hilsa Reference Articles (cited in text) 1. Bladon, A.J., A. Syed, S.M. Hassan et al. (2016). Finding evidence for the impact of incentive￾based hilsa fishery management in Bangladesh: combining theory testing and remote sensing methods. IIED Working Paper. IIED, London. [online] URL: http://pubs.iied.org/16625IIED/ 2. DOF (2014). Fishery Statistical Yearbook of Bangladesh, Department of Fisheries, Dhaka, Bangladesh. 3. Hossain, M.S., M.F. Rahman, S. Thompson, M.R. Nabi and M.M. Kibria (2013). Climate Change Resilience Assessment Using Livelihood Assets of Coastal Fishing Communities in Nijhum Dwip, Bangladesh, Pertanika Sci. and Technol. 21(2):397–422. 4. Rahman, M.J, Rahman M.A. and Bhaumik U. (2012). Biology and ecology of Hilsa shad, Tenualosa ilisha (Ham.) In: Hilsa: Status of Fishery and Potential for Aquaculture. Proceedings of the regional workshop held in Dhaka 16–17 September 2012. The WorldFish Centre, Bangladesh and South Asia Office, Dhaka. 5. Rahman, MJ, M.A. Wahab1, S.M. Nurul Amin, and M. Nahiduzzaman (2017). Stock Status, Maximum Sustainable Yield (MSY), and Management of Hilsa Shad (Tenualosa Ilisha), manuscript in press. 6. van Brakel, Martin. Md. Nahiduzzaman, A.B.M Haque, Md. Mustafa, Md. Rahman, Md. Wahab (2018), Re-imagining large-scale open water fisheries governance through adaptive co￾management in Hilsa shad sanctuaries, Ecology and Society - ES-2017-9917 (Version 3 of ES-2017- 9565), in press. 7. Wahab, M.A., Phillips, M.J., Mohammad, E.Y. (2013). Payments for Hilsa fish (Tenualosa ilisha) conservation in Bangladesh, 170–189. In: Economic Incentives for Marine and Coastal Conservation: Prospects, Challenges and Policy Implications. (Mohammed, E.Y., ed.). New York: Routledge. Hilsa Reference Articles Reviewed 1. Ahmed, M.S., Sharif, A.S.M., & Latifa, G.A. (2008). Age, growth and mortality of Hilsa shad, Tenualosa ilisha in the River Meghna, Bangladesh. Asian Journal of Biological Science 1:69–76. 2. Ahsan, Dewan Ali, M. Niamul Naser, Utpal Bhaumik, Sugata Hazra, & Subhra Bikash Bhattacharya (2014). Migration, Spawning Patterns and Conservation of Hilsa Shad (Tenualosa ilisha) in Bangladesh and India, Academic Foundation and IUCN, New Delhi. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 79 3. Ahsanullah, M. (1964). Population dynamics of Hilsa in East Pakistan. Agriculture Pakistan 15:351– 365. 4. Alam, A. (2001). Comparison of the parasite fauna of Tenualosa ilisha from different locations in Bangladesh. In: International Terubok Conference Proceedings (S.J.M. Blaber, D.J. Brewer, D.A. Milton, & C. Biano, eds.), Sarawak Development Corporation, Kuching, Malaysia, 124–138. 5. Alam, A.K.M.N., Mohanty, B.P., Hoq, M.E., & Thilshed, S. (2012). Nutritional values, consumption and utilization of Hilsa Tenualosa ilisha (Hamilton 1822). Proceedings of regional workshop on Hilsa: potential for aquaculture. 16–17 September 2012. Dhaka, Bangladesh, 40 pp. 6. Amin, S.M.N., Haldar, G.C., Rahman, M.A., Mazid, M.A., & Milton, D.A. (2004). Stock assessment and management of Tenualosa ilisha in Bangladesh. Asian Fisheries Science 17:51–59. 7. Amin, S.M.N., Rahman, M.A., Haldar, G.C., Mazid, M.A., & Milton, D.A. (2002). Population dynamics and stock assessment of Hilsa shad, Tenualosa ilisha in Bangladesh. Asian Fisheries Science 15:123–128. 8. Amin, S.M.N., Rahman, M.A., Haldar, G.C., Nahar, S., Dewan, S., & Mazid, M.A. (2000). Population Dynamics of Jatka (Jatka Hilsa) in the Meghna River, Bangladesh. Asian Fisheries Science 13:383–389. 9. Bhaumik, U. (2015). Migration of Hilsa Shad in the Indo-Pacific Region – A Review. International Journal of Current Research Academic Review 3(11):139–155. 10. Bhaumik, U., & Sharma, A.P. (2012). Present status of Hilsa in Hooghly-Bhagirathi River. Central Inland Fisheries Research Institute (CIFRI), Barrackpore, India, 47 pp. 11. Bhaumik, U., & Sharma, A.P. (2011). The fishery of Indian Shad (Tenualosa ilisha) in the Bhagirathi-Hooghly river system. Fishing Chimes 31:21–27. 12. Blaber, S.J.M. (2000). Tropical estuarine fishes: ecology, exploitation and conservation. Blackwell Science, Oxford. 13. Blaber, S.J.M., Farmer, M.J., & Milton, D.A. (1997). The ichthyoplankton of selected estuaries in Sarawak and Sabah: composition, distribution and habitat affinities. Estuarine, Coastal and Shelf Science 45: 197–208. 14. Blaber, S.J.M., Milton, D.A., Brewer, D.T., & Salini, J.P. (2001). The shads (genus Tenualosa) of tropical Asia: An overview of their biology, status and fisheries. In: International Terubok Conference Proceedings, Sarawak Development Institute (SDI), Kuching, Sarawak, Malaysia, 9– 17. 15. Blaber, S.J.M., Milton, D.A., Chenery, S.R., & Fry, G. (2003a). New insights into the life history of Tenualosa ilisha and fishery implications. In: Biodiversity, Status, and Conservation of the World’s Shads (eds. K.E. Limburg and J.R. Waldman), American Fisheries Society Symposium 35, Bethesda, MD, USA, 223–240. 16. Blaber, S.J.M., Milton, D.A., Brewer, D.T., & Salini, J.P. (2003b). Biology, fisheries and status of tropical shads (Tenualosa) in South and South East Asia. In: Biodiversity, status, and conservation of the world’s shads (K.E. Limburg & J.R. Waldman, eds.), American Fisheries Society Symposium 35, Bethesda, MD, USA, 49–58. 17. BOBLME (Bay of Bengal Large Marine Ecosystem). (2010). Status of Hilsa (Tenualosa ilisha) management in the Bay of Bengal. BOBLME-2010-Ecology-01. 18. BOBP (Bay of Bengal Program). (1985). A review of the biology and fisheries of Hilsa ilisha in the upper Bay of Bengal. Marine Fisheries Resources Management in the Bay of Bengal, Colombo, Sri Lanka, BOBP/WP/37, 58 pp. 19. Das, N.N. (1985). Factors affecting timing and size of runs of Hilsa Shad (Hilsa ilisha) in Bangladesh and Pakistan. Masters of Science thesis, The University of British Columbia, Vancouver, Canada, 118 pp. 20. Dewhurst-Richman, N., Mohammed, E.Y., Ali, M.L., Hassan, K., Wahab, M.A., Ahmed, Z.F., Islam, M.M., Bladon, A., Haldar, G.C., Ahmed, C.S., Majumder, M.K., Hossain, M.M., Rahman, A., & Hussein, B. (2016). Balancing carrots and sticks: incentives for sustainable Hilsa fishery USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 80 management in Bangladesh. London: International Institute for Environment and Development, 72 pp. 21. DOF (Department of Fisheries). (2008). Hilsa fisheries conservation, development and management technique (in Bengali). Department of Fisheries, Matshya Bhaban, Dhaka, Bangladesh, 42 pp. 22. Dutta, S.D., Maity, S., Bhattacharyya, S.B., Sundaray, J.K., & Hazra, S. (2014). Diet composition and intensity of feeding of Tenualosa ilisha (Hamilton, 1822) occurring in the Northern Bay of Bengal, India. Proceedings of the Zoological Society 67:33–37. 23. Dutta, S.D., Maity, S., Chanda, A., & Hazra, A. (2012). Population structure, mortality rate and exploitation rate of Hilsa shad (Tenualosa ilisha) in West Bengal coast of Northern Bay of Bengal, India. World Journal of Fish and Marine Sciences 4:54–59. 24. FAO (Food and Agricultural Organization). (2017). FishStat J Database, FAO, Rome, http://www.fao.org/fishery/statistics/software/fishstatj/en. 25. Faruque, M.D.H., & Ahsan, D.A. (2014). Socio-economic status of the Hilsa (Tenualosa ilisha) fishermen of Padma River, Bangladesh. World Applied Sciences Journal 32:857–864. 26. Habib, A.B.M.Z. (2014). Determination and implementation of action plan for development and sustainable management of national fish Hilsa. Paper presented (in Bengali) at Planning Commission workshop, Dhaka, Bangladesh. 27. Halder, G.C. (2002). Hilsa fishery management action plan for Bangladesh. Completion report of the studies conducted under the ARDMCS, GEF component; and FFP. Report No. 38.9, Department of Fisheries, Dhaka, Bangladesh. 28. Halder, G.C. (2004). Present status of the Hilsa fishery in Bangladesh. Completion report of the studies conducted under the ARDMCS, GEF component; and FFP. Report No. 38.8, Department of Fisheries, Dhaka, Bangladesh. 29. Haldar, G.C., & Amin, S.M.N. (2005). Population dynamics of male and female Hilsa, Tenualosa ilisha of Bangladesh. Pakistan Journal of Biological Science 8, 307–313. 30. Haldar, G.C., Mazid, M.A., Rahman, M.A., & Amin, S.M.N. (2001). The present status of Hilsa (Tenualosa ilisha) fishery in Bangladesh. In: International Terubok Conference Proceedings (eds. S.J.M. Blaber, D.J. Brewer, D.A. Milton and C. Biano), Sarawak Development Corporation, Kuching, Malaysia, 57–66. 31. Haroon, Y. (1998). Hilsa shad: Fish for the teeming millions: New management alternatives needed for the Hilsa young. Shad Journal 3:7–12. 32. Hasan, K.M.M., Ahmed, Z.F., Wahab, M.A., & Mohammed, E.Y. (2016). Food and feeding ecology of Hilsa (Tenualosa ilisha) in Bangladesh’s Meghna River basin. IIED Working Paper. IIED, London, 20 pp. 33. Hossain, M.M. (1985). Spawning times, early history of Hilsa ilisha in Bangladesh waters. Master’s thesis, University of British Columbia, Canada, 90 pp. 34. Hossain, M.J., & Hossain, M.S. (2016). Hilsa shad mariculture site suitability modeling in the maritime zone and adjacent river ecosystems of Bangladesh. Master’s thesis, Institute of Marine Sciences and Fisheries, University of Chittagong, Bangladesh, 122 pp. 35. Hossain, M.S., Sarker, S., Sharifuzzaman, S.M., & Chowdhury, S.R. (2014). Discovering Spawning Ground of Hilsa Shad (Tenualosa ilisha) in the coastal waters of Bangladesh. Ecological Modelling 282:59–68. 36. Hossain, M.S., Sarker, S., Sharifuzzaman, S.M., & Chowdhury, S.R. (2014). Habitat modelling of juvenile Hilsa Tenualosa ilisha (Clupeiformes) in the coastal ecosystem of the Northern Bay of Bengal, Bangladesh. Journal of Ichthyology 54:203–213. 37. Hossain, M.S., Sharifuzzaman, S.M., Chowdhury, S.R., & Sarker, S. (2016). Habitats across the lifecycle of Hilsa shad (Tenualosa ilisha) in aquatic ecosystem of Bangladesh. Fisheries Management and Ecology 23:450–462. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 81 38. Hossain, M.S., Chowdhury, S.R., & Sharifuzzaman, S.M. (2017). Blue Economic Development in Bangladesh: A policy guide for marine fisheries and aquaculture. Institute of Marine Sciences and Fisheries, University of Chittagong, Bangladesh, 32 pp. 39. Huq, K.A., Ferdous, K.S., Basak, B.K., & Ali, M.Y. (2006). Status of Hilsa fishery in the Meghna River adjacent to Bhola district. Bangladesh Journal of Life Science 18:97–105. 40. Islam, M.S., Huq, Q.M., Hossain, M., Azad, S.A., & Das, N.N. (1986). Maturity and spawning of Hilsa shad, Hilsa ilisha of Bangladesh. Hilsa investigation in Bangladesh, Marine Fishery Resources Management, BOBP/REP36, RAS/81/051:81–95. 41. Islam, M.M., Mohammed, E.Y., & Ali, L. (2016). Economic incentives for sustainable Hilsa fishing in Bangladesh: An analysis of the legal and institutional framework. Marine Policy 68:8–22. 42. Islam, M.M., Shamsuzzaman, M.M., Sunny, A.R., & Islam, N. (2016b). Understanding fishery conflicts in Hilsa sanctuaries of Bangladesh. In: Inter-sectoral governance of inland fisheries (A.M. Song, S.D. Bower, P. Onyango, S.J. Cooke, & R. Chunepagdee, eds.). St. John’s, NL, Canada: TBTI Publication Series. 43. Miah, M.S. (2015). Climatic and anthropogenic factors changing spawning pattern and production zone of Hilsa fishery in the Bay of Bengal. Weather and Climate Extremes 7:109–115. 44. Miah, M.H., Rahman, M.A., & Haldar, G.C. (1999). Analytical approach to the spawning ground of Hilsa Tenualosa ilisha (Ham.) in Bangladesh water. The Indian Journal of Animal Sciences 69:141– 144. 45. Milton, D.A. (2009). Living in Two Worlds: Diadromous fishes and factors affecting population connectivity between tropical rivers and coasts. In: Ecological Connectivity among Tropical Coastal Ecosystems (I. Nagelkerken, ed.). Dordrecht, Netherlands: Springer Science+Business Media B.V., 325–355. 46. Milton, D.A., & Chenery, S.R. (2003). Movement patterns of the tropical shad (Tenualosa ilisha) inferred from transects of 87Sr/86Sr isotope ratios in their otoliths. Canadian Journal of Fisheries and Aquatic Science 60:1376–1385. 47. Mome, M.A. (2007). The potential of the artisanal Hilsa fishery in Bangladesh: an economically efficient fisheries policy, Master’s Thesis, Department of Economics, University of Iceland, 57 pp. 48. Mome, M.A., & Arnason, R. (2008). The artisanal Hilsa fishery: an economically efficient fisheries policy. IIFET Vietnam Proceedings, 12 pp. 49. Murshed-e-Jahan, K., Belton, B., & Viswanathan, K.K. (2014). Communication strategies for managing coastal fisheries conflicts in Bangladesh. Ocean and Coastal Management 92:65–73. 50. Pomeroy, R., Garces, L., Pido, M., & Silvestre, G. (2010). Ecosystem-based fisheries Management in small-scale tropical fisheries: Emerging models of governance arrangements in the Philippines. Marine Policy 34:298–308. 51. Quddus, M.M.A. (1982). Two types of Hilsa ilisha and their population biology from Bangladesh waters. PhD Thesis. University of Tokyo, Japan, 180 pp. 52. Rahman, M.J., Mustafa, M.G., & Rahman, M.A. (1998). Population dynamics and recruitment pattern of Hilsa, Tenualosa ilisha. Proceedings of BFRI/ACIAR/CSIRO 6:28–36. 53. Rahman, M.J. (2001). Population biology and management of Hilsa shad (Tenualosa ilisha) in Bangladesh. PhD Thesis, University of Hull, England, 253 pp. 54. Rahman, M.J., & Cowx, I.G. (2006). Lunar periodicity in growth increment formation in otoliths of Hilsa shad (Tenualosa ilisha, Clupeidae) in Bangladesh waters. Fisheries Research 81, 342–344. 55. Rahman, M.J., & Cowx, I.G. (2008). Population dynamics of Hilsa shad, (Tenualosa ilisha, Clupeidae) in Bangladesh Waters. Asian Fisheries Science 21, 85–100. 56. Rahman, M.A., Amin, S.M.N., Haldar, G.C., & Mazid, M.A. (2000). Population dynamics of Tenualosa ilisha of Bangladesh water. Pakistan Journal of Biological Sciences 3:564–567. 57. Saifullah, A.S.M., Rahman, M.S., & Khan, Y.S.A. (2004). Fecundity of Hilsa ilisha (Hamilton, 1822) from the Bay of Bengal. Pakistan Journal of Biological Sciences 7:1394–1398. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 82 58. Salini, J.P., Milton, D.A., Rahman, M.J., & Hussain, M.G. (2003). Allozyme and morphological variation throughout the geographic range of the tropical shad, Tenualosa ilisha. Fisheries Research 66:53–69. 59. Shafi, M., Quddus, M.M.A., & Islam, N. (1977). Studies of gonad weight, sex-ratio and fecundity of Hilsa ilisha (Hamilton-Buchanan) from the river Meghna. Journal of Asiatic Society of Bangladesh (Science) 2:51–58. 60. Shafi, M., Quddus, M.M.A., & Islam, N. (1978). Maturation and spawning of Hilsa ilisha (Hamilton￾Buchanan) of the River Meghna. Dacca University Studies Series B 26:63–71. 61. Siddique, M.A.L. (2009). Conservation of juvenile hilsa (jatka) in Bangladesh: Need to address the livelihood of fishers. American Fisheries Society Symposium 69:757–768. f. Relevant Global Fisheries and Community Resilience Reference Materials (cited in text) 1. Allison, Edward H., Allison L. Perry, Marie-Caroline Badjeck, W. Neil Adger, Katrina Brown, Declan Conway, Ashley S. Halls, Graham M. Pilling, John D. Reynolds, Neil L. Andrew, & Nicholas K. Dulvy. (2009). Vulnerability of national economies to the impacts of climate change on fisheries, Fish and Fisheries, 1–24; DOI: 10.1111/j.1467-2979.2008.00310.x 2. Branch, T., & D. Kleiber (2017). Should we call them fishers or fishermen? Fish and Fisheries 18(1), 114–127, http://onlinelibrary.wiley.com/doi/10.1111/faf.12130/abstract 3. Hotta, K., & I.M. Dutton (eds.). (1995). Coastal Management in the Asia-Pacific Region, JIMSTEF, Tokyo. 421 pp. 4. Hulsenbeck, M. (2012). Ocean-based Food Security Threatened in a High CO2 World, Oceana, Washington D.C. 5. Lauck, T., C.W. Clark, M. Mangel, & G.R. Munro. (1998). Implementing the Precautionary Principle in Fisheries management Through Marine Reserves, Ecological Applications 8(1):572– 578. http://onlinelibrary.wiley.com/doi/10.1890/1051- 0761(1998)8%5BS72:ITPPIF%5D2.0.CO;2/full 6. Olsen, S.B. (ed.). (2003). Crafting Coastal Governance in a Changing World, CRC/USAID Coastal Management Report #2241, University of Rhode Island, Narragansett. 7. Osgood, K. (2014). Ecosystem-based Fisheries management: Council Member Training, NOAA Fisheries. http://www.nmfs.noaa.gov/sfa/management/councils/training/2014/o_p1_ecosystem_based_fisher ies_management.pdf 8. Parks, John E., Ghislaine Llewellyn, Ian M. Dutton, & Robert S. Pomeroy. (2003). Building the Capacity of Those Who Do the Work. In: Best B.A. (R.S. Pomeroy & C. Balboa, eds.). Implications for Coral Reef Management and Policy: Relevant Findings from the 9th International Coral Reef Symposium, A publication of the U.S. Agency for International Development, in collaboration with, the World Resources Institute, Conservation International, and the International Society for Reef Studies, 66–7. 9. Staples, D., Brainard, R., Capezzuoli, S., Funge-Smith, S., Grose, C., Heenan, A., Hermes, R., Maurin, P., Moews, M., O’Brien, C. & Pomeroy, R. (2014). Essential EAFM. Ecosystem Approach to Fisheries Management Training Course. Volume 1–For Trainees. FAO Regional Office for Asia and the Pacific, Bangkok, Thailand, RAP Publication 2014/13, 318 pp. 10. Witek, C. (2017). Managing Fisheries Management Uncertainty, Marine Fish Conservation Network: http://conservefish.org/2017/02/01/managing-fisheries-management-uncertainty/ g. Relevant Global Fisheries and Community Resilience References (Not all cited in text) 1. Abdullah, S. (2014). Local community resilience in the context of global climate change: A case from Maluku Indonesia. Sociol Anthr 2(7):309–16. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 83 2. Agrawala, S., Ota, T., Ahmed, A.U., Smith, J., & Aalst, M.V. (2003). Development and Climate Change in Bangladesh: Focus on Coastal Flooding and the Sundarbans. Organisation for Economic Co-operation and Development (OECD), Paris. 3. Alam, E., & Collins, A.E. (2010). Cyclone disaster vulnerability and response experiences in coastal Bangladesh. Disasters 34(4):931–54. 4. Badjeck, M.C., Allison, E.H., Halls, A.S., & Dulvy, N.K. (2010). Impacts of climate variability and change on fishery-based livelihoods. Marine Policy 34:375–83. 5. Brenkert, A.L., & Malone, E.L. (2005). Modeling vulnerability and resilience to climate change: a case study of India and Indian states. Climate Change 72:57–102. 6. Chambers, R. (1980). Rapid rural appraisal: rationale and repertoire. Discussion paper, Institute of Development Studies, Sussex. 7. Chowdhury, M.S.N., Hossain, M.S., Das, N.G., & Barua P. (2011). Environmental variables and fisheries diversity of the Naaf River Estuary, Bangladesh. Journal of Coastal Conservation 15 (1):163–80. 8. Chowdhury, S.R., Hossain, M.S., Shamsuddoha, M., & Khan, S.M.M.H. (2012). Coastal Fishers’ Livelihood in Peril: Sea Surface Temperature and Tropical Cyclones in Bangladesh. Foreign and Commonwealth Office through British High Commission and Centre for Participatory Research and Development (CPRD), Dhaka, Bangladesh. 9. de Silva, D.A.M., & Yamao, M. (2007). Effects of the tsunami on fisheries and coastal livelihood: a case study of tsunami-ravaged southern Sri Lanka. Disasters 31(4):386–404. 10. Deb, A.K., & Haque, A.E. (2011). Sufferings start from the mothers’ womb: Vulnerabilities and livelihood war of the small-scale fishers of Bangladesh. Sustainability 3(12):2500–27. 11. DFID (1999). Sustainable livelihoods guidance sheets, Department for International Development, London, UK. 12. Gibbs, M.T. (2009). Resilience: What is it and what does it mean for marine policymakers? Mar Policy 33:322–31. 13. Guleria, S., & Edward, J.K.P. (2012). Coastal community resilience: analysis of resilient elements in 3 districts of Tamil Nadu State, India. Journal of Coastal Conservation 16(1):101–10. 14. Heltberg, R., Siegel, P.B., & Jorgensen, S.L. (2009). Addressing human vulnerability to climate change: Toward a ‘no-regrets’ approach. Global Environmental Change 19:89–99. 15. Holling, C.S. (1973). Resilience and stability of ecosystems. Ann Rev Ecol System 4:1–23. 16. Hossain, M.S. (2013). Conserving Mangrove Ecosystem for Climate Change Adaptation in the Ganges Basin. In: Global challenges in integrated coastal zone management. (Moksness E., Dahl E. Stottup J., eds.), Wiley-Blackwell, UK; 85–100. 17. Hossain, M.S., Chowdhury, S.A., & Chowdhury, M.A.T. (2007). Integrated Remote Sensing, GIS and Participatory Approach to Study Coastal Island Resources of Bangladesh. Songklanakarin J Soc Sci Humanities 13(3):413–33. 18. Perry, R.I., & Sumaila, U.R. (2007). Marine ecosystem variability and human community responses: The example of Ghana, West Africa. Marine Policy 31(2):125–34. 19. Perz, S.G., Shenkin, A., Barnes, G., Cabrera, L., Carvalho, L.A., & Castillo, J. (2011). Connectivity and Resilience: A Multidimensional Analysis of Infrastructure Impacts in the Southwestern Amazon. Soc Indic Res 106(2):259–85. 20. Pimm, S.L. (1984). The complexity and stability of ecosystems. Nature 307:321–26. 21. Sethi, S.A. (2010). Risk management for fisheries. Fish and Fisheries 11:341–65. 22. US-IOTWS (2007). How Resilient Is Your Coastal Community? A Guide for Evaluating Coastal Community Resilience to Tsunamis and Other Coastal Hazards. U.S. Indian Ocean Tsunami Warning System Program supported by the United States Agency for International Development and partners, Bangkok, Thailand. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 84 Annex V: PERFORMANCE INDICATORS ANALYSIS Measuring Progress on ECOFISH Outcomes The evaluation team explored linkages between project inputs and outcomes by undertaking a detailed comparison of targets and actual results of the ECOFISH program based on the 15 indicators outlined in the ECOFISH Monitoring Evaluation and Learning Plan78 and the ECOFISH Results Framework (Figure 4). This comparison proved more challenging than expected for several reasons. As Table 2 indicates, ECOFISH has not always reported progress in the same format or used the same indicators year-to-year. The project also employs a mix of indicator- and activity-based reporting. For example, in the ECOFISH Year 1 Annual Report (October 1, 2014–September 30, 2015), ECOFISH introduced an activity-based reporting system that described progress on project activities with no specific cross-reference to the project indicators. In the Year 2 Annual Report (October 1, 2015–September 30, 2016) progress was reported on only six “standard indicators” with no reporting of progress on custom indicators outlined in the MEL plan. In year 3, ECOFISH staff expanded reporting on the full suite of indicators, but as Table 2 shows, even that report was incomplete. This observation underpins the observation that many ECOFISH interventions are relatively recent, thus limiting performance measurement. A final complicating factor is the way project indicators are explained in the ECOFISH Monitoring, Evaluation, and Learning (MEL) Plan (revised in April 2017). In the MEL Plan each indicator is loosely affiliated with a hypothesis related to each IR and a research question related to that hypothesis. Some of these hypotheses are testable (e.g., IR1 H1 – “If compliance with sanctuary management is improved then hilsa abundance will increase”; the catch per unit effort (CPUE) indicator is appropriate for that test). However, other hypotheses and research questions (such as IR1 H3 question 3.1 “How many science outputs would be better used to formulate better hilsa fisheries management plan?”) are less integrated and testable and hence contribute to unclear targets and indicators.79 ECOFISH has limited baseline data available for many of the indicators and there are some gaps in the scope of indicators that merit further attention. For example, given the emphasis of the project on sanctuaries as the core geographic focus for hilsa conservation, a measurement of the efficacy of different sanctuary types would seem appropriate, particularly give than ECOFISH interventions in most sanctuaries only engage communities on one side of the river sanctuary. 78 ECOFISH-Bangladesh (2015) Monitoring Evaluation and Learning Plan 2015–2019: Revised draft April 19, 2017. 79 For example, there is no clear explanation of or metadata about how targets such as “10 clauses included in the revised Hilsa Management Plan” were established. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 85 Table 9. Reporting of USAID-ECOFISH Activity Indicators ECOFISH Indicator (from 2014 MEL Plan) Reported in Year 1 Reported in Year 2 Reported in Year 3 Notes Ind. 1 (STIR6/C): Number of improved decision-making science documents produced supported by USG assistance N N Y No baseline needed Ind. 2/C: Percent increase in hilsa abundance (CPUE) in the sanctuary areas N N N Baseline established 2016 – too early to report fully?80 Ind. 3/C: Number of clauses supported by science included in the modification of Hilsa Management Plan (HMP) N N N “No plan to measure this indicator” Y3 report Ind. 4/C: Number of fish landing centers practicing improved catch monitoring system due to the intervention of the project N N Y No baseline needed Ind. 5/EG.11-2: Number of institutions with improved capacity to address climate change risks supported by USG assistance N N Y No baseline needed Ind. 6/EG 10.2-2: Number of hectares of biological significance and/or natural resources under improved natural resource management as a result of USG assistance N Y Y No baseline needed, but concern about what is counted Ind. 7/C (4.8.1-29): Number of person hours of training in natural resources management and/or biodiversity conservation N Y Y No baseline needed Ind. 8/ EG.10.2-4: Number of people trained in sustainable NRM and/or biodiversity conservation by USG assistance N N Y No baseline needed Ind.9/C: Percent increase in stakeholder engagement in co-management in the Sanctuary area N N N “No plan to measure indicator in 2017” Y3 report Ind.10/C: Number of people complying with hilsa rules and regulation management actions due to improved delivery of the government compensation package N N N “In 2017, no plan to achieve this indicator” Y3 report Ind.11/C: Number of women benefiting from access to finance and savings in community savings and loan programs N Y Y No baseline established; concern about duplication with other projects81 80 The ECOFISH Year 2 Annual Report (p. 28) details CPUE data collected by BFRI inside and outside sanctuaries. 81 During fieldwork, the evaluation team learned that other projects in ECOFISH villages had also established savings programs. It would appear important to distinguish ECOFISH contributions. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 86 Ind. 12/EG 10.2-3: Number of people with increased economic benefits derived from sustainable NRM and conservation as a result of USG assistance N Y Y No baseline needed Ind. 13/C: Number of women and men from fishing households that have received training on alternative livelihoods N Y Y No baseline needed Ind. 14/C: No. of households with improved well-being and diversified source of income (food security, other aspects of livelihood) N Y Y No baseline established – could this be derived from village profile data? Ind. 15/EG10.2-5: Number of laws, policies, strategies, plans, or regulations addressing climate change (mitigation or adaptation) and/or biodiversity conservation officially proposed or adopted as a result of USG assistance N Y Y No baseline needed F 4.8.2-26: Number of stakeholders with increased capacity to adapt to the impacts of climate change as a result of USG assistance N N n.a. Indicator was removed after DQA in August 2016 USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 87 ANNEX VI: EDITORIAL QUALITY CONTROL English Language Editorial Issues in ECOFISH Reports The following examples from the ECOFISH Year 3 Work Plan are indicative of broader quality control issues described in the body of the evaluation report. o Unclear data: “A total of 10 fish species re-appeared significantly in catch” … references fish survey of Andharmanik River” (p. 21) o Incorrect data format in Standard Indicator Annual Target (ha) – EG 10.2: 1,92,700; Achievement 1,86,050; percent of annual performance target achieved – 97 (p. 17) o Imprecise and scientifically unconventional/inappropriate language: “This ratio of land and water unanimously supports the basic criteria to be a designated MPA… considering a marine ecosystem that support (sic) a number of threatened fishes…and nursing habitats for many majestic fishes…” (p. 51, emphasis added) o Grammar errors: During the Brood Hilsa ban period, ECOFISH Bangladesh project organized some cultural activities like debate and drawing competition for students, sewing competition for fishers’ wives and water ball game for fishers in the fishers’ communities. (p. 54) o Inconsistency: “There are no major challenges so far faced…” juxtaposed with this comment later in the same section “… Keeping their interest in HCG meetings seems a major challenge in some fishing villages” (Challenges p. 63) o Inadequately substantiated claim: “The most visible success of the ECOFISH intervention is the bumper production of Hilsa shad over 2 consecutive years in 2016 and 2017, for the first time in the last 25 years” – while not necessarily an unreasonable conclusion, this statement merits further explanation in the text of annual reports as it was not substantiated by evidence presented in the ECOFISH custom performance indicators (Table 3, p. 20) or elsewhere in that report. This is perhaps the signal achievement of ECOFISH in terms of the overall project objectives. Such claims merit detailed explanation and further substantiation. o Unexplained changes: “The catch data collected by the Citizen Scientist (sic) showed that the average catch per haul for both Hilsa and other fish increased remarkably in September 2017, compared to that was in August 2017” (p. 32, emphasis added). The graph below (Figure 8) showing this dramatic change (4x in hilsa catch in a one-month period) was presented as evidence, with no further explanation of trends or reasons (including sample size limitations). This lack of clarity/precision and editorial discipline in project reporting has the potential to undermine the credibility of project activities and may raise questions among external audiences about the scientific integrity of the project. The evaluation team became less concerned about the scientific integrity of the project activities after in-depth discussions with ECOFISH staff who were able to clarify questions raised by the evaluation team about data, methods, and conclusions in project reports. There are also emerging journal papers from ECOFISH (see van Brakel et al., 2018) that will be subject to peer review; those papers will form the accessible online knowledge base for ECOFISH after the project ends. However, even with those new publications forthcoming, it is important to ensure that all ECOFISH documents are produced at the standard expected of USAID publications. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 88 Figure 8. Example of Inadequately Explained ECOFISH Data USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 89 ANNEX VII: CONFLICT OF INTEREST AND TEAM COMPOSITION Name Ian Dutton Title Principal Organization Nautilus Impact Investing Evaluation Position? ✓ Team Leader Evaluation Award Number (contract or other instrument) AID-388-C-14-00001 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) ECOFISH-Bangladesh I have real or potential conflicts of interest to disclose. ✓ No I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date 01-15-18 Name M. Shahadat Hossain Title Professor Organization Chittagong University Evaluation Position? ✓ Team member Evaluation Award Number (contract or other instrument) AID-388-C-14-00001 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) ECOFISH-Bangladesh I have real or potential conflicts of interest to disclose. ✓ No I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 90 Date 01-15-18 Name Humayun Kabir Title Socio Economic Expert Organization IBTCI Evaluation Position? ✓ Team member Evaluation Award Number (contract or other instrument) AID-388-C-14-00001 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) ECOFISH-Bangladesh I have real or potential conflicts of interest to disclose. ✓ No I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date 01-15-18 USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 91 TEAM COMPOSITION Ian M. Dutton served as Team Leader for the ECOFISH Evaluation. He provided the overall leadership for the team, finalized the evaluation design, coordinated field activities, consolidated individual input from the team, and coordinated the assembly of the final findings and recommendations into this high￾quality document. Dr. Dutton holds a PhD in Geography from the University of Queensland as well as a Master of Science from Griffith University in Environmental Planning. He has extensive experience conducting evaluations – most recently at Nautilus Impact Investing LLC, where his responsibilities included designing strategic action and monitoring plans – and conducting independent project evaluations for foundations, corporations, and government organizations. Dr. Dutton has more than 20 years of experience working in South and East Asia and throughout the Pacific managing fisheries and marine resource management programs for The Nature Conservancy and international development agencies. He is currently a co-principal investigator of a major fisheries synthesis project at the National Center for Ecological Analysis and Synthesis (NCEAS) at the University of California, Santa Barbara and a mentor for the Global Fish 2.0 competition. Humayun Kabir served as the team’s Socioeconomic Expert for the ECOFISH Evaluation. He participated in all meetings and interviews, drafted sections of this report relevant to his expertise, and contributed to the development of the findings and recommendations of this evaluation report. Mr. Kabir has a Master of Science in Fisheries Biology and Limnology, and a Master of Business Administration. He has more than 28 years of hands-on experience as a socioeconomic and rural livelihoods expert. He has worked extensively with transferring and management, value chain development, project review, and evaluation in the field of natural resource management. Mr. Kabir has specifically worked through conducting assessments, establishing management evaluation approaches, and conducting training sessions. He is an expert in fisheries science/policy, co-management, and social impact. Md. Shahadat Hossain served as the team’s Local Fisheries Expert. He participated in all meetings and interviews, drafted sections of this report relevant to his expertise, and contributed to the development of the findings and recommendations of this evaluation report. Dr. Hossain holds a Postdoctorate in Coastal Climate Change Risk Modeling from Texas A & M University and a PhD in Ecological Modeling for Fisheries from the University of Chittagong. He also received his Master of Science in Integrated Tropical Coastal Zone Management from Asian Institute of Technology and a Master of Science in Marine Science from the University of Chittagong. Mr. Hossain is a Bangladeshi national with 20 years of experience in fisheries geo-informatics, sustainable aquaculture, integrated coastal zone management, and community-based research and monitoring. He specializes in fisheries project development and has extensive experience conducting national capacity assessments for coastal management and impact evaluations in Bangladesh for USAID, DFID, UNDP, FAO, British Council, and the Bangladesh government. Dr. Hossain has a solid understanding of USAID MEL and has worked on previous USAID projects in Bangladesh. In conjunction with his extensive work as a consultant, he has also won numerous research grants focusing on the development of economic-environmental models and best practices for resource management and sustainable fish sector development. USAID/Bangladesh ACME ECOFISH Midterm Performance Evaluation Report February 2018 92 US Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523