This publication was produced at the request of the United States Agency for International Development. It was prepared independently by John A. Hargreaves, Abul Hossain, and Monirul Islam for the Accelerating Capacity for Monitoring Evaluation (ACME) activity held by International Business & Technical Consultants, Inc. (IBTCI). This publication was produced at the request of the United States Agency for International Development. It was prepared FINAL PERFORMANCE EVALUATION Aquaculture for Income and Nutrition (AIN) Final Report February 3, 2017 FINAL PERFORMANCE EVALUATION Aquaculture for Income and Nutrition (AIN) Final Report FINAL PERFORMANCE EVALUATION FINAL REPORT AQUACULTURE FOR INCOME AND NUTRITION February 3, 2017 Original Submission: December 1, 2016 AID-388-C-14-00001 (USAID/Bangladesh Accelerating Capacity for Monitoring and Evaluation) DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government CONTENTS Acronyms .................................................................................................................... Error! Bookmark not defined. Tables........................................................................................................................... Error! Bookmark not defined. Executive Summary ..................................................................................................... Error! Bookmark not defined. Introduction ................................................................................................................. Error! Bookmark not defined. Activity Background .................................................................................................... Error! Bookmark not defined. Evaluation purpose & evaluation questions ................................................................. Error! Bookmark not defined. Evaluation methods and limitations ............................................................................. Error! Bookmark not defined. Findings and Conclusions ............................................................................................ Error! Bookmark not defined. Recommendations ....................................................................................................... Error! Bookmark not defined. Lessons learned ........................................................................................................... Error! Bookmark not defined. Annexes ....................................................................................................................... Error! Bookmark not defined. Annex I: Evaluation Statement of Work .....................................................Error! Bookmark not defined. Annex II: Reference List .................................................................................Error! Bookmark not defined. Annex III: Evaluation DESIGN Matrix .........................................................Error! Bookmark not defined. Annex IV: Data Collection Instruments .....................................................Error! Bookmark not defined. Annex V: FtF Districts Where the AIN activity Was Active and Where the AIN-PE Team Visited ..............................................................................................................................Error! Bookmark not defined. Annex VI: List of Persons Contacted ..........................................................Error! Bookmark not defined. Annex VII: Evaluation Team Disclosures ....................................................Error! Bookmark not defined. Annex VIII: Evaluation Team Composition ........................................................................................................... 83 i ACRONYMS ACME Accelerating Capacity for Monitoring and Evaluation AIN Aquaculture for Income and Nutrition ATDP Agro-based Industries and Technology Development Projects BCC Behavior Change Communication BFRI Bangladesh Fisheries Research Institute CDCS Country Development Cooperation Strategy CFC Community Feed Center CREL Climate-Resilient Ecosystems and Livelihoods DG Director General DoF Department of Fisheries, Bangladesh DO Development Objective DSAP Development of Sustainable Aquaculture Project EG Economic Growth FGD FtF Focus Group Discussion Feed the Future FY Fiscal Year GHERS Greater Harvest for Economic Returns in Shrimp GIFT Genetically Improved Farmed Tilapia GIP Genetic Improvement ProgramProgram GoB Government of Bangladesh HH Household IBTCI International Business & Technical Consultants, Inc. IP Implementing Partner IR Intermediate Results KII Key Informant Interview LSP Local Service Provider M&E Monitoring and Evaluation MoFL Ministry of Fisheries and Livestock MOU Memorandum of Understanding NGO Non-governmental Organization OSP Orange Sweet Potato PCR Polymerase Chain Reaction PITT Performance Indicator Tracking Table PL Post Larvae ii PRICE Poverty Reduction by Increasing the Competitiveness of the Enterprises SIS Small Indigenous Species SOW Scope of Work SPF Specific Pathogen-Free TBN Tilapia Breeding Nucleus ToT Training of Trainers TSH Tilapia Satellite Hatchery USAID U.S. Agency for International Development WSSV White Spot Syndrome Virus ZoI Zone of Influence iii TABLES 1. Activity Components and Objectives 2. AIN activity Basic Performance Indicator Values for Life of Activity 3. Gender Disaggregated Performance Data for AIN activity 4. Perception of Activity Utility by Select Beneficiary Types 5. Proportion (%) of activity farmers adopting specific fish pond management practices in 2012 and 2016 and net change in proportion of farmers using the practice 6. AIN activity Indicators for PE Question 1 (Improved Seed) 7. AIN activity Indicators for PE Question 4 (Household Nutrition) 8. Number of focus group discussions with AIN activity beneficiaries and non-beneficiaries in which a particular item was mentioned 2 EXECUTIVE SUMMARY ACTIVITY PURPOSE AND BACKGROUND The United States Agency for International Development (USAID)/Bangladesh contracted International Business & Technical Consultants, Inc. (IBTCI), through the Accelerating Capacity for Monitoring and Evaluation (ACME) activity contract (AID-388-C-14-00001) to conduct a final performance evaluation of the Aquaculture for Income and Nutrition (AIN) activity. The AIN activity, with total funding of $25 million, was implemented between September 2011 and December 2016 by WorldFish. The main goal of the activity was to increase income and nutrition through increased productivity of fish and shrimp in 18 of 20 Feed the Future (FtF) districts in southwest Bangladesh. This goal was addressed through four components that address four specific objectives:  Dissemination of improved quality lines of fish and shrimp seed;  Improving the nutrition and income status of farm households;  Increasing investment, employment, and fish production through commercial aquaculture; and  Policy and regulatory reform and institutional capacity building to support sustainable aquaculture growth. The purpose of the AIN final performance evaluation was to assess the effectiveness of the AIN activity in achieving its program objectives, to evaluate any constraints to achieving expected results, and to provide recommendations and lessons learned for future aquaculture development activities. EVALUATION QUESTIONS The questions addressed in the final performance evaluation of the AIN activity were: 1. To what extent have AIN interventions contributed to improved usage of aquaculture technologies and management practices at the hatchery, nursery, and farm (small and large) levels? 2a. How effective was the private sector engagement approach for genetic improvement of tilapia and carp (rohu, catla, and silver carp)? 2b. What can be done to ensure continuation of the current genetic improvement efforts until they are ready for commercialization (large-scale spawn production for commercial culture)? 2c. What can be done to ensure the maintenance of the genetic purity of these fish species? 3. To what extent have AIN interventions promoted and strengthened aqua businesses and enterprises for improved quality of fish/shrimp seed stock and feed? What were the challenges these enterprises faced in carrying out their businesses? 4. How relevant was the activity’s behavior change communication (BCC) approach in increasing the consumption of nutrient-rich food at the household level, with a particular focus on women and children? What were the gaps identified in the design and implementation of BCC, and how can BCC be further improved? 5. What were the key lessons learned from the institutional capacity building, and policy and regulatory reform interventions of the activity? METHODOLOGY The evaluation team employed a mixed-method approach using quantitative and qualitative analysis. Data collection tools included document reviews, key informant interviews, stakeholder and beneficiary interviews, focus group discussions, and site visits. A telephone survey was conducted of 370 activity beneficiaries representing carp hatchery owners, tilapia hatchery owners, prawn hatchery owners, shrimp hatchery owners, and fish nursery operators. The forms of information gathered were 3 complementary, allowing triangulation of findings and improvement in the confidence of conclusions. An Evaluation Design Matrix (Annex III) was developed to explicitly link each evaluation question to specific data collection approaches and data sources. With this approach, the evaluation team conducted 70 structured stakeholder interviews in Dhaka, Jessore, Khulna, Faridpur, Barisal, and Cox’s Bazar. In addition, 26 focus group discussions (FGDs) were held with activity beneficiaries and non-beneficiaries in districts of Jessore, Khulna, Faridpur, and Barisal. An additional 11 focus group discussions were held with local service providers (LSPs), mostly nursery pond owners (Annex IV). The evaluation team experienced minimal limitations with respect to mobility and implementation of the evaluation work plan. The range of interview sources reflected all stakeholder and beneficiary categories and included a representative sample of each. Efforts were made to minimize bias in responses of interview subjects and groups. 4 KEY FINDINGS Key findings are based on the five evaluation questions and activity focus areas (improved practices, genetic improvement programs, improved seed and feed, household nutrition, policy, and institutional capacity). The main findings include the following: General  The AIN activity was effective in achieving three of its four activity objectives. Activities related to the objective of institutional capacity building and policy development remain unfulfilled because the activity placed priority on field activities. Improved Seed and Feed  Carp hatcheries in AIN-assisted areas have now replaced most poor-performing broodfish with those provided by the activity, resulting in improved performance at the farm level.  Many tilapia hatcheries in the FtF zone of influence (ZoI) are using genetically improved broodfish provided through activity-supported tilapia breeding nucleus (TBN) and selling monosex (male) tilapia seed to area farmers at a premium price.  Carp and tilapia hatcheries are using equipment and technology provided by the activity to improve hatchery water quality and are implementing better hatchery management practices provided through activity-supported training.  The activity supported brackish water shrimp hatcheries by providing disease testing of seed. Practical difficulties with seed testing have led to a shift in emphasis to production of specific pathogen-free (SPF) post-larvae (PL), currently produced by only one hatchery in Cox’s Bazar.  At the farm level, training and access to improved seed provided by AIN activity-supported hatcheries have resulted in improved productivity of household ponds for carp and small indigenous species (SIS), and commercial ponds for carp, tilapia, and shrimp.  Shrimp farmers now recognize the benefit of stocking quality PCR-tested PL, resulting in less disease, better survival, and greater production. However, the supply of PCR-tested PL is insufficient to meet demand.  Business success of the semi-auto feed mills has been mixed and dependent on unique combinations of personal and site-specific attributes. Improved Practices  Nursery pond operators have been able to improve productivity through AIN activity￾supported efforts to link them with hatchery owners that provide improved quality carp seed and have also benefited from direct training support about improved management practices. Genetic Improvement Programs  Although benefits won’t be realized until the AIN activity has ended, the rohu genetic improvement program (GIP) is well conceived, planned, and managed. Household Nutrition  The AIN activity has promoted a polyculture production model in household ponds that consists of large fish (carps) and small indigenous species (SIS), especially mola, and dike cropping of vegetables, especially the orange sweet potato (OSP). This system has increased the availability of nutrient-rich foods that are easy to harvest in small portions for household consumption, with surplus sold to supplement household income.  Annual consumption of fish in AIN activity beneficiary households (19.5 kg/capita) was greater than that in non-AIN activity households (16.8 kg/capita). Activity beneficiaries understand the nutritional value of whole fish, especially SIS, and OSP consumption. Activity beneficiaries have 5 notably greater knowledge about nutrition than non-activity beneficiaries.  The AIN activity has created opportunities for women to generate income from aquaculture by providing mola broodfish, gill nets to facilitate pond harvest, and training on pond management. Behavior Change Communication  Food plates, pictorial festoons, and food cards were the most effective and valuable BCC interventions to increase household fish consumption, dietary diversity, and awareness of the importance of a balanced diet, especially among women and children.  The AIN activity took a Training of Trainers (ToT) approach that was effective to maximize activity reach. Activity technical staff trained non-governmental organization (NGO) implementing partner staff, government fisheries officers, and LSPs. The most effective LSPs were nursery pond operators, seed traders, and feed traders. Policy Reform  Despite formal agreements between WorldFish and Government of Bangladesh (GoB) institutions, these linkages and mission integration with the AIN activity appear to be inconsistently applied and relevant only on a case-by-case basis. RECOMMENDATIONS The evaluation team makes the following recommendations for implementing similar activities in the future: Improved Seed and Feed  The initial work on genetic improvement conducted by the AIN activity should receive timely and serious consideration for continued donor support. Ultimate ownership and management of genetic improvement programs must come from a qualified GoB agency, such as DoF or the Bangladesh Fisheries Research Institute (BFRI), or from qualified private partners that are willing to provide improved quality seed as a business activity.  More brackish water shrimp hatcheries (beyond the one currently operating) that can produce SPF seed are needed to increase the availability of quality seed to farmers.  brackish waterTo develop traceable shrimp production at the farm level, farmers need to be organized to coordinate production using SPF seed and better practices to manage disease risk. An extension effort is needed to motivate farmers to use SPF seed and implement better management practices. Future activities can serve a supportive role by coordinating linkages between buyers and groups (clusters) of trained farmers conforming to a set of standardized production practices on disease management, water management, stocking density, and feeding.  Farmers are increasingly intensifying production by using manufactured feed to produce fish and shrimp. Future activities should consider additional approaches to production intensification. Additional emphasis on the propagation and culture of air-breathing fish is warranted.  If the semi-auto feed mill initiative will be continued in future activities, removal of the equipment subsidy and identification of the criteria that best predict success should be emphasized in selecting individuals and sites for feed mill location. Joint procurement of feed ingredients among the mills has the potential to lower costs and improve competitiveness with commercially available feed. Resolution of the applicability of the GoB Feed Law to community feed centers is needed because they are currently not in compliance. Behavior Change Communication 6  The use of LSPs to deliver training is good in theory, but more rigorous vetting and selection is needed to identify effective LSPs. Follow-up by activity staff is then necessary to provide guidance, coordination, and development of additional capacity. Policy Reform  WorldFish should continue to collaborate with the Department of Fisheries (DoF) to discuss and review policies associated with implementation of the Fish Feed Acts and Rules and the Fish Hatchery Acts and Rules, and the requirement for PCR testing or use of SPF shrimp seed. Introduction After rice, fish is the most important food in Bangladesh, representing 60 percent of animal protein consumption and the main source of micronutrients. Fish is critical to the food security of the country. Per capita consumption in the country is close to the global average, but in rural households it is lower than in urban households. The demand for fish is expected to grow due to population growth and increasing affluence. In Bangladesh, almost 80 percent of the population live in rural areas and more than half are employed in agriculture. About one-third of the rural population can be considered poor, although the poverty rate may be higher. Bangladesh is the fifth-largest aquaculture producer in the world. In 2014, production from aquaculture contributed 55 percent of the total fish production of 3.55 million tons, recently surpassing inland and marine capture fisheries. There are more than 3 million fish farmers in Bangladesh, most with a small household pond of about 600 m2. Around one-third of farmed fish are produced by smallholders. The southwest region is a priority for the government, development partners, and WorldFish. The southwest is a focus for development attention because agricultural productivity lags behind much of the country (Master Plan for the Southern Delta). It has many poor and food insecure areas that are also the most vulnerable to the effects of climate change and natural disasters. The southwest is also a major center for aquaculture, including the carp seed production hub of Jessore. The southwest has abundant surface water resources and high average temperatures that are suitable for aquaculture production. Aquaculture is a national policy priority, featuring prominently in Bangladesh’s Poverty Reduction Strategy Paper. It is also a key element in its Country Investment Plan and Sixth Five-Year Plan for agriculture, food security, and nutrition. Diversifying agricultural production in higher value, nutritious agricultural production such as aquaculture is one of the areas for intervention identified in the Feed the Future FY 2011-2015 Multi-Year Strategy. International bilateral donor agencies (USAID, Danida, and GIZ) have invested heavily in aquaculture in Bangladesh for the past three decades, based on results in contributing to the country’s development goals. Several USAID-funded activities have specifically targeted aquaculture. Agro-based Industries and Technology Development Activity (ATDP) I (1995-2000) and ATDP II (2000-2004) had aquaculture (shrimp and fish) components. The USAID-funded Poverty Reduction by Increasing the Competitiveness of the Enterprises (PRICE) activity (2008-2012), including the Greater Harvest for Economic Returns in Shrimp (GHERS) initiative, adopted value-chain focused approaches to interventions in the fish hatchery and shrimp sectors of southern Bangladesh. PRICE was the first USAID-funded activity with an aquaculture component to realign its focus to contribute to meeting goals of the Feed the Future Presidential Initiative under the USAID mission’s Country Development Cooperation Strategy 7 Development Objective 2: Food Security Improved. The Cereal Systems Initiative for South Asia (CSISA), established in 2009, and implemented partly by WorldFish, also includes some work around improved fish seed and feed value chains. USAID’s Climate-Resilient Ecosystems and Livelihoods (CREL) activity (2012-2017) promoted aquaculture to diversity livelihoods that are environmentally sustainable and to build resilience to climate change. AIN contributed to USAID’s Feed the Future (FtF) Presidential Initiative under the Mission’s Country Development Cooperation Strategy (CDCS) Development Objective (DO) 2: Food Security Improved. This goal is fully consistent with the development objectives of the government of Bangladesh’s Country Investment Plan. The activity initially (2011) was called the Feed the Future Aquaculture activitiy but was renamed Feed the Future – Aquaculture for Income and Nutrition (FtF-AIN) in early 2013 and USAID – Aquaculture for Income and Nutrition (USAID-Aquaculture) later in 2013. In mid-2014, the activity became known as USAID – Aquaculture for Income and Nutrition (USAID-AIN). The implementing partner (WorldFish) has had a continuous presence in Bangladesh for more than two decades. The emphasis of the AIN activity is on technology development for improved fish strains, and capacity building in hatcheries and nurseries for wider dissemination and uptake among small- and medium-scale household and commercial producers. ACTIVITY BACKGROUND Aquaculture for Income and Nutrition (AIN) was a five-year activity that started in September 2011 and was implemented by WorldFish in collaboration with the Ministry of Fisheries and Livestock, Government of Bangladesh (GoB). It follows other recent USAID-funded aquaculture development activities (PRICE, DSAP, ADTP I and II, SQSP I and II, and CAARP). The AIN activity was one of the key activities under DO 2: Food Security Improved, of USAID Bangladesh’s CDCS. DO2 is also the flagship DO for the Feed the Future (FtF) strategy and its objective in Bangladesh: “Availability, Access, and Utilization of Domestically Produced and Nutritious Foods Increased.” Thus, the AIN activity became an important part of the FtF program, addressing vulnerable household constraints to food availability, access, and utilization. The AIN activity directly contributed to intermediate result (IR)-2.1, “Sustainably Increase Agricultural Productivity” under the DO2 result framework, as well as to the FtF goal of “sustainably reducing poverty and hunger by tackling their root causes and employing proven strategies for achieving large￾scale and lasting impact.” The main goal of the activity was to increase income and nutrition through increased productivity of fish and shrimp in 18 of 20 FtF districts (85 upazilas) in southwest Bangladesh (Dhaka, Khulna, and Barisal divisions). Specifically, AIN aimed to increase aquaculture productivity by developing hatcheries and nurseries, disseminating improved fish and shrimp seed, enhancing farm management skills of smallholder farmers, promoting new technologies to expand commercial aquaculture, developing backward and forward market linkages, supporting policy reform, and building capacity of the public and private sectors resulting in increased farmers’ productivity and revenue. Increased productivity at the household level was anticipated to result in more food for consumption and more produce for sale. This would contribute to a further adoption of these improved technologies and management practices, and would result in an overall increase in rural households’ income, thereby sustainably reducing poverty. The activity contributed to achieving the FtF goal of sustainably reducing poverty and hunger through four objectives, each linked to one of four activity components. 8 Table 1. Activity Components and Objectives The objectives were intended to enhance skills and improve capabilities in farm productivity and management, as well as in aqua-business and aqua-entrepreneurship that will lead to increased innovation, competitiveness, and higher incomes. To achieve the above four strategic objectives, the AIN activity was designed to implement activities within four USAID/Bangladesh FtF intermediate results (IR) components: IR-1 shrimp and fish brood and seed stock improved; IR-2 household aquaculture productivity increased; IR-3 commercial aquaculture productivity increased; and IR-4 capacity of Bangladesh institutions to support aquaculture increased. The premise or development hypothesis of the AIN activity was that: 1) Systemic improvements to the hatchery sector will increase seed quality and ultimately increase the productivity and profitability of existing farming systems; 2) Training for small and marginalfarmers on improved pond management and horticulture, combined with use of improved seed, increases farm productivity, household incomes, and household nutrition; 3) Improved seed quality, coupled with improved management practices and stronger linkages with value chain actors, will increase the profitability and output of commercial aquaculture and stimulate employment on-farm and in associated value chains; and, 4) Support for regulatory reform in the hatchery and feed sectors will provide an enabling environment for the growth of commercial aquaculture enterprises. The activity aimed to intervene in the aquaculture sector among fish and shrimp value chain actors to build the capacity for sourcing inputs, such as quality broodfish at the hatchery level; to improve hatchery management practices; and to work with forward linkage steps in the value chain to create an enabling environment for nursery operators and farmers to obtain quality seed and facilitate improved farm management practices, which will lead them toward sustainable income and fulfill the nutritional needs of the target actors. The AIN activity proposed the following activities to address each activity component: Objective 1: Improved fish and shrimp seed This component will develop and/or supply improved quality/ genetically improved tilapia, carp (rohu, mrigel, and catla), prawn, and shrimp to farmers in Khulna, Barisal, and Dhaka divisions, providing the basis for a 12 percent to 27 percent increase in fish yield in the southern delta, benefiting around 1.1 million farm households by 2016. Improved aquatic farming systems are expected to provide additional 40,000 metric tons of fish and shrimp by the end of 2016. Government brood centers and private hatchery operators will be assisted with sourcing quality brood stock, establishing management systems to maintain and develop seed quality, and accelerating the distribution of improved quality of fish, shrimp, and prawn to farmers. Objective 2: Improved household aquaculture Trainings on improved nutrition and income-enhancing aquaculture and horticulture activities will be extended to 64,000 poor and vulnerable households through this component. Interventions with No. Activity Component Objective 1 Fish and shrimp seed Disseminate improved quality lines of fish and shrimp seed 2 Household aquaculture Improve the nutrition and income status of farm households 3 Commercial aquaculture Increase investment, employment, and fish production through commercial aquaculture 4 Institution and policy Implement policy and regulatory reform and increase institutional capacity building to support sustainable aquaculture growth 9 implementing partners will be primarily focused on introducing WorldFish aquaculture technologies into existing livelihood initiatives through training, demonstrations, and communication programs. The number of meals containing fish per month will double, indicating improved nutrition. Production of small indigenous fish species will also be promoted. Objective 3: Improved commercial aquaculture This component will stimulate investments, employment, and income in the aquaculture sector of Bangladesh. The activity will provide technical support to around 50,000 shrimp and prawn farmers and 15,000 carp farmers, assist them with adopting high-value commercial fish and shrimp culture practices, and will identify new options for brackish water aquaculture that will reduce pressure on natural sources and will increase income. Objective 4: Policy reform and increased institutional capacity This component will focus on assisting the government in the implementation of existing policy and regulatory measures in the Hatchery and Feed Acts. It will also facilitate the recommendation and adoption of new policies. Institutional capacity building will also be supported. This will include assistance in implementing the Aquaculture and Fisheries Memorandum of Understanding (MOU) between Bangladesh and India on cooperation in fish seed improvement and germplasm conservation and management. The activity is working to improve the capacity of private sector associations and business, and is assisting both public and private bodies to ensure their effective participation in the activity. Evaluation purpose & evaluation questions Evaluation purpose The purpose of the evaluation was to assess the effectiveness of the programmatic approaches of the AIN activity. It explored and evaluated the activity’s key critical interventions that USAID/Bangladesh should continue in its future aquaculture activity and other relevant Economic Growth (EG)/FtF activities. It also assessed whether there were any successful pilot interventions that could be scaled up for greater impact in the future across the sector, and made recommendations in such instances. The findings also discussed the obstacles and challenges the activity has faced, and how it addressed those challenges during implementation. Evaluation questions Evaluation of the AIN activity was comprehensive looking at all activity’s components. The evaluation team ensured that data was gender-disaggregated and subjected to gender analysis. The following questions were addressed in order of priority: 1. To what extent have AIN interventions contributed to improved usage of aquaculture technologies and management practices at the farm (small and large), nursery, and hatchery levels? 2a. How effective was the private sector engagement approach for genetic improvement of tilapia and carp (rohu, catla, and silver) fishes? 2b. What can be done to ensure continuation of the current genetic improvement efforts until they are ready for commercialization (large-scale spawn production for commercial culture)? 10 2c. What can be done to ensure the sustainability (maintenance) of the genetic purity of these fish species? 3. To what extent have AIN interventions promoted and strengthened aqua businesses and enterprises for improved quality of fish/shrimp seed stock and feed? What were the challenges these enterprises faced in carrying out their businesses? 4. How relevant was the activity’s behavior change communication (BCC) approach in increasing the consumption of nutrient-rich food at the household level with a particular focus on women and children? What were the gaps identified in the design and implementation of BCC, and how can BCC be further improved? 5. What were the key lessons learned from the institutional capacity building, and policy and regulatory reform interventions of the activity? Evaluation methods and limitations The AIN performance evaluation (AIN-PE) scope of work required the evaluation team to address five questions. With the exception of questions 2a-2c, the team assessed progress toward FtF standard and activity custom indicators as a component of the response to evaluation questions. For questions 2a-2c, regarding sustainability of the genetic improvement program, a qualitative assessment based on Key Informant Interviews (KII) was made. The evaluation team employed a mixed-method approach using quantitative and qualitative analysis. Data collection tools included document reviews, KIIs, stakeholder and beneficiary interviews, focus group discussions, and site assessments. The forms of information gathered were complementary and interrelated. An Evaluation Design Matrix (Annex III) was developed to explicitly link each evaluation question to specific data collection approaches and data sources. To assess the performance of the AIN activity, the team implemented a three-tiered approach that provides analytic breadth and depth.  Literature Review. The evaluation team conducted a literature review of the AIN activity and its subcomponents, drawing on secondary data routinely produced by the AIN activity, such as quarterly and annual progress reports, monitoring data, baseline surveys, and other AIN deliverables and documents. It included a review of published and unpublished literature on the activity, analysis of AIN performance or program data, peer-reviewed journal articles, and other reports and documentation such as baseline and mid-term surveys.  Site Assessments. The evaluation team traveled to selected AIN activity sites in the FtF ZoI in southwest Bangladesh to assess AIN implementation outcomes. Three methodologies were applied during these field trips: key informant interviews, focus group discussions, and direct observation.  Survey Questionnaire. ACME commissioned a subcontractor (e.Gen) to conduct a quantitative survey of stakeholders to address Evaluation Questions 1 and 3 to validate the AIN activity estimation of the extent of activity impacts and outcomes. Component 1: Literature Review Under this component the AIN-PE team conducted a review of secondary data to:  Provide data and input to address the evaluation questions and comply with other requirements for substantiation of evaluation findings, and  Compile and compare planned versus actual outputs and activity level outcome data. AIN activity performance and program reports were collected and indicators analyzed. Secondary source documents (Annex I) were collected and reviewed and data collated, extracted, analyzed, 11 summarized, and presented in tables, charts, and figures. Each AIN Intermediate Result had its own set of measurable indicators. To the extent possible, routine activity documents (especially quarterly and annual reports) were consulted to assess achievement of activity work plan objectives. Component 2: Site Assessments The PE team worked closely with AIN activity staff to select sites. Because the AIN performance evaluation is descriptive and lacks control groups, the PE team employed purposeful cluster sampling as opposed to a random sampling mode. Site selection criteria included but were not limited to:  Sites in the FtF Zone of Influence in southwest Bangladesh, including Khulna, Jessore, Barisal, and Faridpur activity hubs;  Sites with a range of activities in near proximity;  Sites with a large number of beneficiaries affected by AIN activities, as well as non-beneficiaries; and  Sites where farmers and other beneficiaries are exemplars of success and best practice, as well as sites where farmers and other beneficiaries have not performed as well despite reasonable investment of AIN resources. Given the dispersed location of the AIN field activity sites, the evaluation team carefully balanced the mix of labor and travel to optimize cost-effectiveness in conducting the evaluation tasks. Data collection instruments were prepared prior to site selection to allow comparisons among sites with respect to evaluation questions. After sites were selected, the team closely examined available data and conducted interviews with the AIN regional hub office staff to determine which data collection methodologies were best suited to a given site. Site assessment data collection methodologies included key informant interviews, focus group interviews, and direct observation. Key Informant Interviews Key informant interviews included AIN activity staff, GoB (DoF and BFRI) officials; key partner NGO leaders; owner/managers of carp, tilapia, brackish water and freshwater shrimp hatcheries; tilapia breeding nuclei and satellite hatcheries; community feed centers and large feed mills; and fish and nurseries. In preparation for KIIs, the evaluation team developed a list of open-ended and closed-ended questions for each category of key informant (Annex IV). For each stakeholder type, interviews were structured to ensure that each interview was arranged with the same types of questions in the same order. This enabled the evaluation team to reliably aggregate interview findings and ensure that accurate comparisons could be made between stakeholders surveyed. The evaluation team used interview best practices and standard guidelines to ensure that the questions were worded clearly, sequenced effectively, and asked in a neutral manner to elicit valid and reliable responses from the interviewees. The evaluators documented the interviews by taking detailed notes. Focus Group Discussions The focus group discussions were coordinated by Nutrition Specialist Dr. Monirul Islam, a native Bengali speaker. During the focus group discussions, the evaluation team observed social attributes and conducted open-ended discussions with focus groups to identify firsthand findings and accumulate relevant field data. Each category of focus group had its own open-ended question checklist (Annex IV). The evaluation team conducted FGDs with AIN activity beneficiaries and non-beneficiaries to assess differing perspectives among the groups to inform the assessment of the AIN-PE evaluation Question 4 about improvements in household nutrition, particularly among women and children, as a result of activities. Questions were also asked about BCC techniques. To verify notes and findings, focus group interviews were transcribed by a note-taker contracted by ACME, who accompanied the team on field visits. Direct Observation 12 The team used direct observation of the implementation of activities at AIN activity sites to validate findings of activity accomplishments derived from documentation and interviews. Observation was useful to triangulate findings using the expert judgement of the evaluators to identify anomalies and “ground truth” information collected prior to direct observation. The evaluation team visited key facilities related to AIN activities, including genetic improvement program sites, carp hatcheries, tilapia breeding nucleus centers, tilapia hatcheries, shrimp nurseries, carp nurseries, and semi-auto feed mills (community feed centers). Common questions asked included (see Annex IV):  When were the AIN interventions established?  How did they connect to AIN?  How did the activity assist them?  What were the results of that assistance?  What were the strongest and weakest aspects of that assistance?  What types of training tools were used, and which were most effective?  What additional assistance might be useful?  What are the plans for continuing activities without AIN?  Are there any particular constraints to further growth, either by government policies/regulations or other internal/external constraints? Component 3: Survey Questionnaire AIN-PE Survey Questionnaires (Annex IV) were intended to provide quantitative data, and analysis to assist with the responses to the AIN-PE scope of work evaluation questions 1 and 3: Evaluation Question 1: To what extent have AIN interventions contributed to improved usage of aquaculture technologies and management practices at the farm (small and large), nursery, and hatchery levels? Evaluation Question 3: To what extent have AIN interventions promoted and strengthened aqua businesses and enterprises for improved quality of fish/shrimp seed stock and feed? What were the challenges these enterprises faced in carrying out their businesses? The intention of administering a survey questionnaire was to validate accomplishment of indicator targets by the AIN activity and to quantify the impact or outcome of activity interventions. Survey Informant Selection AIN-PE Survey Questionnaires were administered over telephone with the following stakeholders and beneficiary groups, with the approximate number of group members indicated in parentheses:  Carp hatchery operators (84)  Tilapia hatchery operators (29)  Shrimp hatchery operators (24)  Prawn (golda) hatchery operators (7)  Nursery pond operators (600 × 25 percent = 150 + 10 percent = 165)  Community feed centers (61) Stakeholder/beneficiary lists were obtained from the AIN activity. Selection of nursery pond operators was stratified by activity hub area (n=4; Khulna, Jessore, Barisal, and Faridpur). Survey Design The evaluation team designed a survey instrument in English to conduct interviews with select groups of activity beneficiaries. The instrument included a data sheet and questionnaire. The evaluation team translated the instrument into Bengali. This instrument included 1) informant identification data including informant’s household, location (district), and gender, 2) questions about technical capacity achieved as a 13 result of AIN activities, and 3) questions about the perceived value of AIN activity support. Informants were anonymous to protect their privacy and enhance the reliability of the data. The identification data was used by a survey team (e.Gen) subcontractor to stratify, disaggregate, and analyze survey findings. The survey contained components to address Evaluation Question 1 about adopting improved technologies and management practices, and Evaluation Question 3 about strengthening aqua-businesses. Survey Results, Analysis, and Presentation The survey team provided a brief discussion of the analysis of the data collected through administration of the survey. The analysis included non-parametric and parametric analyses, including frequency distribution, mean, median and mode, and variance analysis. The survey team also provided data presentation that included tables, charts, and graphs. Fieldwork Summary and Limitations With this overall approach, the evaluation conducted 48 person-days of field work in 13 of the 18 FtF districts where the activity was active, plus Cox’s Bazar (Annex V). The evaluation included 70 structured KIIs, over 30 meetings and discussions with AIN, GoB, and university staff; 26 structured FGDs with farmer groups on nutrition and household aquaculture; and 11 structured FGDs with LSPs. For the FGDs on nutrition and household aquaculture, half were with beneficiaries and half with non￾beneficiaries, with 10-12 individuals per group. Data collection addressed major activities, including improved fish and shrimp seed quality, household aquaculture, commercial aquaculture and institutional capacity building, and policy development. The evaluation team experienced minimal limitations with respect to mobility and implementation of the workplan. The range of interview sources reflected all stakeholder and beneficiary categories and included a representative sample of each. Efforts were made to minimize bias in responses of interview subjects and groups. 14 FINDINGS AND CONCLUSIONS FINDINGS AIN Overall Performance The general approach of the activity to focus efforts on the “upstream” feed and seed parts of the value chain resulted in considerable “downstream” multiplier effects. The emphasis on fish and shrimp seed quality, coupled with farmer training on improved practices, resulted in a substantial increase in aquaculture production in the FtF ZoI. The activity got off to a slow start and some initiatives (e.g., improved broodfish distribution to hatcheries) were not particularly well planned, but these activities improved and the activity became more effective as time went on. Most broodfish in carp hatcheries in the FtF area are now of good quality as a result of the improved broodfish distribution activity. Hatchery production from improved quality broodfish has allowed dissemination of improved seed at the grassroots level in the FtF ZoI. AIN activity quarterly and annual reports indicated that the activity conducted many training activities, trained many farmers (129,000), and the training and field staff of the AIN activity were very active (WorldFish 2016). The activity developed an effective set of training materials to support training activities. The activity has had many field-level activities. Training was key to the growth in aquaculture production in the FtF ZoI. Areas of emphasis to implement the activity varied by activity hub. In Jessore, hatcheries were used; in Khulna, shrimp depots were used; and in Barisal, with a large number of household aquaculture farmers, NGOs were used to implement the activity. The activity has accelerated the involvement of women in fish farming through the household aquaculture component. The genetic improvement activities of the activity were not part of the original proposal; however the initiation of this activity has the potential for significant long-term impacts if support for the activity can be sustained. The tilapia breeding nucleus concept is working very well and has been successful at supplying improved seed to farmers. The AIN activity served a critical coordinating function to advance aquaculture in the FtF ZoI. In particular, the activity developed and strengthened linkages from hatcheries to nursery pond owners to grow-out farmers. It is a challenge to evaluate the AIN activity in terms of indicator values because indicator targets varied from year to year. In general, the activity met its targets for the indicators selected for review by the evaluation team. In some cases, targets for an upcoming year were based on achievements in a previous year. In other cases, targets were progressively adjusted downward so that the activity was able to meet the target only in the last year. This may have been affected by changes in FTF indicator definitions. Although the supply of improved quality shrimp seed to farmers was near the targeted quantity, the amount of shrimp produced by farmers was only 65 percent of the target in the last year of the activity. Shrimp hatcheries did not take up polymerase chain reaction (PCR) testing to the degree anticipated and desired because the owners and managers were unwilling to keep seed batches separate and did not want to discard seed that tested positive for White Spot Syndrome Virus (WSSV), and farmers were unwilling to pay the added costs. Thus, the activity was not able to achieve targets for the area covered and increase shrimp production. 15 Table 2. AIN activity Basic Performance Indicator Values for Life of Activity The activity established five custom indicators (Table 2) to evaluate progress in three of the four activity components. Life-of-activity indicator values for each of these core indicators demonstrate that the activity met its objectives with respect to number of households reached, area covered by the program, increase in fish production, and additional value from fish and shrimp production. The activity fell somewhat short of its target for increased shrimp production. The reasons for this are related to a shorter supply of PCR-tested seed than planned and a lower price for shrimp on the global market that reduced shrimp production value and served as a disincentive for investment in shrimp production. Of the three main activity components (improved seed, household aquaculture, and commercial aquaculture), the greatest impact across all indicators was improved fish and shrimp seed, which accounted for 83 percent of households reached, 66 percent of the area covered by the program, 75 percent of the increase in fish production, 30 percent of the increase in shrimp production, and 56 percent of the additional value from fish and shrimp production (Table 2). The AIN activity initially established custom indicators that applied to the FtF ZoI only. It quickly became apparent that activities were having broader effects on aquaculture development in the country. Thus, targets were revised and partitioned between FtF ZoI and non-FtF ZoI. For simplicity, and given that the mandate of the activity was to work in the FtF area, the activity evaluation restricted its focus to the FtF ZoI only. However, it is necessary to appreciate that activities and impacts extended beyond the FtF area. Depending on the indicator selected, about 40-50 percent of activity impacts occurred outside the FtF ZoI. Given that the improved seed component of the activity had the greatest impact as measured by core indicators, it is not surprising that significant impacts occurred outside the FtF ZoI because activity-supported carp and tilapia hatcheries were not obligated to restrict seed sales to farmers in the FtF area. Indeed, Jessore is a key center for carp seed production for the whole country and so the value of activity interventions extended to farms throughout Bangladesh. 16 Table 3. Gender Disaggregated Performance Data for AIN activity For the Feed the Future indicators (Table 3) that are disaggregated by gender, AIN activity performance suggested gender parity in terms of activity outcomes relative to targets. Indeed, nearly all the individuals trained under the household aquaculture component were women. For that component, the activity reached more than 65,000 households (Table 2). There were more jobs attributed to activity implementation for men, principally because most of these occurred in hatcheries, nurseries, and seed traders, occupations that are traditionally held by men. There were more women who received short￾term training on farming practices and household nutrition than men. Compared with the baseline in 2011, the gross margin per hectare of carp for women increased by 53 to 188 percent, depending on year. The gross margin per hectare of shrimp for women increased by 87 to 330 percent, compared with the 2011 baseline. The proportional increase in gross margin of carp and shrimp was greater for women farmers than men farmers because women farmers received both training and improved seed. Men farmers were more likely to receive improved seed only. The number of boys and girls under five reached by nutrition training of activity beneficiary farmers was similar. Table 4. Perception of Activity Utility by Select Beneficiary Types Of the categories of activity beneficiaries questioned about the value of activity interventions, the minimum proportion that rated activity interventions at least somewhat useful was 87 percent (Table 4). Reasons for dissatisfaction varied by beneficiary group. Some carp hatchery owners in the Jessore area 17 are dissatisfied that the activity hasys supported hatcheries in outlying areas, taking business away from the traditional carp seed production center. Some shrimp hatchery owners were dissatisfied that there was a lack of a greater incentive for them to produce PCR-tested or SPF PL because the price differential to normal seed was too low. Usage of Improved Aquaculture Technologies and Management Practices at the Hatchery Level Carp Hatcheries Jessore is well-known by fish farmers in Bangladesh as a hub of carp hatchery production. AIN staff began work with carp hatcheries in the Jessore region early in the activity (2012). The activity took a three-pronged and complementary approach: 1) provide carp hatcheries with a supply of new male and female broodfish from three natural river sources (discussed below), 2) provide a technical package to upgrade hatchery infrastructure and operation, and 3) provide training in broodfish pond management practices. One of the key issues affecting hatchery output prior to AIN activity involvement was poor survival of fertilized eggs and yolk-sac larvae. Nearly all hatcheries in the Jessore area use groundwater as the primary water source, and this water is typically deficient in dissolved oxygen (<1.5 mg/L) and enriched with dissolved iron. The activity provided a tiered aeration screen (called an oxygen tower) and a dissolved oxygen meter to 40 hatcheries. Implementation of this simple technology had an immediate and profound impact on improving the survival of sensitive yolk sac larvae. Another activity recommendation was to reduce broodfish stocking density from 20-25 kg/decimal to 12-15 kg/decimal, a recommendation demonstrated to result in a greater production of spawned eggs. All carp hatcheries visited by the evaluation team have implemented the lower stocking density. Other technologies and practices adopted by AIN activity hatcheries include use of activity-supplied microscopes to evaluate the suitability of eggs for fertilization (ripeness), testing hatchery water pH to maintain between 7.5 and 8.5, and using a quality feed to enhance broodfish nutrition and egg quality and quantity. The activity also arranged for study tours of carp hatcheries in Mymensingh, elsewhere in Bangladesh, Thailand, and Vietnam for carp hatchery owners to gain knowledge and professional linkages with other progressive hatchery owners in the country and to observe high-performing hatcheries in other countries. Two carp hatcheries in Jessore have installed a biofiltration system to allow water recirculation, a promising technology to reduce the risk to hatchery operations of low groundwater availability during the dry season. Recirculated water has better water quality than fresh groundwater, particularly with respect to dissolved oxygen and iron. Recirculation also reduces water pumping costs because a low￾head recirculation pump can be used instead of a high-head well pump. Interventions of the AIN activity to improve carp hatchery technology and management practices have resulted in increased output of quality seed from activity-associated hatcheries. Tilapia Hatcheries Similar to the approach taken with carp hatcheries, the activity provided a combination of 1) Genetically Improved Farmed TilapiaTilapia (GIFT) broodfish (discussed below), 2) the same technical package provided to carp hatcheries, and 3) classroom and hands-on training and study tours in Bangladesh and abroad. All activity-associated hatcheries received broodfish derived from 11th- generation GIFT-strain fish produced by WorldFish in Malaysia. In the greater Jessore area, GIFT broodfish were distributed to hatcheries through the Tilapia Hatchery Owners Association. Compared with carp culture, tilapia culture in Bangladesh is relatively new. The technology implemented for egg incubation and early fry rearing was developed by the Asian Institute of Technology in Thailand and is now the standard in tilapia 18 hatcheries in Bangladesh. Tilapia hatcheries practice hormonal sex reversal to produce monosex male tilapia seeds that are sold to farmers. Activity-provided microscopes are used to evaluate the extent of sex reversal through visualization of gonads. The activity also provided mixing machines to the Tilapia Hatchery Owners Association to aid in the production of fry feed for sex reversal. Relatively small-scale, activity-supported hatcheries using GIFT strain tilapia are now able to compete with large-scale commercial hatcheries. Some activity supported hatchery owners are unofficially branding monosex male fry as “USAID” or “WorldFish” to compete with commercially available branded seed. Shrimp Hatcheries The AIN activity did not make a large effort to change technology or practices in black tiger shrimp hatcheries, with two notable exceptions. One key issue is that broodstock are captured from the wild and arrive at shrimp hatcheries in poor condition and under stress. Between 35 and 50 percent of broodstock test positive for White Spot Syndrome Virus (WSSV) when they arrive at the hatcheries. This virus can be vertically transmitted to seed. The hatchery management practice with the greatest potential to improve seed quality, if changed, is to shift from the use of wild-caught broodstock toward genetically improved broodstock that is pathogen-free or pathogen-resistant. Changing this practice at this time is not practical because the options are seen as too costly to hatchery owners. To manage the disease risk associated with vertical transmission of WSSV from brood to seed, the activity introduced the concept of “one mother – one tank” to four participating AIN-supported hatcheries. This individual spawning system is an alternative to the mass spawning system that is standard in black tiger shrimp hatcheries in Bangladesh. It permits testing of broodstock and seed to confirm presence or absence of WSSV from batches of seed produced by a single brood female shrimp. Through training, all hatcheries are aware of the value of PCR testing for the presence of WSSV, but only four (of 60 total) shrimp hatcheries are using individual spawning systems, most in conjunction with a larger proportion of seed production through mass spawning. The AIN activity has invested in analytical equipment for a PCR testing laboratory that is operated by DoF to provide hatcheries with the means to comply with the PCR Act, which mandates PCR testing of all seed sold. The activity has provided similar support to a laboratory at BFRI – Cox’s Bazar that has PCR testing equipment and other disease diagnosis (e.g., histopathology) capability. PCR testing and individual spawning have not been widely adopted because there is a large surplus of hatchery capacity in Bangladesh. Existing black tiger shrimp hatcheries have the capacity to produce around 20 billion PL and the demand is for only 8 billion PL (Debnath et al. 2015). As a consequence, hatchery owners compete with each other for market share, driving down price and providing no incentive to produce quality seed. Therefore, there is little incentive to improve practices, especially those that add cost, such as PCR testing. The cost to produce PCR-tested PL is nearly double that of non-tested PL. Finally, farmers are not willing to pay much extra for PCR-tested PL. PCR testing is now recognized as a valuable but intermediate step in addressing the core issue of virus￾carrying broodstock. The ultimate solution is to close the system and use broodstock that are free of viruses. The AIN activity has been working closely with one hatchery (MKA Hatchery) in Cox’s Bazar on PCR testing, individual spawning systems, and more recently with the introduction of specific pathogen-free (SPF) brood black tiger shrimp from Hawaii (discussed in greater detail below). The AIN activity has also assisted the MKA Hatchery with development of a water recirculation system to provide more stable water quality than that which occurs in hatcheries that use water from the Bay of Bengal, which has pronounced seasonal fluctuations in water quality. After working out failures, problems, and constraints, the activity’s successful interventions with the MKA Hatchery represent a model for other hatcheries. The AIN activity has also established a live foods laboratory at BFRI in Cox’s Bazar that produces phytoplankton (algae) species that can be used by hatcheries as live food for larval shrimp. Such a facility 19 has the potential to serve as a reference lab and a source for pure cultures of algae that can be used as hatcheries start their annual operation cycle. Hatcheries do not have adequate facilities to maintain algal stock cultures that are not contaminated during the several months when they are not in operation. Usage of Improved Aquaculture Technologies and Management Practices at the Nursery and Farm Level Although the evaluation team visited fish nurseries and farms, there was no systematic evaluation of the usage of improved aquaculture technologies and management practices at these levels during evaluation field work. However the evaluation team found no reason to question the activity’s reported results. One measure of adoption of improved practices by nursery pond operators is the number of fingerlings produced per kilogram of spawn obtained from hatcheries. In 2014, there were about 41,500 fingerlings/kg spawn. That value increased to about 71,000 fingerlings/kg spawn in 2015 and about 70,000 fingerlings/kg spawn in 2016, the result of implementing improved management practices in nursery ponds. Beginning in 2012, the activity conducted surveys among its beneficiaries to evaluate adoption of verifiable better management practices. In the survey results, there was no disaggregation by nursery or farm level. Given that nursery ponds and grow-out ponds are similarly managed, the survey results were assumed to be relevant and applicable to both pond types. The adoption survey indicates that the practice with the highest level of adoption (81 percent) in 2016 was three-layer stocking, a measure of polyculture effectiveness and productivity. The practice also had the greatest rate of increase (by 21 percent) between 2012 and 2016. Other practices with high rates of adoption included 1) managing ponds to have green water, 2) cleaning the pond bottom after harvest, 3) ensuring ponds get adequate sunlight by cutting away surrounding trees, and 4) stocking SIS. The practices with the lowest rates of adoption were dike cropping of vegetables and knowledge sharing with neighbors. Table 5. Proportion (%) of activity farmers adopting specific fish pond management practices in 2014 and 2015 and net change in proportion of farmers using the practice The technical training package for household and commercial aquaculture focused on a suite of best management practices. Periodically the activity surveyed farmers to measure adoption rates of particular practices. Table 5 lists these practices in order of adoption rate in 2015 The practice with the greatest adoption rate in 2015was a three-layer polyculture scheme in which fish are stocked to exploit surface, water column, and bottom niches in the pond. The activity developed a number of demonstration models that commonly include carp or other relatively high-value fish (e.g., climbing perch) for marketing and small indigenous species (e.g., mola) for household consumption. The greatest increase in 20 use of practices were three-layer polyculture, regular harvesting of small fish, and sharing knowledge with neighbors, although this latter practice was among the least adopted. Genetic Improvement Program for Carp Prior to implementation of the AIN activity, the quality of carp broodfish in hatcheries in the AIN activity area had deteriorated from inbreeding, genetic drift, and poor broodfish management and selection practices, resulting in poor-performing fish at the farm level. The genetic improvement program (GIP) for carp is one of the cornerstones of the AIN activity, although it was not part of the original activity proposal. Considerable effort was made to identify suitable hatchery partners and develop the infrastructure and procedures necessary to conduct a long-term breeding program. The activity originally planned to conduct GIPs for rohu, catla, and silver carp. This discussion is restricted to the rohu GIP because the fish is the most popular culture species in the country and the rohu GIP is the most advanced. (A founder population for the silver carp GIP has been established and work has been initiated on the catla GIP.) The genetic improvement program was designed to require 10 years from the collection of wild broodfish to the release of fourth-generation rohu to farmers. The activity is currently in the fourth year of that process. Assuming an improvement of 10 percent per generation, fish released to farmers in 2022 will grow 40 percent better than now, a sufficiently large difference for farmers to appreciate the value and be willing to pay more for it. The activity made efforts to involve public- and private-sector stakeholders in the genetic improvement process. An initial attempt to give one farmer responsibility for care of the catla system was ultimately changed so that activity staff attended to all routine as well as critical activities of the rohu GIP. Private￾sector engagement was limited to leasing ponds and spawning facilities from established hatchery owners. WorldFish does not plan to release genetically improved rohu to farmers until the F4 generation is produced, only after a level of culture performance that is considered “remarkable” is attained. WorldFish staff believe that fish farmers will not adopt use of the fish unless a large difference between current stock and improved stock can be demonstrated. The current experience with tilapia strains in the country and the difficulty in obtaining a price premium for GIFT-strain tilapia reinforces the strategy to delay release of rohu. Genetic Improvement Program for Tilapia Strictly speaking, there is no genetic improvement program for tilapia in Bangladesh. The AIN activity introduced eight cohorts of 56 families of 11th-generation GIFT-strain tilapia from Malaysia to Bangladesh in 2012. After quarantine, about 36,000 fingerlings (future broodfish) were distributed to 30 hatcheries around the country, including five within the FtF ZoI. Each hatchery–tilapia breeding nucleus (TBN) received 1,600 fingerlings from each of eight cohorts. The initial concept to preserve the performance value of the improved fish was to rotate males among TBN hatcheries. Due to logistical challenges associated with executing this plan, and mutual suspicion among TBN managers, the approach was changed to set up a male rotation system among the eight cohorts, each held in a separate hapa net, with a different scheme applied in alternate years. The system of rotating males among cohorts within￾farm is simple and straightforward, and, if executed properly, can preserve the value of improved broodfish. Improved Access to Quality Seed The indicator values presented in Table 6 show the effects of the distribution of improved carp, tilapia, and brackish water shrimp seed to farmers. Relative to the baseline, there were increases in all indicators. In some years, the activity did not achieve its target, in part because the target was adjusted based on the previous year’s performance. Distribution of improved broodstock in FY 2014 and 2015 21 resulted in an increased supply of improved seed, near or above the targeted level. A combination of improved seed and farmer training resulted in a greater area of production and a greater gross margin of fish (carp and tilapia) and shrimp. Table 6. AIN activity Indicators for PE Question 1 (Improved Seed) Carp Recognizing the low quality of existing hatchery-produced seed and the lengthy period necessary to develop genetic improvement programs for carp, the AIN activity implemented a medium-term (activity cycle) approach of providing 85 carp hatcheries with 71 tons of broodfish collected from natural river sources in the country. About 50 percent of carp broodstock were replaced in more than 90 percent of carp hatcheries in the FtF ZoI. This approach, coupled with hatchery infrastructure upgrades and training, has resulted in immediate benefits. Between 2012 and 2015, about 70 tons of quality spawn was produced by the 85 activity-supported hatcheries. The activity estimates that carp production increased by about 113,000 tons from the efforts of more than 790,000 farmers in the FtF ZoI in 2015-2016. An important outcome of improving access to quality carp seed was the decentralization of carp hatchery production from Jessore. Nursery operators once dependent on carp hatcheries in the Jessore hub are now able to procure spawn from quality broodfish from hatcheries in districts throughout the FtF ZoI that formerly did not have well-functioning carp hatcheries. Carp hatcheries supported by the AIN activity were not obliged to restrict marketing of spawn to the FtF area and so the impact of improved access to quality carp seed extended beyond the FtF ZoI. Tilapia As indicated previously, the AIN activity introduced the GIFT strain in 2012. The yields of GIFT strain tilapia are 25-30 percent greater than other available strains. GIFT-strain tilapia, as mixed-sex fry, were introduced by the activity to private and government hatcheries across the country. About 0.8 million 22 fry were distributed to AIN activity-supported hatcheries and about 0.3 million fry were distributed to non-AIN activity-supported hatcheries. There are about 450 tilapia hatcheries in Bangladesh and 100 of these were supplied with 11th-generation GIFT fish from TBN, including 51 tilapia hatcheries in the FtF area supported by the activity. About 25 percent of the tilapia fry supply to farmers in the FtF zone is from these activity-supported hatcheries. Each hatchery provides seed to 100-200 farmers. In the FtF ZoI, the activity established five TBN that were linked with seven TSH. The criteria for being considered as a TBN were an expression of interest, adequate facilities, technical capability, and a willingness to bear all costs to maintain the TBN. In turn, the activity provided all technical support. Most tilapia hatcheries in the FtF area are now using genetically improved tilapia broodfish to produce monosex male fry. Based on feedback from farmers, the GIFT fish are providing yields that are 25-30 percent greater than other commercially available strains. Small tilapia hatcheries supported by the AIN activity are now able to compete with large-scale tilapia hatcheries because the activity-supported hatcheries received high-quality GIFT seed, training, and hatchery infrastructure upgrades. Tilapia seeds are being branded by hatchery owners as “USAID,” “WorldFish,” or “AIN.” Shrimp The meaning of “quality” for carp and tilapia refers specifically to performance in culture, especially growth rate. In contrast, for shrimp, “quality” refers specifically and primarily to disease resistance, especially to WSSV, thereby affecting survival. (Most of the following discussion refers to brackish water shrimp, also known as the black tiger shrimp Penaeus mondon, and known locally as bagda. Freshwater prawn Macrobrachium rosenbergii, known locally as golda, are also cultured in Bangladesh.) The activity made significant investments in the hatchery sector for bagda in the Cox’s Bazar area. The initial approach, commencing in 2012, was to test PLs for the presence of WSSV using PCR in laboratories in Cox’s Bazar and at Khulna University supported by the activity. Over one billion PL were tested by PCR between 2012 and 2015. In 2015, the activity reduced the number of hatcheries it worked with from 22 to four that agreed to produce only negative PCR-tested shrimp seed. These four hatcheries produced 18 million PCR-tested seed and no non-tested seed using individual spawning tanks. Use of PCR-tested seed increased survival on farms using traditional methods from 10-12 percent to 25-30 percent. On semi-intensive farms, survival is about 80 percent using tested seed. The AIN activity evaluated the effect of quality seed support and training on shrimp production. Shrimp production by farmers receiving training was 20 percent greater than by non-trained farmers. Shrimp production and gross margin of ponds managed by farmers that did not receive PCR-tested seed was similar, whether farmers received training or not. About 45 percent greater shrimp production was achieved by farmers receiving PCR-tested seed, irrespective of whether farmers received training. These results indicate that seed quality was a critical factor and that training had a smaller effect in traditional shrimp farming. While PCR testing was proceeding, AIN activity staff were involved with a technical committee that developed a protocol for the importation of brood shrimp, a process that was completed in 2013. One hatchery (MKA Hatchery) was selected as a pilot facility, and the AIN activity provided support to purchase SPF brood shrimp from a breeding center in Moana, Hawaii, but these did not perform well due to the stress of transportation. In 2014, the hatchery received 4,000 SPF PL from Moana. The activity assisted MKA Hatchery with support for plastic liners for ponds used to grow the PLs to sub￾adults and then adults. In 2015, the hatchery was successful in producing SPF PL from the resulting brood shrimp produced from the original shipment of 4,000 SPF PL. Now, MKA Hatchery imports batches of PLs three times per year from Moana to serve as brood shrimp. The growth rate of SPF seed is 40 percent greater than locally available shrimp breeds. In addition, SPF shrimp are certified free of not just WSSV but 11 other shrimp viruses and pathogens. Use of SPF seed allows farmers to reduce the duration of the culture period, thereby permitting in some cases the 23 production of two shrimp crops, and ensures some shrimp harvest even in the event of a disease outbreak due to the short rearing period. Farmers using PCR-tested or SPF shrimp seed have more confidence that production will be successful and are more willing to invest in feed and other inputs. However, farmers are not willing to pay much extra for PCR-tested or SPF PL. To increase the availability of quality seed to farmers, the activity provided no-cost PCR testing to the shrimp hatcheries. For seed destined for activity farmers, hatcheries agreed to provide new transport boxes and to pack PL at a lower density than normally used. Seed testing and reduced density in transport boxes improved the condition of PL. The activity also assisted farmers by developing linkages to shrimp seed traders. Farmers paid the market price for tested PL in advance. The use of quality seed by shrimp farmers is hampered by logistical difficulties of transporting seed from Cox’s Bazar to growing areas in the Khulna,Satkhira and Bagerhat regions, a journey that requires 30-36 hours. Most hatcheries in Cox’s Bazar sell seed to agents who transport seed to growing areas in Khulna,Satkhira, and Bagerhat by plane via Jessore or by truck or microbus directly to growing areas. Some hatcheries produce nauplii in Cox’s Bazar and ship these to seed nurseries in the Khulna area for production of PL that are then transported to growing ponds and ghers. The current demand for shrimp seed in Bangladesh is estimated to be at most 8 billion. Only about 142 million SPF seeds were produced in 2015-2016, stocked by more than 17,000 farmers. The target production of SPF seed by the MKA Hatchery for the next production season is 400-500 million. Even with this anticipated increase in hatchery production by MKA Hatchery, the current supply of SPF seed is not adequate to meet demand. Improved Access to Feed In Bangladesh, fed ponds (with either commercial or farm developed feed) account for only 5 percent of total pond area but 36 percent of fish production (Belton et al. 2011). In remote areas, feed is not available at a reasonable cost. Despite increasing availability of high-quality commercially produced fish feeds, farmers are suspicious about the composition of these feeds and lack trust in feed companies. Some farmers hold the view that making their own feed using ingredients they procure on their own will result in a feed of better quality. With the objective of increasing awareness of the value of using feed, the AIN activity developed 61 community feed centers (CFCs), also known as semi-auto feed mills, in partnership with qualifying commercial farmers. The activity used the following criteria to select feed millers for receipt of feed milling equipment:  Has his own ponds to use at least 5 tons of feed per year;  Is surrounded by at least 100 farmers who can use feed produced in a semi-auto feed mill;  Is well-accepted in the community;  Is literate (Class Eight);  Has some business experience;  Is capable and interested in investing in a feed mill;  Has space to set up a mill and a feed drying area;  Has access to electricity or a diesel engine to power milling equipment;  Is located in a remote area where commercial feeds are not available;  Has good access to transportation infrastructure (roads);  Expresses willingness to serve as a trainer; and  Is non-political. The activity provided machinery on a 50 percent cost-share basis and training support for mill operation, and feed formulation and manufacture. The small mills include a grinder, mixer, pelletizer, and, in some cases, a drier provided by the activity, and have a capacity of 200-400 kg/day of feed produced using 24 ingredients procured in local markets. Based on 36 responses to the telephone survey questionnaire, the average mill owner produced 23 tons of feed in 2016. Relative to records of output collected by the activity, survey respondents appeared to exaggerate feed production from CFCs. In general, mills appear capable of producing 20-25 t/yr. Of 61 CFCs established by the activity, 16 (26 percent) had produced no feed, or less than one ton of feed, as of June 2016, some because owners only recently established their mills, but others because of a lack of capacity or commitment. Since inception of the CFC initiative, 17 mills (28 percent) produced 1- 10 tons each, an output that can be considered satisfactory. About 28 CFCs (46 percent) are working well, producing more than 10 tons of feed since inception, including three mills that produced more than 50 tons since inception. The characteristics of successful CFCs and CFC owners include:  Individual capacity – relatively well-educated, has money to invest;  Personally motivated by the profit potential associated with producing feed; mill owner also owns ponds to demonstrate results of feeding to other farmers;  Strategic location – many nearby farmers, poor availability of commercial feed, electricity available;  Good linkages with farmers and/or local NGOs operating in surrounding area;  Trustworthy, respected in community, considered a lead farmer; and,  Has interest in training farmers, functions as an effective LSP. Each feed mill owner provides feed to 20-50 farmers (around 2000 total), about half of who are associated with the AIN activity. . Mill owners sold about half (54 percent,range: 10–100 percent) of feed produced to area farmers. Many mill owners are willing to sell small quantities of feed to farmers, normally 5-10 kg, but as little as 1-2 kg. Since the AIN activity demonstrated the value of supplementary feeding, the use of sinking or floating pellets has increased significantly. In areas where AIN has been active, LSPs and CFC owners report that 80-90% of farmers are using pelleted feed. Around 20 percent of semi-auto feed mills are not functioning well. The reasons for relatively poor performance include:  The AIN activity did not apply established selection criteria with sufficient rigor to identify potentially poor-performing mill owners;  The CFC did not provide the benefits expected by mill owners;  Working capital was not available to mill owners;  A joint family business structure resulted in dilution of profits;  The costs of CFC-produced feed were greater than the costs of commercial feed because mill owners purchase small quantities of ingredients to produce each batch of feed; and,  High-quality commercially produced feeds are offered on credit to farmers. One challenge faced by mill owners is the difficulty in drying feed during the monsoon season, corresponding to the main season of fish production. In response, the activity has assisted some mill owners with procurement of feed dryers. Also, mill owners purchase small batches of ingredients at prices that are not competitive with those paid by large feed mills that gain a price advantage by making bulk ingredient purchases. In addition, the seasonal availability of certain ingredients affects ingredient price. In locations closer to the coast, the relatively low price and good availability of dried fish to make fishmeal makes small-scale feed production competitive with commercial-scale production. Some ingredients (e.g., rice polish) are also used in poultry and dairy feeds and thus compete with use in fish feeds. 25 Household Aquaculture Consumption, Nutritional Awareness, and Women’s Empowerment There were 65,709 households, representing 8,798 ha of fish ponds that received training in aquaculture best management practices and household nutrition. Between 74 and 84 percent of farmers increased fish consumption as a result of the activity’s effortsactivities (Table 7). Annual per capita fish consumption by AIN activity beneficiaries was 19.5 kg, in contrast to that of non-beneficiaries, which was 16.8 kg. The “desirable dietary intake” of fish recommended by the GoB is 18-22 kg. The increase in dietary diversity of women as a result of AIN’s work was a modest 5 to 8 percent from the low baseline of 3.97 (of 9). FGDs indicated a relatively small difference in fish consumption between AIN activity beneficiaries and non-beneficiaries, with most consuming fish five days per week and two to three times per day. Activity beneficiaries were more likely to sell fish than non-beneficiaries. Table 7. AIN activity Indicators for PE Question 4 (Household Nutrition) The AIN activity promoted a polyculture production system in household ponds consisting of large fish (carps and tilapia), nutrient-rich small indigenous fish species (SIS, especially mola, Amblypharyngodon mola), and dike cropping of vegetables, especially orange sweet potato (OSP). Over the life of the AIN activity, 4,700 kg mola were distributed to 9,100 households. Further sharing of mola from farmer to farmer occurred but was not measured. About 2.8 million OSP vines were distributed to 26,890 households. In addition, activity farmers distributed OSP vines to neighbors for cultivation. Ninety-six percent of the targeted pond dike area is now under vegetable cropping. Activity households produced about 8-10 kg/yr of mola, about half of which is consumed and half sold. Mola and other SIS are consumed whole (with head and bones), which makes the fish a package of protein and micronutrients, especially calcium and vitamin A. Consumption of whole mola and OSP was greater among AIN activity beneficiaries than among non-beneficiaries. AIN beneficiaries now eat whole mola, whereas most non-beneficiaries do not consume mola or remove the head prior to consumption. A cast net is considered the most convenient way to catch mola; however, proper deployment of this gear requires considerable skill. Attraction of mola by water flow or light is not effective. Promotion of a cost-effective mola gill net by the AIN activity increased women’s participation and resulted in a consumption rate of mola (3 days/wk) that was 1.9 times greater than that of farmers without gill nets (1 26 day/wk). Regular harvesting of mola with a gill net increased household fish consumption. Training and demonstration of this production model has resulted in an increase in the availability and consumption of nutrient-rich local food sources that are easy to harvest in small portions for household consumption. About 78 percent of activity farmers increased their fish consumption compared with before the activity. AIN activity beneficiary farmers consume fish about six days/week and twice/day, and in some families, three times/day. Non-beneficiary farmers in the same area consume fish four days/week and twice/day. About half the mola produced in AIN household ponds is consumed by household members. There are fluctuations in fish consumption between activity years because farmers decided to sell more or less fish and use more or less for consumption, as dictated by the need for food or cash. Some farmers prefer to sell large high-value fish (or even mola) and purchase low-cost smaller fish for household consumption or farmers simply have a greater need for cash at a particular time and consume less fish. AIN activity beneficiaries placed a greater priority on household consumption compared with non-activity farmers, who placed greater priority on sale. In general, activity interventions resulted in a significant increase in the knowledge of the nutritional benefit of consuming SIS and OSP and their utilization for good absorption by the body (Table 8). In particular, the nutritional value of consuming whole mola (with head) is understood by activity beneficiaries, and local recipes have been adapted to incorporate whole mola. The household aquaculture initiative created increased knowledge of production, consumption, and nutritional awareness in household members, especially women. Women are now aware of the value of a diversified diet. Most women consume grains, leafy green and orange-colored vegetables, fruits, legumes, eggs, milk, and milk products. The dietary diversity index of AIN beneficiaries increased slightly (by 0.18) compared with the baseline (Table 7). AIN farmers are aware of the nutritional benefits of OSP but most non-AIN farmers were not very familiar with the crop. Awareness of proper sanitation (handwashing) was much greater in AIN activity farmers compared with non-activity farmers. 27 Table 8. Number of focus group discussions with AIN activity beneficiaries and non￾beneficiaries in which a particular item was mentioned There was a strong involvement of women in small-scale aquaculture such as mola production, homestead gardening, and dike cropping in areas where the AIN activity was active. During FGDs, most women indicated that their personal status had improved from involvement with the activity. They now tend to play a stronger role in economic decision-making in the household, including those decisions concerning education of children, attend social functions without any barriers, allow guests in the home, and attend religious functions. The higher status of women has improved childhood nutrition because such women have better nutritional status, better care for themselves, and provide higher-quality care to their children. In Barisal, the district with the greatest number of household farmers, most ponds were derelict when the AIN activity began activities in the area. After receiving technical support and training, a large number of ponds are now in production, increasing the availability of fish for household consumption and household income from fish sales. Many non-beneficiaries are now interested in growing fish in their underutilized ponds. Non-beneficiaries who visited mola ponds are encouraged by seeing the production and value of mola and have indicated an interest in stocking mola in their ponds. Farmers who received mola from the AIN activity shared small quantities of live mola with neighboring farmers. Effectiveness of Behavior Change Communication Approaches Information, Education, and Communication (IEC) and promotional materials were designed and produced to increase nutritional awareness of household members, particularly women and children. Nutrition training covered many topics, including the benefits of dietary diversity and fish consumption, and the importance of nutrition and fish consumption for pregnant and lactating women. Hygiene was another important topic because sick people cannot utilize food efficiently. The following BCC materials were developed by the AIN activity: 28  Leaflet on mola fish culture  Leaflet on household aquaculture and dike cropping management  Leaflet on role of fry traders for extension of aquaculture  Leaflet on GAP in shrimp farming management  Leaflet on GIFT monosex Tilapia commercial farming management  Leaflet on Vietnamese koi nursery management  Leaflet on orange sweet potato  Poster on small fish consumption  Poster on essential nutrients  Poster on vegetable cultivation and consumption  Poster on essential nutrients and essential hygienic action  Poster on gender in aquaculture  Farmers guidebook on household aquaculture and dike cropping management (for LSPs and farmers)  Farmers guidebook on commercial carp culture and dike cropping management (for LSP & farmers)  Farmers guidebook on GAP in shrimp farming and dike cropping management (for LSP & farmers)  Farmers guidebook on GAP in golda farming and dike cropping management(for LSP & farmers)  Farmers guidebook on tilapia and koi farming management and dike cropping management (for LSP & farmers)  Food cards  Festoons (7 types)  Food plate  Photo booklet  Promotion of mola gill net  Community radio (for coastal areas)  T-shirt  Sticker on handwashing and other 10 types technical message carrying stickers  30 types of TIN plate carrying different technical messages  Gambhira (Barisal)  Video  Cartoon (2D animation) on mola gill net and handwashing The AIN activity training unit distributed training materials to farmers, CFs, and LSPs. Festoons were distributed to LSPs, four Aquaculture Service Centers, local tea stalls, training centers, and fish and shrimp input sellers. Stickers were distributed to each of the fish farmers’ groups, and the field trainers (activity field staff) held discussions about the stickers in their training sessions. As a result of these activities, activity beneficiary farmers increased their knowledge about aquaculture practices and about human nutrition, especially the value of fish consumption in the household. The AIN activity used various approaches to transmit behavior change communication messages to household members. The methods used included LSPs, printed materials, outreach workers, dramatic performances and video shows, through which the direct activity and wider communities were targeted with electronic BCC messages. More than 200 news articles were published in different local and national newspapers of Khulna and Barisal on dike cropping, mola nutrition, and other nutrition-related subjects. These approaches reinforced messages transmitted through direct training activities. Food plates, pictorial festoons, and food cards were the most effective BCC interventions to increase food consumption and dietary diversity, and to create awareness of the value of balanced diets in 29 women and children (Table 8). Pictorial festoons and food plates were distributed only to LSPs (not activity farmers) for demonstration to the audience during training. The community radio (Radio Nalta, Satkhira / and Amadi, Koyra Bazar) and television broadcasts were not effective or well-accepted by the AIN activity beneficiaries. Training Training was the most important and effective BCC activity in the AIN activity. Most beneficiaries considered training to be the most valuable assistance received from the activity. AIN supported the training of approximately 65,000 household farmers, 85 percent of whom are women, and about 55,000 commercial farmers (50,000 shrimp, 5,000 fish), most of whom are men. The training received by household and commercial farmers on improved practices and household nutrition was similar. Based on KIIs with Directors General of DoF and with AIN activity hub managers, strong extension capacity at the grassroots level is generally lacking in Bangladesh. DoF officers were stretched thin and unable to provide services to all beneficiaries, especially at the union level. The activity conducted a Training of Trainers (ToT) approach to maximize reach. Technical staff from the AIN activity trained staff of implementing partner NGOs (e.g., CODEC, SpeedTrust, and Save the Children [2011-2013]), DoF extension agents operating at the upazila and union levels, and local service providers (LSPs), discussed below. Training took the form of classroom training, hands-on training, problem-solving and troubleshooting, and on-the-spot inspection of activities. The activity used active demonstrations with small-scale constructed models placed on the ground in training sites. Demonstration ponds were used to model a range of polyculture systems featuring large fish, SIS, and dike cropping, and field days were held to demonstrate system features and results. The activity also provided study visits, especially for hatchery owners, to aquaculture farms and hatcheries within Bangladesh and regionally prominent aquaculture countries (e.g., Thailand and Vietnam). Farmers were trained in groups of 20-30 and training was designed so that each group received 14 training sessions to reinforce messaging. As an example, in 2012, farmers received eight half-day training sessions from AIN activity technical staff. In 2013, farmers received four half-day training sessions from AIN activity technical staff. In 2014, farmers received two half-day training sessions from AIN activity staff. During 2015, LSPs were recruited and provided with additional training. In 2016, farmers received one half-day training provided by LSPs. One of the key approaches taken by the AIN activity was the use of LSPs to transmit information about improved practices and nutrition to activity farmers. These LSPs often have deep local knowledge of the communities they serve, especially at the union level, where government extension support is scarce. Beneficiaries can easily contact LSPs to provide training, guidance, and information about household aquaculture and nutrition. In Barisal, the division with the greatest number of household aquaculture farmers, 45,000 farmers were trained through implementing partners and 25,000 were trained through LSPs. LSPs were selected from activity beneficiaries who had previously received training and showed technical aptitude and skill in public speaking. The criteria for selecting LSPs included being an activity beneficiary, having received training, having an aquaculture business or farm, and having positive acceptance by the local community. There was an initial pool of training LSPs as identified by NGO training partners and activity training staff. Based on the outcome of interviews with potential LSPs, the number was reduced to 1,275 based on those judged most likely to be effective, or about one-third of the initial pool. The goal was to have about 20 LSPs operating in each upazila in the activity area and to link these with DoF Upazila Fisheries officers. The 1,275 LSPs trained 74,044 farmers, 82 percent of whom were new to the AIN activity (WorldFish 2016). Thus, each LSP served about 60 farmer beneficiaries. The most effective LSPs were those with 30 direct linkages to farmers, especially fish nursery pond owners (33 percent), fish seed traders (20 percent), and feed or other input traders. Support of LSPs by the activity allowed them to expand their business through strengthening linkages to their farmer-buyers of fish seed and farming inputs. Development and support of LSPs started late in the activity (mid-2015). Based on KIIs and FGDs with LSPs and AIN activity staff with knowledge of the initiative, the process of selecting LSPs was not sufficiently rigorous and was rushed. There is considerable uncertainty about the sustainability of training activities by LSPs. About half of the LSPs are active and effective, and the other half became LSPs opportunistically and are not likely to continue to function independently as LSPs. Based on KIIs with AIN activity hub managers, as few as 10-20 percent and as many as 50 percent of LSPs will continue training activity because it is perceived to be in their own business interest. Business motivation is considered to be the primary motivation, and the activity encouraged this. The activity also encouraged motivation of LSPs through a sense of altruism toward others. An LSP coach is an extension employee of the activity. The coach takes training courses, but also gives training courses. The focus of the LSP coach is on group formation, assisting with providing basic information in trainings and conducting a quarterly survey. The original idea was to have one LSP coach per upazila. In Faridpur, one group of LSPs has formed an association of 300 members. The association (Quality Seed Group) wants to brand improved seed under the name of the upazila. In Barisal, an LSP committee of 50-60 members encourages members to conduct trainings and discusses marketing and production constraints. Institutional Capacity Building / Policy and Regulatory Reform The objective of this component was to provide support for the development and reform of policies and regulations and to build institutional capacity within public and private sectors that can sustain and grow aquaculture within the FtF area. Although Institution and Policy was an explicit component of the AIN activity, there was relatively little activity in this area. The emphasis of activities was on visible implementation of other, more field-oriented activity components: seed quality, household aquaculture, and commercial aquaculture. The activity established two indicators for the institution and policy component, both related to preparation of policy briefs regarding the Fish Feed Act of 2010 and Fish Feed Rule of 2011, and the Fish Hatchery Act of 2011 and Fish Hatchery Rule of 2011. These policy briefs have not been prepared, although there are plans to do so in the short time remaining. The AIN activity arranged a series of informational and awareness raising meetings, workshops, and dialogues with DoF and other stakeholders regarding implementation of policy and regulatory measures of the fish feed and fish hatchery acts. There is an umbrella MOU between WorldFish and DoF, and the activities of the AIN activity fall under this umbrella. The partnership between WorldFish and DoF appears to be strong and meaningful. The DoF assisted the AIN activity with the location and procurement of seed that would eventually become the quality broodstock that was distributed to carp hatcheries and served as the foundation stocks for the RGIP. The AIN activity is seeking 6 ha of pond space from DoF to conduct genetic improvement programs for rohu, catla, and silver carp, but no decision has been made by the Director General of DoF. The AIN activity established a “model tilapia hatchery” and a TBN on a DoF farm. AIN provided technical capacity development through training of DoF field officers and including technical staff as trainers in beneficiary training activities. AIN provided support to DoF officials for study tours within the country and to aquaculture facilities in nearby countries. AIN supported development of laboratory facilities for PCR testing of shrimp seed and evaluation of feed ingredients. Field-level DoF technical extension agents supported AIN activities to train farmers about better practices. In general, the activities of the AIN activity were in alignment with DoF objectives, and the mission of the AIN activity and DoF is the same, with the same targeted beneficiaries. 31 The Bangladesh Fisheries Research Institute (BFRI) is the research arm of the Ministry of Fisheries and Livestock. The organization has conducted aquaculture research on emerging and established species and has established a genetic improvement program for tilapia since the original introduction of GIFT fish to Bangladesh in 2005. The BFRI has considerable capacity and the AIN activity has provided equipment for facilities and training of staff to enhance that capacity. A Memorandum of Agreement was signed between WorldFish and BFRI in 2014, three years into the activity. One of the objectives of the MOA was “to conduct research for development of improved strains.” There appears to have been little to no cooperation on this, as well as other objectives. There are obvious potential synergies between the activities of the AIN activity and BFRI, particularly with regard to the genetic improvement program, but so far the linkage has not been made. Genetic improvement is a continuous process, building on progress, and so, once the AIN activity ends and WorldFish is no longer active in Bangladesh, only a local institution can carry out this work and, if supported with sufficient resources, BFRI is the organization best positioned to take ownership of the genetic improvement program. The activity provided some assistance to implement the Aquaculture and Fisheries MOU between Bangladesh and India on cooperation with fish seed improvement and germplasm conservation and management. The effort did not come to fruition, but not for lack of effort by the activity. The intention of the cooperation was to import the Jayanti strain of rohu to BFRI. CONCLUSIONS Based on the key findings of the performance evaluation, the following conclusions can be drawn:  The AIN activity was effective in achieving most activity objectives because of successful implementation of the approach to provide improved quality seed and training of farmers and other value-chain actors on better management practices. Adoption of Improved Practices  At the hatchery level, AIN activity interventions have had strong multiplier effects along forward value-chain linkages with benefits accruing to nursery pond operators, seed traders, and farmers.  AIN activity interventions in the carp hatchery sector have resulted in increased hatchery output, thereby increasing hatchery revenue, and have enhanced the availability of improved quality carp seed throughout a broader area of the FtF ZoI than before initiation of the activity.  AIN activity interventions in the tilapia hatchery sector have increased supply of genetically improved tilapia seed to fish farmers throughout the country.  AIN activity interventions in the brackish water shrimp (bagda) hatchery sector have increased the supply of seed that tested negative for WSSV to shrimp farmers in the FtF ZoI.  AIN activity interventions in the freshwater prawn (golda) hatchery sector have not improved hatchery output or the consistency of seed supply.  At the nursery level, AIN activity interventions, particularly through backward linkage to activity-supported hatcheries and training on better management practices, have increased nursery pond output and the availability of quality seed to farmers. Genetic Improvement Programs and Improved Seed and Feed  Given the demonstrated productivity gains associated with the introduction of improved-quality carp broodstock by the AIN activity, and assuming support will continue beyond the activity end, genetic improvement programs for rohu, catla and silver carp are very likely to lead to substantial long-term benefits across the farmed carp value chain.  Rotational breeding by TBN owners will allow preservation but not improvement of the 32 performance value of genetically improved tilapia introduced by the AIN activity into the FtF ZoI and elsewhere in the country. Managers of TBNs have expressed willingness and some have demonstrated capability to execute proper management of this system. However, without activity oversight, there is uncertainty whether all TBN managers will continue to make the effort to rotate males. Technically the TBNs are capable of preserving genetic quality, but eventually they will need to replace broodfish with those with further performance improvements.  Improved productivity of household and commercial ponds has increased household income and fish consumption. It is not clear whether activity interventions have increased on-farm employment.  AIN activities in the commercial shrimp farming sector have increased the availability of quality seed. However, for reasons related to the structure and function of shrimp marketing and processing for export and the global nature of shrimp trade, shrimp farmers have yet to realize the full benefits of activity interventions.  PCR-tested and SPF seed is more expensive and farmers are currently unwilling to pay more for it, despite demonstrated benefits. We believe that farmers will come to accept the value of SPF seed and be willing to pay more than the current price for regular seed because a) farmers will experience much greater return from using SPF seed compared to either PCR-tested or normal seed and b) expanding the number of hatcheries producing SPF seed to increase SPF seed supply will bring prices down.  As indicated by increased sales of carp and tilapia seed produced from quality broodfish, carp and tilapia hatchery owners were able to improve and strengthen their businesses.  AIN activity interventions into hatcheries throughout the FtF ZoI have allowed decentralization of the carp fry market from Jessore.  Although the business success of owners of community feed centers (semi-auto feed mills) has been mixed, the broader objective of demonstrating the value of using feed to farmers as an approach to intensify production has been realized. Household Aquaculture and Nutrition  AIN efforts to develop household aquaculture have resulted in a modest improvement in household fish consumption but have had a more pronounced effect on household income. Although distribution of mola by the AIN activity was intended primarily to improve household fish consumption, activity beneficiaries sell a significant portion of the mola harvest and use the proceeds to purchase lower-value fish or other necessities.  The AIN activity has encouraged and accelerated the active participation of women in aquaculture. The activity has enhanced income, nutrition, and well-being of women and children in activity beneficiary households. Training, Institutional Capacity Building, and Policy Development  AIN activity BCC interventions were effective in increasing awareness of nutrition and sanitation among beneficiary households. Activity training materials were well designed and easily adopted by LSPs for effective farmer training.  Training efforts of the AIN activity were exemplary in terms of approach, reach, topic coverage, and follow-up. The LSP concept was valuable and effective, but should have been implemented earlier in the life of the activity. 33  In general, communication, cooperation, and coordination between the missions and activities of the AIN activity and DoF was meaningful and substantive, especially at the district and upazila level.  Providing policy support to the GoB was not a priority of the AIN activity, despite an explicit activity component on institution and policy. The major emphasis of the activity’s work was placed on fish and shrimp seed quality, and training. 34 Recommendations The evaluation team makes the following recommendations for implementing similar activities in the future: Evaluation Question 1 – Technology and Management Practices at the Hatchery, Nursery, and Farm Level  At the farm level, training and access to improved fish seed have resulted in improved productivity of household ponds for carp and SIS and commercial ponds for carp, tilapia, and shrimp. Improved productivity of household and commercial ponds has increased household income and fish consumption. It is not clear whether activity interventions have increased on￾farm employment. Intervention at “upstream” parts of the value chain has significant multiplier effects “downstream.” Thus, future activities can make cost-effective investments in seed improvement that have strong impacts on fish production, household income, and fish consumption.  To develop traceable shrimp production at the farm level, farmers need to be organized to coordinate production using SPF seed and better practices to manage disease risk. An extension effort is needed to motivate farmers to use SPF seed and implement better management practices. Future activities can serve a support role by coordinating linkages, either through direct support or in a facilitation role, between buyers and groups (clusters) of trained farmers conforming to a set of standardized production practices on disease management, water management, stocking density, and feeding. If this can be implemented, it opens the door to certification according to internationally recognized, third-party eco-labeling schemes. Evaluation Question 2a – Effectiveness of Private Sector Engagement Approach for Genetic Improvement of Tilapia and Carp  For rohu GIP, private sector engagement extended only to leasing ponds. The activity managed all other activities. For catla GIP, farmers were given responsibility for fish husbandry and did not fulfill their obligations. For tilapia, the tilapia breeding nucleus and tilapia satellite hatchery approach using private sector farmers was effective in dissemination of improved tilapia seed to farmers, but activity oversight and coordination was necessary. In general, private sector engagement in the genetic improvement program was superficial. Meaningful engagement of the private sector would require identification of qualified hatcheries with significant technical capacity to serve as partners. The genetic improvement programs need to move from private￾sector facilities to government facilities to obtain better leasing terms. Evaluation Question 2b – Assurance of Continuation of Genetic Improvement Programs  Genetic improvement of the major species cultured in Bangladesh is a long-term and ongoing effort, essential for future aquaculture expansion in the country. Although the carp genetic improvement programs were not part of the initial activity proposal, the work conducted thus far by the AIN activity should be continued. The overall effort should receive timely and serious consideration for continued support. The relatively modest investment in genetic improvement programs has the potential to provide substantial benefits in the future, but sustained funding is needed to continue the work. Although implementing partners such as WorldFish are uniquely positioned to coordinate and support such activities, ultimate ownership and management of genetic improvement programs must come from a qualified GoB agency, such as BFRI or DoF, or from qualified private partners that are willing to provide improved quality seed as a business activity, because these entities will outlive development project life cycles and genetic improvement programs should be ongoing and open ended. Public-private partnerships between 35 BFRI and private hatcheries can reduce the business risks associated with long-term genetic improvement programs. Although there is an informal commitment from DoF and BFRI to implement genetic improvement programs, there is a need to formalize the arrangement as part of the national aquaculture development strategy. In general, improvement and strengthening of the relationship between WorldFish and BFRI is needed. Evaluation Question 2c – Ensuring Genetic Purity  By definition, genetically improved species are not “pure” but are selected. However, preserving genetic quality (culture performance) can be maximized by appropriate broodstock management practices to limit inbreeding (e.g., rotation among hatcheries). Genetic improvement is a continuous and ongoing process and periodic introductions may accelerate potential gains. Genetic improvement programs need to preserve genetic diversity (maintenance of rare alleles) while also selecting for better performance. Evaluation Question 3a – Improved Access to Quality Fish Seed (Carp and Tilapia)  Intervention in the key upstream value chain areas of seed and feed was cost-effective and had strong multiplier effects through the value chain. Through improvement in seed quality, hatchery practices, and nursery pond practices, the AIN activity was successful in stimulating aquaculture production in the FtF area. Anecdotal information from several KIIs with DoF officials and value￾chain actors (hatchery and TBN owners, small-scale feed manufacturers, feed dealers) suggests that the abundant supply put downward pressure on fish prices, especially for tilapia, in multiple locations of the FtF ZoI. Future activities should place additional emphasis on downstream parts of the value chain, including aggregation, marketing, processing and value-added product development, transportation, cold storage, and export. Basic small-scale processing by local women’s groups has the potential to improve regional marketing.  Branding of quality carp and tilapia seed should be pursued. Two activity hubs (Faridpur and Barisal) initiated the formation of Quality Seed Groups toward the end of the activity (mid￾2016). These groups consist primarily of nursery pond operators and seed traders, but also include owners of hatcheries that produce quality seed. Quality Seed Groups have the potential to develop a unified strategy to market quality seed in specific geographic areas, at the upazila level, to improve profitability through the seed production and supply parts of the value chain. Creation and strengthening of Quality Seed Groups should be encouraged. Evaluation Question 3b – Improved Access to Quality Shrimp Seed  In the case of brackish water shrimp, more hatcheries (beyond the one currently operating) that can produce SPF nauplii and PLs are needed to increase the availability of quality seed to farmers because the current supply is insufficient. One option is to establish the one operating hatchery producing SPF seed as a breeding nucleus, producing nauplii or PL to supply satellite hatcheries, similar to the model applied by the AIN activity to the distribution of improved tilapia seed. Alternatively, additional hatcheries could employ the same SPF broodstock procurement model as demonstrated by the MKA Hatchery. A final alternative is to begin the process of establishing specific pathogen free or resistant shrimp broodstock using existing stocks of shrimp in natural waters, although considerable time would be required to develop genetically improved stocks of bagda locally. Evaluation Question 3c – Improved Access to Quality Feed  The community feed centers (semi-auto feed mills) were most successful as an approach to demonstrate and raise awareness of the value of using feed. The business success rate of community feed centers suggests the need for more rigorous vetting of mill owners. If this initiative will be continued in future activities, removal of the equipment subsidy and identification of the criteria that best predict success should be emphasized in selecting individuals and sites for feed mill location. Joint procurement of feed ingredients among mills has 36 the potential to lower costs and improve competitiveness with commercially available feed. Resolution of the applicability of the GoB Feed Law to community feed centers is needed because they are currently not in compliance. District-level licensing for small feed mills that use most of the feed produced on their own farms is one approach that could be evaluated.  Farmers are increasingly using manufactured feed to produce fish and shrimp. High-quality, extruded feed is now widely available in the FtF region. Future activities should consider additional approaches to production intensification, including water exchange and pond circulation and aeration. Additional emphasis on the propagation and culture of air-breathing fish, such as climbing perch (Vietnamese koi), Pangasius (pangas), Clarias (magur), and snakehead (Channa), is warranted. Evaluation Question 4a – Relevance of Behavior Change Communication Approach  The FtF approach of improving income and nutritional outcomes through a market-driven, value-chain approach is generally supported by activity outcomes, although specific, complementary interventions were needed to improve nutritional outcomes at the household level. The training component of the AIN activity was exemplary in terms of developing technical capacity among NGO implementing partners and DoF staff, LSPs, and, ultimately, farmers. Training was the most important and effective BCC activity. The LSP concept is good in theory, but the initiative began too late in the activity cycle for LSPs to fully realize their potential. The activity was able to identify and conduct initial technical training and evaluation of LSP effectiveness. The activity ended before activity staff could follow up after the first year of LSP activity to more fully develop the capacity of LSPs as trainers. Future activities should either begin this kind of initiative earlier in the activity cycle or the LSPs identified and trained in the AIN activity should be recruited and used in future activities as farm-level trainers in household aquaculture, but more rigorous vetting and selection is needed to identify effective LSPs. Follow￾up by activity staff is then necessary to provide guidance, coordination, and development of additional capacity. Evaluation Question 4b – Gaps in Design and Implementation of BCC  Most of the BCC materials were distributed at the LSP level. Only leaflets and booklets were distributed at the household level. Efforts should be made to distribute the most effective BCC intervention materials (especially food plates and food cards) to households. There was not as much follow-up training on nutritional messages compared with aquaculture production techniques. More emphasis can be given to nutritional messaging if this area has a higher priority in future activities.  BCC materials (poster/festoons) should be pictorial and long-lasting in a calendar form (maybe up to full activity period). Materials should be distributed to every household. BCC messages should be delivered frequently through media campaigns, television commercials, street shows, road shows, drama, and mini-cartoons for children. Nutrition sessions should be provided to primary school teachers in the activity area. Evaluation Question 5 – Policy and Regulation  WorldFish should continue to collaborate with DoF to discuss and review policies associated with implementation of the Fish Feed Acts and Rules, the Fish Hatchery Acts and Rules, and the requirement for PCR testing or use of SPF shrimp seed. Based on these discussions, policy briefs should be written that provide a framework for implementation of the regulations. Adoption of new policies that stimulate growth and investment of the shrimp farming sector, such as mandating stocking of SPF shrimp PL, should be pursued. Other Recommendations  The shrimp depot (collection center) and the associated e-traceability system is a worthwhile initiative but was not well-received by local shrimp buyer/processors because the shrimp costs 37 more to produce but processors get the same price. As the global demand for traceable shrimp increases, markets are developed, and Bangladesh shrimp become more competitive in those markets, the initiative will have more relevance. Future activities can assist processors and exporters to identify and develop suitable markets for brackish water shrimp grown in the FtF area, thereby increasing demand for traceable shrimp. Increasing the number of buyers of shrimp from collection centers with the e-traceability system will ensure that farmers will receive a better price for their products. The number of collection centers should be expanded with locations selected on the basis of centers of clustered production.  To develop traceable shrimp production at the farm level, farmers need to be organized to coordinate production using SPF seed and better practices to manage disease risk. Future activities can serve a support role by coordinating linkages between buyers and groups (clusters) of trained farmers conforming to a set of standardized production practices on disease management, water management, stocking density and timing, and feeding. Formal linkages between specific growers and specific processors is needed, and future activities can directly facilitate this connection. If traceable shrimp production can be implemented, it opens the door to certification according to internationally-recognized, third-party ecolabeling schemes.  The AIN activity conducted a range of efforts that were peripheral to the stated objectives and main approaches. In most cases, these initiatives did not require significant activity resources to implement, and did not detract from overall activity success, but they diverted attention from AIN priorities. Future aquaculture efforts should maintain clear focus on AIN activity goals and objectives. 38 Lessons learned Carp and tilapia hatcheries: Investment in the upstream seed-production node of the value chain had considerable positive downstream impacts with nursery pond operators and household and commercial farmers. The aggregate increase in fish production through intervention in production and distribution of improved seed was greater than that achieved through training farmers in improved pond management practices, which required considerable mobilization of manpower and activity resources. Improving water quality with the “oxygen towers” was a key technological intervention that allowed increased survival and hatching rates of carp and tilapia spawn. Broodstock replacement: There has been little incentive for hatchery owners to replace broodstock with better performing strains. Hatchery owners need to be convinced of the profit potential of using better quality broodstock to invest in long-term planning for broodstock replacement. Positive feedback from growers that increases demand from hatcheries with better quality broodstock can result in market differentiation and higher prices for quality seed. That can serve as an incentive for hatchery owners to implement regular exchange of hatchery broodstock for higher quality strains. Genetic improvement programs: Genetic improvement programs (GIPs) are complicated and require technical specialists to operate properly. In the case of rohu, it takes three generations, or about 8-10 years, before a “remarkable” improvement in culture performance can be achieved. In the case of the catla GIP, the private sector approach was a failure because the fish hatchery owner did not adequately care for the fish. In the case of the rohu GIP, private sector involvement extended only to leasing of ponds. Although the appropriate government agency (BFRI) has the mandate to conduct genetic improvement, in Jessore, the institute does not have the capacity, in terms of facilities and skilled technicians, to manage the rohu GIP. The approach of activity-supported technical staff to conduct a GIP, coupled with facilities located on a well-managed private farm, was functional and effective. Presumably facilities located on a GoB fish farm would also be suitable, and it would cost less to lease ponds. Tilapia breeding nucleus: Tilapia hatchery owners operating a TBN have experience with the male rotation program to preserve genetic quality. It is unclear if TBN owners will conduct the rotation without activity oversight and without the motivation of a premium price. External support in the form of advice from technical specialists will likely be required to assure that the rotation program is conducted properly. PCR-testing program for shrimp seed: The PCR-testing laboratories at BFRI in Cox’s Bazar and at Khulna University were important investments in demonstrating the value of using tested seed. However, the program was not sustainable. Hatcheries did not want to discard infected PLs and most did not have the infrastructure to conduct “one mother – one tank” breeding. Farmers were not willing to pay much extra for tested PLs, despite the demonstrated advantage of using tested seed, and this would ultimately affect the ability of PCR testing laboratories to be self-sufficient without government (DoF) subsidies. The SPF shrimp seed provided good results at the farm level in terms of faster growth, less disease, and greater production. It also allows the production of two shrimp crops per year. Farmers using tested PLs had more confidence and were more willing to invest in feed and other inputs. The future of brackish water shrimp farming in Bangladesh will likely be based on SPF seed, but a massive scale-up of SPF seed production in many more hatcheries is needed. E-traceability / collection center initiative: The intention of this pilot initiative was to provide small-scale producers with a mechanism to market small batches of shrimp in a way that would provide product traceability to the farm. The e-traceability system did not work in part because buyers do not require traceability for the international markets they serve. The cost of the system is high, but this can be 39 reduced to take advantage of scale by engaging more farmers. Training of buyers, depot managers, and small-scale farmers is needed to develop the ICT skills necessary to operate the system. Farmers and buyers do not yet understand and appreciate the long-term benefits of e-traceability. Community feed centers: The community feed center (semi-auto feed mill) initiative was effective in increasing awareness of the value of using feed. The main interest of mill owners was to produce feed for their own farms. Although the activity established a set of criteria for potential mill owners and required a 50 percent cost-share on equipment, this was not sufficient to guarantee success of every mill. A deeper understanding of the motivations and capacity of potential mill owners can contribute to a successful outcome. Mills located in more remote areas, where commercially produced feeds are not available or are costly, are more likely to be successful. Mill owners considered to be farmer-leaders in their community also increases the likelihood of success. LSP trainers: Over time, the AIN activity progressively shifted the responsibility for training from activity staff to NGO partners and ultimately to private-sector local service providers (LSPs), who are deeply embedded in and well accepted by the communities they serve. LSPs were recognized as providing a valuable mechanism for delivering training content to beneficiaries, including many outside the activity. Many of these LSPs provided training as a value-added embedded service in addition to their livelihood as nursery pond operator, seed trader, feed trader, or feed miller. Most LSPs view ongoing training to be in their business interest. Unfortunately, training and empowerment of LSPs only occurred late in the activity (mid-2015). As can be expected, LSPs represent a range of skill and ability. LSPs should be included in future activities, but a rigorous selection process is needed and then close oversight and further capacity building of successful LSPs by activity staff should be provided. Nutrition promotion: A major component of the activity was to improve nutritional outcomes of activity beneficiaries, particularly with respect to fish consumption. The activity had only a modest effect on increasing dietary diversity because people already consume fish at high rates. Nonetheless, the nutritional promotion was effective in increasing nutritional awareness among women, particularly about the benefits of whole small fish consumption to increase intake of calcium and vitamin A. The mola gill net was easy to use and resulted in increased fish consumption in households with gill nets. Most fish produced in household ponds are consumed by household members and only surplus fish are sold at final harvest. The FtF approach of improving income and nutritional outcomes through a market-driven, value-chain approach is generally supported by activity outcomes, although specific, complementary interventions in the form of nutrition training were needed to improve nutritional outcomes at the household level. 40 Annexes Annex I: Reference List Annex II: Evaluation Statement of Work Annex III: Evaluation Design and Methods Matrix Annex IV: Data Collection Instruments Annex V: FtF Districts Where the AIN activity Was Active and Where the AIN-PE Team Visited Annex VI: List of Persons Contacted Annex VII: Disclosures 41 ANNEX I: EVALUATION STATEMENT OF WORK Scope of Work (SOW) for the Aquaculture for Income and Nutrition (AIN) Activity’s Final Performance Evaluation ACTIVITY TO BE EVALUATED Activity Name Aquaculture for Income and Nutrition (AIN) Contract Number Contract number: EEM-G-00-04-00013-00 Original Activity Date September 2011 – September 2016 Original Funding $25,000,000 Implementing Partner WorldFish Center I. BACKGROUND Aquaculture for Income and Nutrition (AIN) is a five-year activity that started in September 2011 and is being implemented by WorldFish Center, in collaboration with the Ministry of Fisheries and Livestock, Government of Bangladesh (GOB). This is one of the key activities under “Development Objective (DO) 2: Food Security Improved,” of USAID Bangladesh’s Country Development Cooperation Strategy (CDCS). DO2 is also the flagship DO for the Feed the Future (FtF) strategy and its objective in Bangladesh: “Availability, Access, and Utilization of Domestically Produced and Nutritious Foods Increased.”1 Thus AIN becomes an important part of the FtF program, addressing vulnerable household constraints to food availability, access, and utilization. The AIN activity directly contributes to “IR 2.1. Sustainably Increase Agricultural Productivity” under the DO2 result framework, as well as to the FtF goal of “sustainably reducing poverty and hunger by tackling their root causes and employing proven strategies for achieving large-scale and lasting impact.” AIN extends aquaculture in the FtF Zone of Influence (ZoI) thereby covering 20 southern districts in the Barisal, Khulna, and Dhaka divisions of Bangladesh. Specifically, AIN aims to increase aquaculture productivity through development of hatcheries and nurseries, dissemination of improved fish and shrimp seed, enhancing farm management skills of the smallholder farmers, promoting new technologies to expand commercial aquaculture, developing backward and forward market linkages, supporting policy reform, and building capacity of the public and private sectors, resulting in increased farmers’ productivity and revenue. Increased productivity at the household level will result in more food for consumption and more produce for sale. This will contribute to a further adoption of these improved technologies and management practices, and will result in an overall increase in rural households’ income, thereby sustainably reducing poverty. The premise or development hypothesis of USAID-AIN activity is that: 1) Systemic improvements to the hatchery sector will increase seed quality and ultimately increase the productivity and profitability of existing farming systems; 2) Training for small and marginal women farmers on improved pond management and horticulture, combined with use of improved seed, increase farm productivity, household incomes, and household nutrition; 3) Improved seed quality, coupled with improved management practices and stronger linkages with value chain actors, will increase the profitability and output of commercial aquaculture and stimulate employment on-farm and in associated value chains; and 1USAID/Bangladesh Multi-Year Feed the Future Strategy 2010-2015 42 4) Support for regulatory reform in the hatchery and feed sectors will provide an enabling environment for the growth of commercial aquaculture enterprises. Results Framework of AIN activity II. ACTIVITY COMPONENTS AIN is composed of four components listed below: Component 1: Disseminating high-quality fish and shrimp seed This component supports developing and distributing improved quality and/or genetically improved quality of tilapia, carp (rohu, silver carp, and catla), prawn, and shrimp seeds to farmers in Khulna, Barisal, and Dhaka divisions, which is the basis for a 40% - 50% increase in fish and shrimp yield in the FTF zone over 5 years and thereby will provide increased benefits to approximately 1.7 million farm households by 2016. Particularly, it assists government brood centers and private hatchery operators in sourcing quality brood stock, establishing management systems to maintain and develop seed quality, and accelerating distribution of improved quality fish, shrimp, and prawn to farmers. Component 2: Improving the nutrition and income status of farming households Through this component, AIN assists the training of approximately 65,000 households on improved aquaculture and horticulture production practices to increase household income and nutrition. The primary focus of this training is introducing WorldFish aquaculture technologies into existing livelihood initiatives. Production of small 43 indigenous fish species and orange sweet potato (OSP) are also covered in the training in order to improve the nutritional status of the farm households. Component 3: Increasing investment, employment, and fish production through commercial aquaculture This component stimulates investments, employment, and income in the aquaculture sector of Bangladesh. It provides technical support to approximately 50,000 shrimp and prawn farmers and 15,000 fish farmers, and assists them with adopting high-value commercial fish and shrimp cultural practices and identifying new options for brackish water aquaculture. These initiatives are proposed to reduce pressure on natural sources and lead to increased income. Component 4: Reforming policy and regulations, and building institutional capacity to support sustainable aquaculture growth This component assists government in the implementation of existing policy and regulatory measures in the Hatchery and Feed sectors, and facilitates the recommendation and adoption of new policies and supports for institutional capacity building, which include assistance in implementing the Aquaculture and Fisheries Memorandum of Understanding (MOU) between Bangladesh and India on cooperation in fish seed improvement and germplasm conservation and management. The component is also working to improve the capacity of private sector associations and businesses, and is assisting both public and private bodies in ensuring their effective participation in the activity. AIN works closely with several other USAID activities. AIN provides trainings on aquaculture and related small business development to Local Service Providers (LSP) supported by the Climate Resilient Ecosystems and Livelihoods (CREL) activity under the Mission’s DO 4 portfolio. AIN also supports beneficiaries of the Strengthening Partnerships, Results, and Innovations in Nutrition Globally (SPRING) activity to access good-quality fish seeds. In addition, AIN regularly coordinates with Horticulture, SHIKHA, and Integrated Agriculture and Health-Based Interventions (IAHBI) activities on nutritional training materials and nutrition messaging. Among other donor activities, AIN is collaborating with Katalyst on disease problem identification in freshwater prawn hatcheries and on capacity building in the fish feed sector. The Katalyst activity uses a pro-poor market development approach and is funded by the UK Department for International Development, The Swiss Development Cooperation, the Embassy of the Netherlands, and the Canadian International Development Agency. AIN has close working relations with the Department of Fisheries (DoF) under the Ministry of Fisheries and Livestock (MoFL). The AIN activity is not USAID/Bangladesh’s first attempt to develop the aquaculture sector. The Mission has been designing and implementing activities for aquaculture sector development for the past two decades. Of particular relevance are Development of Sustainable Aquaculture Activity (DSAP) (2000-2005, $5.5 million), Agro-based Industries and Technology Development activities (ATDP I and II) (1995-1999, $10 million and 2000-2005, $10 million), Shrimp Quality Support Activity (SQSP I and II), and, more recently the Greater Harvest for Economic Returns in Shrimp (GHERS) component (2008-2012) under the Poverty Reduction by Increasing the Competitiveness of the Enterprises (PRICE) (2008-2013). III. EVALUATION OBJECTIVE The objective of the evaluation is to assess the effectiveness of the programmatic approaches of the activity. It will explore and look at the activity’s key critical interventions that USAID/Bangladesh should continue in its future aquaculture activity and other relevant EG/FtF activities. It will also assess and recommend if any successful pilot 44 interventions can be scaled up for greater impact in the future across the sector. The findings will also discuss the obstacles and challenges the activity has faced, and how it addressed those challenges during implementation. IV. EVALUATION QUESTIONS The contractor’s evaluation of AIN must be comprehensive. The evaluation team will ensure that data is gender disaggregated and subject to gender analysis. The following questions should be addressed in order of priority: 1. To what extent have AIN interventions contributed to improved usage of aquaculture technologies and management practices both at farm (small and large), nursery, and hatchery level? 2a. How effective was the private sector engagement approach for genetic improvement of tilapia and carp (rohu, catla, and silver) fishes? 2b. What can be done to ensure continuation of the current genetic improvement efforts until they are ready for commercialization (large-scale spawn production for commercial culture)? 2c. What can be done to ensure the sustainability (maintenance) of the genetic purity of these fish species? 3. To what extent have AIN interventions promoted and strengthened aqua businesses and enterprises for improved quality of fish/shrimp seed stock and feed? What were the challenges these enterprises faced in carrying out their businesses? 4. How relevant was the activity’s behavior change communication (BCC) approach in increasing the consumption of nutrient-rich food at the household level with a particular focus on women and children? What were the gaps identified in the design and implementation of BCC, and how can BCC be further improved? 5. What were the key lessons learned from the institutional capacity building, and policy and regulatory reform interventions of the activity? V. EVALUATION METHDOLOGY The M&E contractor for the USAID Bangladesh Economic Growth (EG) Office, Accelerating Capacity for Monitoring and Evaluation (ACME), will carry out this evaluation. Based on the SOW from USAID, ACME will recruit and manage the evaluation team. The evaluation work plan and evaluation protocol drafted by the evaluation team will be reviewed and approved by USAID. The detailed methodology of the evaluation will be designed by the evaluation team in the work plan; this will include presentation of a data collection matrix that will explicitly link evaluation questions to particular data collection approaches and data sources. It is strongly suggested that the evaluation team use a mixed-method approach utilizing both quantitative (including non￾experimental design) and qualitative analysis (key informant interviews, stakeholder interviews/mini-surveys, and focus group discussions). The evaluation team should develop the best evaluation design methodology in light of the evaluation questions, timeframe, budget, data collection requirements, quality of existing data sources, and potential biases. If a survey sample is chosen as a research method to be employed in this evaluation, the sample of activity beneficiaries must be representative of AIN beneficiary population. The evaluators should utilize several different, yet complementary and inter-related forms of gathering information/data. Suggestions are provided below: Document Review: Evaluation team members will review documents throughout the evaluation process, including program reports and relevant studies, to ensure that comprehensive and grounded best practices will be identified. 45 Key Informants Interview: The team will conduct one-on-one interviews with a variety of stakeholders including USAID and AIN stakeholders, and other activities supporting aquaculture sector development and nutrition programming. This will be done administering a semi-structured or structured questionnaire. Self-assessment: The implementing partners (IPs) will respond to a self-assessment either through a questionnaire or standard interview checklist put together by the evaluation team and approved by USAID before use. Expert Opinion Survey: Utilizing expert opinion is a technique used increasingly in the market system sector. The Evaluation team, with approval of USAID, can apply this method as well. Participatory Rural Appraisal (PRA): This form of survey method will be applied to seek views and opinion of the local community on the issues related to program interventions. This qualitative survey requires sufficient triangulation. 2 Focus Group Discussions (FGD): FGDs (small groups of 6 to 10 people) will be used to lead open discussion through a skilled moderator to gather semi-structured qualitative data. The preselected participants will discuss issues and concerns based on a list of key themes drawn up by the moderator. No more than 10 questions will be addressed by a group. These sessions will encourage free-flowing discussion about the activity. Mini Survey: This type of survey is small (30-40 participants) and can be performed rapidly in the field without analytical software such as SPSS or a large questionnaire. The sample size is not statistically significant; however, this type of analysis can be used to triangulate with other methods. Because of the small sample size this type of survey can be implemented quickly when time and resources are constrained. Social and Economic Analysis: This type of analysis might require a sample survey. Due to time and resource constraints, the survey should be kept to a manageable size since it is neither possible nor desirable to carry out a complete census for social and economic analysis. The team should select a sample that is representative of the beneficiary population and triangulate the findings from this survey with other sources. Regardless of data collection and analysis methods, USAID requires qualitative and quantitative data disaggregated by gender. Methodological limitations and challenges for this evaluation are expected to include:  Ensuring that samples of interview sources are sufficient to support evaluation findings;  Taking systematic actions to counter any biases in (a) reporting by data collection sources and (b) interpretations of collected data by the evaluation team; and ensuring “actual” results can be measured, which will only be possible if data can be gathered and analyzed beyond respondent perceptions. All the methodological strengths and weaknesses should be explicitly described in the evaluation report. VI. EXISTING SOURCES OF INFORMATION The evaluation team should consult a broad range of background documents apart from activity documents provided by USAID/Bangladesh. These should include, but are not limited to, documents such as the National Aquaculture Policy and Plan, Country Investment Plan, and Bangladesh Feed the Future Multiyear Strategy, as well 2 Davis (Davis, a.C.S , ‘Participatory Rural Appraisal’, Rural Travel and Transport Program, 2001, 5.6.a, TRL Limited) reveals that PRA triangulation is often carried out in groups of at least 3 (three) to increase the credibility of each survey technique. 46 as other relevant national strategies and policies. USAID, ACME, and the AIN activity will provide the assessment team with a package of briefing materials, including:  The agreement for AIN activity  M&E plan of AIN activity  Activity quarterly and annual reports, work plans, and management reviews developed as part of routine monitoring  Training reports  DQA reports  USAID/Bangladesh Country Development Cooperation Strategy 2011-2016 (Public version)  USAID Bangladesh DO:2 PMP  Aquaculture sector gap analysis report  Technical/Activity proposal  Any other relevant reports (inception, expo, fare, etc.)  Case and success story, if any  Any research/study reports (gap study, sector study, BCC study, etc.), if any  Name of partners with contact details - International and National NGOs  MOU/Agreement signed with GoB wings - DoF under the MoFL  MOU/Agreement signed with other USAID’s IPs - CREL, SHIKHA, SPRING, IAHABI, and Katalyst  Performance Indicator Tracking Table (PITT)  Beneficiary database  Training database  Training modules  Training reports, if any  M&E/data collection tools  Mid-term Evaluation report, if any  GIS Maps - demonstrated the zone of influence (ZOI)  Year-wise activity expenditure  One-pager/brochure/pamphlet, if available 47  Activity Organogram VII. EVALUATION TEAM COMPOSITION The evaluation team will consist of the following three members: Team Leader/Evaluation Specialist: The Team Leader should have a post-graduate degree in agricultural economics, agribusiness management, aquaculture enterprise development, economics, or an applicable social sciences field. The Team Leader will be an international/Bangladeshi national and should have 10 years of work experience. S/he should have demonstrated experience of leading at least two evaluations of activities with similar scope and complexity within past 5-6 years. S/he should have extensive experience in conducting quantitative and qualitative evaluations and strong familiarity with business and value/supply chain development. The Team Leader must be familiar with USAID regulations and systems, including Feed the Future performance monitoring guidance, gender policies and guidance, activity management, budgeting, and financial analysis and reporting. Experience in international donor development program management and overseeing multiple program areas simultaneously is preferred. Excellent oral and written skills in English are required. Relevant experience in Bangladesh or South Asia preferred. The Team Leader will provide overall leadership for the team, and s/he will finalize the evaluation design, coordinate activities, arrange periodic meetings, consolidate individual input from team members, and coordinate the process of assembling the final findings and recommendations into a high-quality document. S/he will lead the preparation and presentation of the key evaluation findings and recommendations to the USAID/Bangladesh team and major stakeholders. Aquaculture Specialist: The Aquaculture Specialist will have a post-graduate degree in aquaculture science or any other relevant subject. S/he will be a Bangladeshi national with at least 8 years of experience in aquaculture activity development, management, and evaluation. S/he will have very good understanding about the aquaculture input/output market system. S/he should have demonstrated knowledge of the latest developments in advancing good/best practices in aquaculture sector development that reaches women, youth, and the very poor. S/he will participate in team meetings, key informant interviews, group meetings, and site visits, and draft the sections of the report relevant to his/her expertise and role in the team. S/he will also participate in presenting the report to USAID or other stakeholders and be responsible for addressing pertinent comments provided by USAID/Bangladesh or other stakeholders. Familiarity with USAID regulations and systems, including Feed the Future performance monitoring guidance, evaluation guidance, gender policies and guidance, and activity management, is preferred. Nutrition Specialist: The Nutrition Specialist must have a post-graduate degree in one of the disciplines relevant to the following areas: Public Health, Food and Nutrition, or Nutrition and Food Science. S/he will be a Bangladeshi national with a minimum of 5 years of experience in integrating nutrition components into the development activities and programs, preferably integrating nutrition education and Behavior Change Communication (BCC) within the scope of improved aquaculture and horticulture production practices. S/he will assess the approaches of integration of BCC and its nutritional impacts on beneficiaries. S/he will focus on nutrition-related findings and analysis, and will draft the sections of the report relevant to her/his expertise and role in the team. S/he will participate in presenting the report to USAID or other stakeholders and be responsible for addressing pertinent comments provided by USAID/Bangladesh or other stakeholders. 48 S/he will have excellent understanding of the country’s overall nutrition context and gender dynamics in nutrition programs. Familiarity with USAID regulations and systems, including Feed the Future performance monitoring guidance, evaluation guidance, gender policies, and guidance and activity management, is preferred. Conflict of Interest All evaluation team members will provide a signed statement attesting to a lack of conflict of interest, or describing an existing conflict of interest relative to the activity being evaluated. USAID will provide the conflict of interest forms. VIII. DELIVERABLES All deliverables are internal to USAID and the Evaluation Team unless otherwise instructed by USAID. Evaluation deliverables are indicated below: Evaluation Team Planning Meeting: During the meeting, the team should review and discuss the SOW in its entirety, clarify team member roles and responsibilities, prepare the work plan, develop data collection methods, review and clarify any logistical and administrative procedures for the assignment and instruments, and prepare for the in-brief with USAID/Bangladesh. Work Plan: Prior to initiation of the evaluation activities, the evaluation team will provide a detailed initial work plan to the ACME COR. The ACME COR will provide any necessary feedback or edits to the work plan, after which the evaluation team will have three days to submit a final version of the document. The initial work plan will include (a) a task timeline, (b) a description of the methodology to answer each evaluation question, (c) team responsibilities, (d) document review process, (e) key informant and stakeholder meetings, (f) site visits, and (g) draft and final report writing. The work plan will be submitted to the ACME COR at USAID/Bangladesh for approval no later than the 5th day after the evaluation team arrives in Bangladesh. In-briefing Meeting: The evaluation team will meet with USAID/Bangladesh within two working days of the International Team Leader’s arrival in the country. Evaluation Design Matrix: A table that lists each evaluation question and the corresponding information sought, information sources, data analysis methods, and limitations. The matrix should be finalized and shared with USAID/Bangladesh before evaluation field work starts. It should also be included as an annex in the evaluation report. Data Collection Instruments: Development and submission of data collection instruments to USAID/Bangladesh during the design phase and after the evaluation is completed. Regular Updates: The Evaluation Team Leader will brief the ACME Contracting Officer’s Representative (COR) and any other designated evaluation POC on progress with the evaluation on at least a weekly basis, in person or by electronic communication. Any delays must be quickly communicated to USAID/Bangladesh to allow quick resolution and to minimize any disruptions to the evaluation. Emerging opportunities to strengthen the evaluation should also be discussed with USAID/Bangladesh as they arise. Preliminary Draft Evaluation Report: The evaluation team will submit a Preliminary Draft Evaluation Report to the ACME COR five working days before the Mission debriefing. Within three working days after receipt, USAID staff will provide preliminary comments prior to the Mission debriefing. Debriefing with USAID: The evaluation team will present the major evaluation findings to USAID/Bangladesh through a PowerPoint presentation before the team leader departs the country. The debriefing will include a 49 discussion of achievements and issues as well as any preliminary recommendations. The team will consider USAID comments and incorporate them in the Draft Evaluation Report. Draft Evaluation Report: A draft report on the findings and recommendations should be submitted to USAID/Bangladesh within 10 business days after the departure of the international team leader. The written report should clearly describe findings, conclusions, and recommendations. The draft report must be of high quality with no grammatical errors or typos. A report is high quality when it represents a thoughtful, well-researched and well￾organized effort to objectively evaluate what worked in the activity, what did not work, and why. The draft report must have well-constructed sentences that are presented in a way that clearly presents findings, conclusions, and recommendations. The report should answer all the evaluation questions, and the structure of the report should make it clear how the questions were answered. The draft report must meet the criteria set forth under the Final Report section below. USAID will provide comments on the draft report within 10 working days of submission. Final Evaluation Report: The evaluation team will submit a Final Evaluation Report that incorporates Mission comments and suggestions no later than 10 working days after USAID/Bangladesh provides written comments on the Draft Evaluation report. The format of the final report is provided below. The report will be submitted electronically in English. The total length of the final report, excluding references and annexes, should be no more than 30 pages. A second version of the evaluation report, excluding any potentially procurement-sensitive information, will be submitted (also electronically, in English) to Development Experience Clearinghouse (DEC) and disseminated among implementing partners and other stakeholders within 10 days following approval from USAID. All quantitative data, if gathered, should be 1) provided in an electronic file in an easily readable format; 2) organized and fully documented for use by those not fully familiar with the activity or the evaluation; 3) owned by USAID and made available to the public, barring rare exceptions. A thumb drive with all the data could be provided to the ACME COR. The final report will be edited and formatted by the evaluation team and provided to USAID/Bangladesh 5 working days after the Mission has reviewed the content and approved the final revised version of the report. IX. REPORTING REQUIREMENTS The total length, excluding references and annexes, should not be more than 30 pages. The following content (and suggested length) should be included in the report: Table of Contents List of Acronyms Executive Summary – Concisely state the activity purpose and background, key evaluation questions, methods, most salient findings, and recommendations (2-3 pp.). 1. Introduction – Country context, including a summary of any relevant history, demography, socio-economic status, etc. (1 pp.). 2. The Development Problem and USAID’s Response – Brief overview of the development problem and USAID’s strategic response, including design and implementation of the AIN activity and any previous USAID activities implemented in response to the problem, (2-3 pp.). 3. Purpose of the Evaluation – Purpose, audience, and synopsis of task (1 pp.). 50 4. Evaluation Methodology – Describe evaluation methods, including strengths, constraints, and gaps (1 pp.). 5. Findings and Conclusions – Describe and analyze findings for each evaluation question using graphs, figures, and tables, as applicable. Include data quality and reporting system that should present verification of spot checks, issues, and outcomes. Conclusions should be credible and should be supported by the findings (12-15 pp.). 6. Recommendations – Prioritized for each evaluation question; should be separate from conclusions and be supported by clearly defined set of findings and conclusions. Include recommendations for future activity implementation or relevant program designs and synergies with other USAID activities and other donor interventions as appropriate (3-4 pp.). 7. Lessons Learned – Provide a brief of key technical and/or administrative lessons on what has worked, what has not worked, and why for future activity or relevant program designs (2-3 pp.). 8. Annexes – Include statement of work, documents reviewed, bibliographical documentation, evaluation methods, data generated from the evaluation, tools used, interview lists, meetings, FGDs, surveys, and tables. The Evaluation Design Matrix must be presented as an annex to the report. Annexes should be succinct, pertinent, and readable. They should also include, if necessary, a statement of differences regarding significant unresolved difference of opinion by funders, implementers, or members of the Evaluation Team on any of the findings or recommendations. The report format should be restricted to Microsoft products, and 12-point type font should be used throughout the body of the report, with page margins of one inch at top/bottom and left/right. X. TIMELINE AND LEVEL OF EFFORTS Work will be carried out from September to November 2016. At this point in time, we anticipate that the evaluation would begin September 17, after the ‘Eid vacation, with field work completed the second week of October and final report and closeout concluding mid-November. The anticipated September 17 in-country start date provides more than one month to finalize, clear the SOW, recruit and approve the consultants, and complete background research prior to approval. We anticipate that September 9–16 would be an unproductive period in which to conduct the evaluation due to a general inability to access stakeholders during this period. Below is an estimate of the evaluation level of effort (LOE): A. Preparatory Work Evaluation Team Leader Aquaculture Specialist Nutrition Specialist Comprehensive document collection and review 5 days 5 days 5 days Travel to Bangladesh 2 days 0 days 0 days Team planning meeting and meeting with USAID/Bangladesh 1 day 1 day 1 day Development of evaluation design and work plan (concurrent with document review and initial meetings). 2 days 2 days 2 days Develop preliminary interview instruments and begin scheduling key interviews 3 days 3 days 3 days 51 B. Data Gathering In-country information and data collection – includes interviews with key informants (stakeholders and USAID staff) and site visits 18 days 18 days 18 days C. Data Analysis/Drafting Report Data analysis in preparation for presentations 3 days 3 days 3 days In-country discussion with USAID and presentation of preliminary analysis and draft of final report 1 day 1 day 1 day Presentation of preliminary results and recommendations to the USAID/Bangladesh Mission and relevant stakeholders 1 day 1 day 1 day Depart Bangladesh 2 days 0 days 0 days Analysis of data and draft of final evaluation report – draft must be submitted within 10 working days after the departure of international team lead 10 days 5 days 4 days Evaluation team has 10 days to finalize report and 5 working days of HQ quality control 5 days 1 day 0 days TOTAL 53 40 38 A six-day workweek (Saturday-Thursday) is authorized for the evaluation team while in Bangladesh. The evaluation team will submit a work plan as part of the evaluation methodology proposal with timeline and develop a Gantt chart displaying the time periods during which activities occur. Pre-departure arrangements should include travel approval, airline tickets, visa, lodging, work facility and vehicle transport arrangements, dates for meetings with USAID/Bangladesh EG staff and key contacts, in-country travel agenda, and accommodations. XI. Logistic Support USAID/Bangladesh’s ACME activity will be responsible for all off-shore and in-country administrative and logistical support, including identification and fielding appropriate local staff. They will take care of arranging and scheduling meetings, international and local travel, hotel bookings, working/office spaces, computers, printing, and photocopying. A local administrative assistant/coordinator may be hired to arrange field visits, local travel, hotel, and appointments with stakeholders and provide translation. 52 ANNEX II: REFERENCE LIST Belton, B., M. Karim, S. Thilsted, K.M. Jahan, W. Collis and M. Phillips. 2011. Review of aquaculture and fish consumption in Bangladesh. Studies and Reviews 2011-53. Penang: The WorldFish Center. Debnath, P., S.H. Khan, M. Karim, B. Belton, C.V. Mohan and M. Phillips. 2015. Review of the history, status and prospects of the black tiger shrimp (Penaeus monodon) hatchery sector in Bangladesh. Reviews in Aquaculture 8:301-313. Debnath, P.P., M. Karim. H.J. Keus, C.V. Mohan and B. Belton. 2016. Effects of White Spot Disease and bio-security on shrimp farming in Bangladesh. Fish Pathology 51:S60-S65. WorldFish (2012-2016). Aquaculture of Income and Nutrition Quarterly and Annual Reports (1-19). Years 1-5. WorldFish. 2014. Fish Consumption Related to Aquaculture. Report on Nutrition Focus Group Discussion. Aquaculture for Income and Nutrition (AIN) Activity, WorldFish. WorldFish. 2014. Aquaculture for Income and Nutrition 12th Quarterly Report (Annual Progress Report Year 3). WorldFish. 2015. Aquaculture for Income and Nutrition 16th Quarterly Report (Draft Annual Progress Report Year 4). WorldFish. 2016. Evaluation Team Presentation 9-10b update 26-10.ppt 53 ANNEX III: EVALUATION DESIGN MATRIX Evaluation Question Evaluation Sub-questions Data Source (Organization/Individual) Data Collection Method 1 To what extent have AIN interventions contributed to improved usage of aquaculture technologies and management practices both at farm (small and large), nursery, and hatchery level? To what extent have AIN interventions contributed to improved usage of aquaculture technologies and management practices at the hatchery level? Quarterly and Annual Reports M&E Reports Annual Workplans Implementation Plans AIN Staff USAID GoB Literature Review Key Informant Interviews Site Visits/Direct Observation Survey Questionnaire To what extent have AIN interventions contributed to improved usage of aquaculture technologies and management practices at the nursery level? Quarterly and Annual Reports M&E Reports Annual Workplans Implementation Plans AIN Staff USAID GoB Literature Review Key Informant Interviews Site Visits/Direct Observation Survey Questionnaire To what extent have AIN interventions contributed to improved usage of aquaculture technologies and management practices at the farm (small and large) level? Quarterly and Annual Reports M&E Reports Annual Workplans Implementation Plans AIN Staff USAID GoB Literature Review Key Informant Interviews Site Visits/Direct Observation Survey Questionnaire 2a How effective was the private sector engagement approach for genetic improvement of tilapia and carp (ruhu, catlta, and silver) fishes? Quarterly and Annual Reports M&E Reports Annual Workplans Implementation Plans AIN Staff USAID GoB Literature Review Key Informant Interviews Site Visits/Direct Observation Survey Questionnaire 2b What can be done to ensure continuation of the current genetic improvement efforts until they are ready for commercialization (large scale spawn production for commercial culture)? Quarterly and Annual Reports M&E Reports Annual Workplans Implementation Plans AIN Staff USAID GoB Literature Review Key Informant Interviews Survey Questionnaire 2c What can be done to ensure the sustainability (maintenance) of the genetic purity of these fish species? Quarterly and Annual Reports M&E Reports Annual Workplans Implementation Plans AIN Staff USAID GoB Literature Review Key Informant Interviews Survey Questionnaire 3 To what extent have AIN interventions promoted and strengthened aqua businesses and enterprises for improved quality of fish/shrimp seed stock and feed? What were the challenges these enterprises faced in carrying out their businesses? To what extent have AIN interventions promoted and strengthened aqua businesses and enterprises for improved quality of fish seed stock? What were the challenges these enterprises faced in carrying out their businesses? Quarterly and Annual Reports M&E Reports Annual Workplans Implementation Plans AIN Staff USAID GoB Literature Review Key Informant Interviews Site Visits/Direct Observation Survey Questionnaire To what extent have AIN interventions promoted and strengthened aqua businesses and enterprises for improved quality of shrimp seed stock? What were the challenges these enterprises faced in carrying out their businesses? Quarterly and Annual Reports M&E Reports Annual Workplans Implementation Plans AIN Staff USAID GoB Literature Review Key Informant Interviews Site Visits/Direct Observation Survey Questionnaire 54 To what extent have AIN interventions promoted and strengthened aqua businesses and enterprises for improved quality of feed? What were the challenges these enterprises faced in carrying out their businesses? Quarterly and Annual Reports M&E Reports Annual Workplans Implementation Plans AIN Staff USAID GoB Partner NGOs Literature Review Key Informant Interviews Site Visits/Direct Observation Survey Questionnaire 4 How relevant was the project’s behavior change communication (BCC) approach in increasing the consumption of nutrient-rich food at the household level with a particular focus on women and children? What were the gaps identified in the design and implementation of BCC; and how can BCC be further improved? How relevant was the project’s behavior change communication (BCC) approach in increasing the consumption of nutrient-rich food at the household level with a particular focus on women and children? Quarterly and Annual Reports M&E Reports Annual Workplans Implementation Plans AIN Staff USAID GoB Partner NGOs? Literature Review Key Informant Interviews Focus Group Interviews What were the gaps identified in the design and implementation of BCC; and how can BCC be further improved? Quarterly and Annual Reports M&E Reports Annual Workplans Implementation Plans AIN Staff USAID GoB Literature Review Key Informant Interviews Focus Group Interviews 5 What were the key lessons learned from the institutional capacity building, and policy and regulatory reform interventions of the project? Quarterly and Annual Reports M&E Reports Annual Workplans Implementation Plans AIN Staff USAID GoB Literature Review Key Informant Interviews 55 ANNEX IV: DATA COLLECTION INSTRUMENTS A. Key Informant Interview Templates by Stakeholder/Beneficiary Group GoB (DoF, BFRI) Officials and University Faculty The purpose of this interview is to learn more about your experiences with and recommendations related to your involvement with the AIN activity.  Tell me about your role and activities.  How long have you been in your current position?  What is your primary responsibility and duty in your job?  What kind of aquaculture activities are you involved in?  Tell me about your involvement with the AIN activity.  What do you know about AIN activities in your area?  Do you consider the activity to be aligned with your mission and activities?  Did you have direct involvement with AIN activities? If so, please describe them.  What forms of support did your department receive from the AIN activity?  Are you satisfied with their support?  What is your view about the effectiveness of the AIN activity?  From what you know, what were the most effective strategies and activities of the AIN activity? What were the least effective?  What do you know about the “Feed the Future” program?  Have you ever participated in any training program of the AIN activity?  Do you use any resource/extension materials produced by AIN to facilitate your work? If not, why?  What is your view about the quality and effectiveness of these training materials?  What are some of the major obstacles and challenges to developing aquaculture in the region?  Do you have any thoughts or suggestions about the key programs, strategies, or activities that you think would be particularly effective for improving aquaculture in the region?  Are there other people in the area whom we should speak with about the AIN activity? Local Service Providers (esp. nursery pond operators) General background:  Please tell us about your work.  How long have you lived in this community?  Did you receive a guidebook on household aquaculture or a food plate?  Tell us about its importance and how it has benefited you.  To what extent are training media attractive to community members?  What problems do you usually face in delivering nutrition messages to others using training tools? (Why or why not?)  What is your view/opinion about the perceptions of community members about the different training tools used to provide messages about nutrition? (Ask about problems/which messages they followed willingly/which they showed rigidness.)  Which training tools are most effective at communicating their message about nutrition? (Mention all BCC tools: leaflets, posters, food card, food plate, stickers, festoons, audio/visual, and radio.) Which are least effective?  Are the messages able to influence changes in behavior (fish consumption) by members of the community? Do the messages appeal to the beliefs of the target audience?  What were your experiences about how people benefit from different types of strategies to stimulate fish consumption? 56  How are people in the community influenced by these messages?  What are your recommendations to improve the messages?  How can we make all those messages more effective?  Based on your experience, do you think there are any changes needed in the pictorial materials? If Yes, which one and why? What would be the possible changes?  Do you have anything else to add about training in nutrition? About their business:  How many years have you been nursing fish?  How many ponds do you have now? What is the total area?  What species do you grow in nursery ponds?  When did you become an LSP for the AIN activity?  What kinds of training have you received from the AIN activity?  What was the most important thing you learned in training that improved output from your nursery ponds?  Have you received any other assistance from the AIN activity?  What is your view about the value of the training you have received from the AIN activity?  Which training materials were most valuable to you?  As an LSP, you had an agreement with the activity to provide four training sessions to farmers or others in the community. Now that the activity is ending, will you continue to provide training to the farmers you provide services to? Why? Thank you for your time. Hatchery Association Leaders The purpose of this interview is to learn more about your experiences with and recommendations related to your involvement with the AIN activity.  Tell me about the hatchery association.  How long have you been the leader of the association?  When was the association first established?  How many members are in the association?  Is the membership decreasing, stable, or increasing?  What are the benefits of being a member of the association?  Do all hatcheries in the area belong to the association?  For those who do not belong, what is the reason?  How much seed produced by members of your association are transported outside of southwest districts?  Are there contract nursery operators who are nursing seed from association members?  Did the association receive assistance from any source/s for organizational strengthening? If so, what forms did this assistance take?  Tell me about your involvement with the AIN activity?  Do you consider yourself to be a partner or beneficiary of the AIN activity?  What forms of support did the organization receive from the AIN activity?  Did you sign an MOU or MOA with the AIN activity?  What types of activities represented your obligations to the activity?  If you are a partner of the AIN activity, are you satisfied with their support?  What is your view about the effectiveness of the AIN activity?  From what you know, what were the most effective strategies and activities of the AIN activity? What were the least effective?  What do you know about the “Feed the Future” program? 57  What are some of the major obstacles and challenges to developing aquaculture in the region?  Do you have any thoughts or suggestions about the key programs, strategies, or activities that you think would be particularly effective for improving aquaculture in the region?  Are there other people in the area whom we should speak with about the AIN activity? Dr. M. Ghulam Hossein – WorldFish Genetics Consultant The purpose of this interview is to learn more about your experiences with and recommendations related to your involvement with the AIN activity.  When did you join WorldFish as a consultant to AIN?  We learned that a consultant named Francois Rajt worked on hatchery issues prior to your arrival. Was their continuity with his work when you started working with WorldFish on the AIN activity? How did that transition occur?  Do you maintain any relationships with hatcheries in the FtF region? Do hatchery operators seek your advice even though you no longer work for the AIN activity?  When did the Rohu Genetic Improvement Program start?  What was your input into the establishment and management of that program?  When do you plan to release improved rohu to commercial hatcheries?  What can be done to ensure continuation of the GIP until improved fish are ready for release to hatcheries?  Once fish are released to hatcheries, what can be done to preserve the genetic quality of these fish (broodstock replacement)?  How effective was the private sector engagement approach for genetic improvement of carps?  What is the status of other GIPs (silver carp, mrigel, and catla)?  Do you think the rohu GIP program could be continued with AIN partner hatcheries when the activity concludes? If not, what can be done to insure the investment made in the GIP can be realized?  What is your long-term vision of carp genetic improvement in Bangladesh?  How many researchers you have trained in AIN who can carry out genetic research in the future?  Describe your role in establishing the TBN – TSH program and introducing 11th- generation GIFT to Bangladesh.  Do you believe that owners of TBN are capable of continuing the rotational breeding and maintaining the genetic quality of tilapia? If so, what makes you think so?  What can be done to assure continuity and preserve genetic quality?  How effective was the private sector engagement approach for dissemination of genetically improved tilapia?  What is your long-term vision for tilapia genetic improvement in Bangladesh?  In your opinion, what were the most effective activities or interventions made by the AIN activity?  What were the least effective activities or interventions?  Do you think any activities or interventions went off track due to changes in activity leadership?  What alternative activities or interventions should have been considered?  What do you think about the sustainability of AIN activity interventions when the activity ends?  Do you have any additional comments on the AIN activity interventions you were associated with?  When you were DG of BFRI, did you have any MOAs or MOUs with the AIN activity? How would you characterize the relationship between the AIN activity and BFRI?  What are your views about the culture of air-breathing fish and making additional fish 58 introductions in the country? Syed Arif Azad – Director General, Department of Fisheries The purpose of this interview is to learn more about your experiences with and recommendations related to your involvement with the AIN activity.  How long have you been working as Director General of the Department of Fisheries?  Did you have any contact or discussions with USAID before formulation of the Feed the Future program? If so, what message did you communicate to USAID?  When WorldFish was formulating the Aquaculture for Income (AIN) activity, did they seek ideas and input from DoF? If so, what was the nature of that input?  We are aware that DoF has a long-standing working relationship with WorldFish. Was there a formal relationship between DoF and the Aquaculture for Income and Nutrition (AIN) activity?  What was the nature of the cooperation between DoF and the AIN activity?  Were there any interventions and activities undertaken collaboratively by the AIN activity and DoF?  Does your field staff in the FtF region report to you about AIN activities in the region?  What is your knowledge and understanding of the activities and accomplishments of the AIN activity implemented by WorldFish?  To the best of your knowledge, what parts of the AIN activity worked well?  What parts did not work so well?  What could the activity have done better?  What is your assessment of the AIN activity’s genetic improvement programs?  What can be done to sustain and continue the efforts on genetic improvement made by the AIN activity?  The AIN activity is ending in a few months. If a new activity were to begin, what kinds of activities from the existing activity should be continued and what new initiatives should be made?  Have you had any dialogue with WorldFish about the Hatchery Law and the Feed Law and regulations about testing of shrimp seed? If so, how would you characterize the input of WorldFish about these laws? What is the status of implementation of these laws?  We know Bangladesh has become the fourth- or fifth-largest aquaculture producing nation in the world. What kinds of interventions and activities are needed to continue the growth of the aquaculture sector in the country?  What are the main problems and constraints facing the development of aquaculture in Bangladesh? How can these problems be addressed? Shirin Afroz – Director-Nutrition, Helen Keller International  What was the role of Helen Keller International (HKI) in the AIN activity? When was HKI involved with AIN activity?  Did HKI organize training for AIN beneficiaries? What was the main focus of training? Was the amount of time allocated for nutrition training sufficient?  What type of behavior change communication (BCC) activities or interventions were used by HKI? Which areas were covered?  Please describe any BCC materials developed by HKI for the AIN activity.  Which BCC interventions were most effective for AIN activity beneficiaries? Were there tools that allowed beneficiaries to be reached easily?  What do you mean by “minimum nutrition packages”? What were its benefit? Did you promote it?  What was the major impact of BCC activities implemented by AIN, with the support of HKI, in 59 terms of fish consumption, dietary diversity, and consumption of small fish (such as mola) and OSP by AIN activity beneficiaries?  How did BCC practices focus on women, pregnant/lactating mothers, and children?  Do you think AIN was a successful activity? If so, what is the evidence of that success?  What are your recommendations to improve BCC? Main Uddin Ahmad – Owner, MKA Hatchery  What kind of support did you receive from the AIN activity?  How did the activity help MKA hatchery in terms of aquaculture technologies?  How did the activity help MKA hatchery in terms of aquaculture management practices?  How has the activity strengthened the farmed shrimp value chain by working with your hatchery?  What were some of the challenges you’ve faced in carrying out your business? B. Site Visit Interview Questions by Stakeholder/Beneficiary Group Carp or Tilapia Hatchery Owner  When did you start your hatchery business?  What is your production capacity? Is it fully utilized?  How much spawn did you produce during 2016?  What carp species do you use for breeding? OR What strains to tilapia do you use in your hatchery?  What is the source of your broodfish?  Do you do business with any contract grower groups?  Where do you sell most of the fry that you produce?  What percentage goes to AIN activity areas?  Did you supply seed to people affected by cyclones Alia and Sidor in the past?  Do you have an over-wintering program or facilities?  Are you making money every year?  What is the main constraint to your business?  Are you a member of a hatchery association?  Do you provide embedded services and advice to buyers? Fish Nursery Pond Owner  Do you belong to any group/association/cooperative?  Why are you in the nursery business? Why not in hatchery?  How big is your nursing capacity?  What do you consider yourself? Partner or beneficiary? Did you sign an MOU with AIN?  What percentage of fry survival is break-even for your nursing business?  Do you nurse fries under mono or mixed basis?  What was your total production of fry in 2016?  Do you have over-wintering nursing business?  Where do you sell your produce? What percent of your customers are AIN beneficiaries?  If you are an AIN beneficiary, what assistance you have received from AIN?  Do your products go beyond AIN areas?  Do you have contract farmers? If so, do you buy back?  What do you do with surplus produce/unsold seeds? 60  What percentage of your nursed seeds are not carp or tilapia?  Are you facing any competition in your business?  What is the returning rate of your buyers? Feed Dealer  Are you a member of national or regional association?  Has your business gotten any boost in recent years? If yes, what is the reason?  What do you consider yourself? Partner or beneficiary? Did you sign an MOU with AIN?  What percentage of your produced feed is fish and shrimp feed?  Do you know the “Feed the Future Program”? Is it helping your business as a whole?  Do you know about the AIN activity? Do you have linkage with it?  AIN reportedly propagates cost-effective feeds to its beneficiaries. Does this hamper your business?  Do you believe AIN activities have increased your sales?  Have you received any assistance as an organization to upgrade or make changes based on the Feed Law of BD?  Do you have any program of collective imports of raw materials in bulk?  What are your main constraints in the feed business?  How effective is your sales network? Do you supply feed to farm gate?  How many commission agents do you have in AIN activity areas? C. Focus Group Discussion Guide Introduction: Good morning and welcome to our session. Thank you for taking the time to join us to talk about AIN programs in your area. My name is Dr. Monirul Islam and this is my colleague Tasmin Ethu. We are working on the final performance evaluation of the AIN activity implemented by WorldFish. We would like to ask you some questions about your experience and perceptions regarding a number of areas related to this program and how the program might be improved to be more effective in the future. We are having discussions like this with several groups around activity sites. Has anyone participated in a focus group before?  This a group discussion. We are not trying to get everyone to agree with each other; we are gathering information and opinions. There are no right or wrong answers. If you feel uncomfortable with any question, you can choose not to respond.  Your responses will not be specifically identified with you; we will not associate your name with anything you say in the focus group.  We would like to tape the focus groups so that we can make sure to capture the thoughts, opinions, and ideas we hear from the group. No names will be attached to the focus groups and the tapes will be destroyed as soon as they are transcribed.  All information gathered will be used only for the final performance evaluation of AIN activity. Q1. Did you get any training from AIN? If so, please describe what you learned from the training.  Who provided the training  How did the trainers deliver the training/message (verbally, with help of poster, leaflet)?  What did you learn in nutrition trainings? Q2. What type of training materials did you receive? Q3. Tell us about those training materials? Acceptability – To what extent are training media attractive to the beneficiaries?  What is your view about nutrition messages disseminated through various training materials? Which materials (such as leaflets, posters, food cards, food plates, stickers, etc.) were most effective at delivering the message? 61  Did you share information you learned with your family? How did your family members react?  Do you keep posters or festoons in your home? What is your family’s perception about putting up these materials on the wall? Message Comprehensibility  Do you understand all the words and picture illustrations in the training materials?  What were the contents of the leaflets/posters/stickers/festoons?  Which pictures/illustrations do you like the most? Why? Q4. Which materials do you think were most useful? Why were they most useful? (Refer to the BCC materials list.) Q5. How did the training materials increase your nutrition knowledge about the health benefits of eating small fish such as mola? How did these materials benefit family members? Q6. What training approaches or activities would you recommend that might be more effective? Q7. Tell us about your fish consumption.  In the past week, how many days did you eat fish?  How many cups of fish did you eat in the past 24 hours?  Are you eating more, less, or about the same amount of fish now, compared with before you became involved with the AIN activity? Q8. Who makes the major decisions in the family? Who gets the first choice of foods that are eaten? Q9. For eating small fishes, what are the advantages of eating the whole fish (with head and bones)? Q10. From your household pond, how much fish goes to the kitchen and how much is sold in the market? Closing That is all of my questions for now. Do you have anything you would like to add? Do you have any questions for us? Thank you for your time. D. Telephone Survey Questionnaires Carp Hatcheries Date ____________________ Time ____________________ Name ___________________________________ Telephone ____________________________ Male ______ Female ______ District ___________________ Questioner: We are evaluating the AIN activity from WorldFish and would like to ask you a few questions. We are not connected to the activity. May I continue? Questions: Q1. What types of fish do you breed in your hatchery? [Multiple responses possible.] Rohu/rui __________ Catla ________ Mrigal ________ Silver carp _________ Others (list)_____________________________________________ Q2. How much total spawn did you produce last year (2015)? ____________________ lakh Q3. How many broodfish ponds do you have? ________ 62 Q4. How many nursery ponds do you have? ________ Q5. What is the source of your broodfish? [Multiple responses possible.] Brood bank ____ Other hatchery____ Farmer____ Other source_____ Q6. Do you keep different stocks of broodfish in separate ponds? Yes ____ No_____ Q7. After induced breeding is finished, what proportion of your broodfish do you replace every year? _____% Q8. Do you have brood transport facilities? Yes ____ No_____ Q9. Do you have seed packaging facilities? Yes ____ No_____ Q10. Do you have over-wintering facilities for seed? Yes ____ No_____ Q11. Do you use quality larval feeds? Yes ____ No_____ Q12. Where do you sell the spawned fry or fingerlings that you produce? [Multiple responses possible. If % not known, leave blank.] Nursery operator _______ If you know, what % are AIN beneficiaries? _______% Patilwala _______ If you know, what % are AIN beneficiaries? _______% Directly to farmer _______ If you know, what % are AIN beneficiaries? _______% Q13. Do you provide information and advice to patilwalas on the importance of quality seed? Yes ____ No_____ Q14. Did the AIN activity help you to obtain better quality broodfish? Yes ____ No_____ Q15. Did the AIN activity assist you with obtaining inbred-free fry and fingerlings? Yes ____ No_____ Q16. Did the AIN activity link you to farmer’s groups to sell seed? Yes ____ No_____ Q17. Did the AIN activity assist you with upgrading hatchery techniques and management? Yes ____ No_____ Q18. If the answer to Q18 is yes, then what kind of support did you receive? Training ________ Water pumps ________ Aeration tower ________ Venturi aerator ________ Q19. How valuable was the support you received? _____ very useful ______ somewhat useful ______ not very useful 63 Tilapia Hatcheries Date ____________________ Time ____________________ Name ___________________________________ Telephone ____________________________ Male ______ Female ______ District ___________________ Questioner: We are evaluating the AIN activity from WorldFish and would like to ask you a few questions. We are not connected to the activity. May I continue? Questions: Q1. What strains or varieties of tilapia do you use in your hatchery? [Multiple responses possible.] GIFT __________ Nam sai ________ CP ______ AIT _______ Others (list)_____________________________________________ Q2. How many monosex fry did you produce last year (2015)? ____________________ lakh Q3. How many broodfish ponds do you have? ________ Q4. How many breeding hapas do you have? ________ Q5. How many egg hatching jars do you have? ________ Q6. How many nursery ponds do you have? ________ Q7. Do you keep different stocks of broodfish in separate hapas or ponds? Yes ____ No_____ Q8. Do you have brood transport facilities? Yes ____ No_____ Q9. Do you have seed packaging facilities? Yes ____ No_____ Q10. Do you have over-wintering facilities for seed? Yes ____ No_____ Q11. Do you use quality larval feeds? Yes ____ No_____ Q12. Where do you sell the spawned fry or fingerlings that you produce? [Multiple responses possible. If % not known, leave blank.] Nursery operator _______ If you know, what % are AIN beneficiaries? _______% Patilwala _______ If you know, what % are AIN beneficiaries? _______% Directly to farmer _______ If you know, what % are AIN beneficiaries? _______% Q13. Do you provide information and advice to patilwalas on tilapia farming? Yes ____No_____ 64 Q14. Did the AIN activity help you to obtain better quality broodfish? Yes ____No_____ Q15. Did the AIN activity link you to farmer’s groups to sell seed? Yes ____No_____ Q16. Did the AIN activity assist you with upgrading hatchery techniques and management? Yes ____No_____ Q17. If the answer to Q18 is yes, then what kind of support did you receive? Training ________ Hapas _______ Water pumps ________ Aeration tower ________ Venturi aerator ________ Q18. How valuable was the support you received? _____ very useful ______ somewhat useful ______ not very useful Brackish water Shrimp Hatcheries Date ____________________ Time ____________________ Name ___________________________________ Telephone ____________________________ Male ______ Female ______ District ___________________ Questioner: We are evaluating the AIN activity from WorldFish and would like to ask you a few questions. We are not connected to the activity. May I continue? Questions: Q1. How many nauplii did you produce last year (2015)? ____________________ lakh Q2. What is the capacity of your hatchery per production cycle? ___________________lakh Q3. How many production cycles did your hatchery have in 2015? _______ Q4. How many brood shrimp do you use per cycle? _______ Q5. Do you obtain your brood shrimp from multiple suppliers? Yes ____ No_____ Q6. Do you have an adequate supply of natural foods for brood shrimp? Yes ____ No_____ Q7. Does your hatchery have the capacity to grow algae and zooplankton? Yes ____ No_____ Q8. Do you nurse all nauplii to PL? Yes ____ No_____ Q9. If no, then do you have a sub-contract to sell nauplii? Yes ____ No_____ Q10. How many seed agents do you do business with? ____________ 65 Q11. What percentage of PL batches do you test for disease? _______% Q12. Did AIN provide resources for disease testing of your PLs? Yes ____ No_____ Q13. Do you provide information about the results of disease testing of PLs to buyers? Yes ____ No_____ Q14. Do you receive premium price for tested/certified PLs? Yes ____ No_____ Q15. Do you train shrimp seed traders/commission agents on the importance of stocking tested PLs in production ponds? Yes ____ No_____ Q16. What is the main way that PLs from your hatchery are transported to growing ponds? Truck ______ Air freight ______ Q17. If you know, what % of nauplii or PLs go to AIN beneficiaries? _______% [If % not known, leave blank.] Q18. Have you received AIN assistance to upgrade hatchery technology and management? Yes ____ No_____ Q18. If the answer to Q18 is yes, then what kind of support did you receive? Training ________ Water pumps ________ Aeration tower ________ Venturi aerator ________ Other ____________________________________ Q19. How valuable was the support you received? _____ very useful ______ somewhat useful ______ not very useful Freshwater Prawn Hatcheries Date ____________________ Time ____________________ Name ___________________________________ Telephone ____________________________ Male ______ Female ______ District ___________________ Questioner: We are evaluating the AIN activity from WorldFish and would like to ask you a few questions. We are not connected to the activity. May I continue? Questions: Q1. How many PLs did you produce last year (2015)? ____________________ lakh Q2. What is the capacity of your hatchery per production cycle? ___________________lakh Q3. How many production cycles in 2015? _______ 66 Q4. How many brood shrimp do you use per cycle? _______ Q5. What is your brood source for hatchery use? [Multiple responses possible.] From farms/ghers __________ From river _________ Q6. Do you have a brood prawn holding and/or rearing facility? Yes ____ No_____ Q7. How many spawning tanks do you have? ____________ Q8. Are you linked to a prawn brood bank? Yes ____ No_____ Q9. Did the AIN activity link you to any quality brood prawn source? Yes ____ No_____ Q10. Do you have over-wintering facilities for PLs? Yes ____ No_____ Q11. What larval feeds do you use? [Multiple responses possible.] Custard ______ Artemia ________ Q12. Where do you sell the seed that you produce? [Multiple responses possible.] Patilwalas________ Direct to farmer_________ Q13. If you know, what % of seed are supplied to AIN beneficiaries? _______% [Leave empty if hatchery owner does not know.] Q14. Do you provide information or training to patilwalas or farmers about good management practices in prawn farming? Yes ____ No_____ Q15. Have you received direct assistance or training from the AIN activity to upgrade hatchery technology and management? Yes ____ No_____ Q16. Have you received support from the AIN activity for prawn broodstock supply? Yes ____ No_____ Q17. If the answer to Q16 is yes, then what kind of support did you receive? Training ________ Water pumps ________ Aeration tower ________ Venturi aerator ________ Q18. How valuable was the support you received? _____ very useful ______ somewhat useful ______ not very useful Nursery Pond Operators Date ____________________ Time ____________________ Name ___________________________________ Telephone ____________________________ 67 Male ______ Female ______ District ___________________ Questioner: We are evaluating the AIN activity from WorldFish and would like to ask you a few questions. We are not connected to the activity. May I continue? Questions: Q1. How many nursery ponds you have? Q2. How much spawn/fry did you nurse last year (2015)? ____________kg Q3. What types of fish do you nurse? [Multiple responses possible.] Rohu/rui __________ Catla ________ Mrigal ________ Silver carp _________ Tilapia _________ Others (list) _____________________________________________ Q4. Do you nurse seed from different hatcheries separately? Yes ____ No_____ Q5. What are the sources of your fry? [Multiple responses possible.] AIN activity supported hatcheries _______ Non-AIN activity supported hatcheries _______ Q6. Do you always receive fry from the same hatchery sources? Yes ____ No_____ Q7. Do you over-winter seed? Yes ____ No_____ Q8. Did the AIN activity help connect your business to hatcheries to obtain quality seed? Yes ____ No_____ Q9. Where do you sell the fry that produce? [Multiple responses possible. If % not known, leave blank.] Patilwala _______ If you know, what % are AIN beneficiaries? _______% Directly to farmers _______ If you know, what % are AIN beneficiaries? _______% Q10. Did the AIN activity help connect your business to farmer’s groups to disseminate seeds? Yes ____ No_____ Q11. Do you provide advice and information to patilwalas or farmer? Yes ____ No_____ Q12. Do you sell better quality seed at a premium price? Yes ____ No_____ Q13. Do you sell fish seed on credit? Yes ____ No_____ Q14. Do you collect feedback from farmers on the performance of seed? Yes ____ No_____ Q15. Did you receive support from the AIN activity for feed or other inputs? Yes ____ No_____ 68 Q16. Do you have a facility for seed packaging and transport? Yes ____ No_____ Community Feed Centers Date ____________________ Time ____________________ Name ___________________________________ Telephone ____________________________ Male ______ Female ______ District ___________________ Questioner: We are evaluating the AIN activity from WorldFish and would like to ask you a few questions. We are not connected to the activity. May I continue? Questions: Q1. How many tons of feed did you produce last year (2015)? ______________ tons Q2. What kinds of different feed do you produce? [Multiple responses possible.] For carps ________ For tilapia ________ For shrimp ________ For golda ________ Other species __________________________________________ (provide names) Q3. Do you produce separate feeds for fry, fingerling, and grow-out? Yes ____ No_____ Q4. Is the main use of the feed you produce to grow fish on your own farm? Yes ____ No_____ Q5. Approximately how many farmers are using the feed that you produce? ____________ Q6. What % of feed is sold to AIN activity farmers? ___________% Q7. Do you sell feed on credit? Yes ____ No_____ Q8. Do you buy ingredients in small batches as you need them? Yes ____ No_____ Q9. Do you buy ingredients jointly with other feed producers? Yes ____ No_____ Q10. Do you use a vitamin and mineral premix in your feed? Yes ____ No_____ Q11. Do you have a good linkage to ingredient sellers? Yes ____ No_____ Q12. Do you have the capacity to store feed? Yes ____ No_____ Q13. Do you provide training to farmers? Yes ____ No_____ Q14. If the answer to Q13 is yes, what are some of the subjects of the trainings that you provide to farmers? [Multiple responses possible.] Liming and fertilization for pond preparation ________ Correct stocking density _______ 69 Proper feeding ________ Water management ________ Others ______________________________________________ Q15. What kinds of support and assistance did you receive from the AIN activity to support development of your feed mill? [Multiple responses possible.] Feed machine ______ Partial support for machine purchase ______ Partial support for ingredient purchase ________ Training for operation of feed equipment and feed formulation _______ Others (specify) ____________________________________________________ Q19. How valuable was the support you received? _____ very useful ______ somewhat useful ______ not very useful Q20. Do you believe you can produce feed without activity support (independently) after the AIN activity ends? Yes ____ No_____ Q21. Do you intend to expand your business in the next 3-5 years? Yes ____ No_____ 70 ANNEX V: FTF DISTRICTS WHERE THE AIN ACTIVITY WAS ACTIVE AND WHERE THE AIN-PE TEAM VISITED 71 ANNEX VI: LIST OF PERSONS CONTACTED Organization/ Address Address Name Position IBTCI ACME Hs# 01, Rd# 54/A, Gulshan - 2, Dhaka- 1212 Chief of Party ACME Hs# 01, Rd# 54/A, Gulshan - 2, Dhaka- 1213 Evaluation Coordinator ACME Hs# 01, Rd# 54/A, Gulshan - 2, Dhaka- 1213 Security Coordinator ACME Hs# 01, Rd# 54/A, Gulshan - 2, Dhaka- 1213 Evaluation Coordinator ACME Hs# 01, Rd# 54/A, Gulshan - 2, Dhaka- 1213 Finance and Admin Manager ACME Hs# 01, Rd# 54/A, Gulshan - 2, Dhaka- 1212 Capacity Building Specialist ACME Hs# 01, Rd# 54/A, Gulshan - 2, Dhaka- 1212 Sr. M&E Specialist Performance Monitoring ACME Hs# 01, Rd# 54/A, Gulshan - 2, Dhaka- 1213 Sr. M&E Specialist Performance Monitoring ACME Hs# 01, Rd# 54/A, Gulshan - 2, Dhaka- 1212 Sr Financial Analyst ACME Hs# 01, Rd# 54/A, Gulshan - 2, Dhaka- 1212 Admin Associate Note taker Implementing Partner Org. WorldFish WorldFish, AIN House #22B, Road #7, Block F, Banani, Dhaka 1213 COP WorldFish, AIN House #22B, Road #7, Block F, Banani, Dhaka 1213 DCOP WorldFish, AIN House #22B, Road #7, Block F, Banani, Dhaka 1213 M&E Manager WorldFish, AIN House #22B, Road #7, Block F, Banani, Dhaka 1214 Value Chain & Feed Specialist WorldFish, AIN House #22B, Road #7, Block F, Banani, Dhaka 1213 Feed Genetic Specialist WorldFish, AIN 66, Hazrat Borhan Shah Road ( Ist Floor), Jessore 7400 Project Manager WorldFish, AIN 66, Hazrat Borhan Shah Road ( Ist Floor), Jessore 7401 Technical Specialist WorldFish, AIN 66, Hazrat Borhan Shah Road ( Ist Floor), Jessore 7402 Technical Specialist WorldFish, AIN 66, Hazrat Borhan Shah Road ( Ist Floor), Jessore 7403 Finance and Admin WorldFish, AIN 66, Hazrat Borhan Shah Road ( Ist Floor), Jessore 7402 Aquaculture Value Chain 72 Organization/ Address Address Name Position WorldFish, AIN 66, Hazrat Borhan Shah Road ( Ist Floor), Jessore 7403 Aquaculture Value Chain WorldFish, AIN 66, Hazrat Borhan Shah Road ( Ist Floor), Jessore 7404 Aquaculture Value Chain WorldFish, AIN Plot # 82, Road # 7, Nirala Residential Area, Khulna 9100 Project Manager WorldFish, AIN Plot # 82, Road # 7, Nirala Residential Area, Khulna 9101 Training Manager WorldFish, AIN Plot # 82, Road # 7, Nirala Residential Area, Khulna 9102 NutritionNutrition Specialist WorldFish, AIN Plot # 82, Road # 7, Nirala Residential Area, Khulna 9103 Communications & Documentations Coordinator WorldFish, AIN Barisal Project Manager WorldFish, AIN Barisal Nutrition Specialist WorldFish, AIN Cox'sBazar PCR Lab Manager USAID USAID American Embassy, Madani Avenue, Baridhara, Dhaka Program Management Coordinator USAID American Embassy, Madani Avenue, Baridhara, Dhaka Project Management Specialist USAID American Embassy, Madani Avenue, Baridhara, Dhaka Deputy Director USAID American Embassy, Madani Avenue, Baridhara, Dhaka Program Management Coordinator USAID American Embassy, Madani Avenue, Baridhara, Dhaka Private Sector Advisor BAU Bangladesh Agriculture University, Mymensingh Mymensingh Proffessor & Head of Department of Fisheries Management Bangladesh Agriculture University, Mymensingh Mymensingh Proffessor, Dept of Fisheries Biology & Genetics Bangladesh Agriculture University, Mymensingh Mymensingh Head of Dept of Fisheries Biology & Genetics Bangladesh Agriculture University, Mymensingh Mymensingh Proffessor & Head of Department of Aquaculture 73 Organization/ Address Address Name Position BFRI Bangladesh Fisheries Research Institute Mymensingh Director General Bangladesh Fisheries Research Institute Mymensingh Principal Scientific Officer Bangladesh Fisheries Research Institute Mymensingh Principal Scientific Officer Bangladesh Fisheries Research Institute Mymensingh Retired Chief Scientist Bangladesh Fisheries Research Institute Freshwater Substation, Jessore Senior Scientific Officer Bangladesh Fisheries Research Institute Paikgacha Chief Scientific Officer Bangladesh Fisheries Research Institute Coxs Bazar DOF DOF Dhaka Director General DOF Jessore Director General DOF Narail District Fisheries Office DOF Bagerhat District Fisheries Office DOF Faridpur District Fisheries Office DOF Barisal Deputy Director DOF Coxs Bazar District Fisheries Office Khulna University Khulna University Khulna Professor & Head Fisheries & Marine Resource Technology Discipline Khulna University Khulna Former Head Fisheries & Marine Resource Technology Discipline Khulna University Khulna Professor, Fisheries & Marine Resource Technology Discipline Helen Keller International Helen Keller International H 10F, Road 82, Gulshan 2, Dhaka Nutrition Director Helen Keller International H 10F, Road 82, Gulshan 2, Dhaka Nutrition Training Officer Other Partners BFFEA Khulna Vice President FIQC Khulna Deputy Director Jessore University of Science and Technology, Jessore Professor, Fisheries and Marin Bioscience Discipline 74 Organization/ Address Address Name Position Other Stakeholders MKA Hatchery Cox'sBazar General Manager Quality Shrimp Project Cox'sBazar Director & Chief Consultant South Bangla Hatchery Cox'sBazar Managing Partner Pranti PCR Laboratory Khulna Managing Director Farook Sea Foods Limited Khulna General Manager AIN stakeholders for the AIN evaluation team # Name Designation Who are the Stakeholders 1 Khulna Hub 1.1 Project Manager AIN staff 1.2 Vice President, BFFEA Shrimp exporter 1.3 Professor and Dean University Partner & PCR lab 1.4 Chief Scientific Officer GoB research partners 1.5 Deputy Director, FIQC, Khulna DoF partner 1.6 District Fisheries Officer DoF partner 1.7 CP Representative India & Thailand CST farmers, CP feed dealer & PCR lab owner 1.8 Focal Person, CODEC Implementing NGO Partner 1.9 Secretary Shrimp collection Center/ e￾traceability 1.10 Owner, Community Feed Center Small feed miller 1.11 Owner, Asha Tilapia Hatchery Mono sex tilapia hatchery with TBN 1.12 M/S. Mritayunjoy Fish Traders Shrimp Depot 1.13 Co-Proprietor, Rupsha Prawn Hatchery Prawn research Partner 1.14 Farmers Group AIN Farmer group 1.15 Fish fry Traders (Patilwala) AIN LSP 1.16 Nurserar (Fish) AIN LSP 1.17 Program Coordinator 1.18 Managing Director Bismillah Fish Feed Company 1.19 Regional Manager 2 Barisal Hub 2.1 Project Manager AIN staff 2.2 Deputy Director, DoF DoF partner 2.3 Hatchery owner Hatchery owner and Convener of QSG-Batazore 2.4 Commercial fish farmers Commercial farmers 2.5 Gill net farmer Mola harvest 2.6 Fish Nurserer AIN LSP 2.7 Small Fish feed mill owner AIN LSP 2.8 Patilwala AIN LSP 75 # Name Designation Who are the Stakeholders 2.9 SIP Super Intensive Pangas 2.10 Fish hatchery owner Quality seed/AIN LSP 2.11 Assistant Professor-PSTU University partner 3.1 Project Manager AIN Staff 3.2 President, Jessore District Carp Hatchery owners’ Association President 3.3 President, Tilapia Hatchery Association President 3.4 Owner, Tilapia Breeding Nucleus (TBN) Owner 3.5 Owner, TBN Owner 3.6 PM, AIN PM, AIN 3.7 PM, AIN PM AIN 3.8 Fish Nurserer Owner 3.9 Fish Nurserer Owner 3.10 Commercial aquaculture Owner 3.11 Commercial Aquaculture Owner 3.12 Water recycling with Solar system Owner 3.13 Water recycling System Owner 3.14 Dyke Cropping AN LSP 3.15 Cage Culture Women group leader 3.16 Mola Brood Pond AN LSP 3.17 Mola Brood AN LSP 3.18 DFO, Jessore DFO, DoF 3.19 DFO, Jhenaidha DFO, DoF 3.20 DFO, Magura DFO ,DoF 3.21 DFO Narail DFO, DoF 3.22 SSO, BFRI, Jessore BFRI 3.23 Professor, Fisheries and Marine Bioscience discipline. Jessore University of Science and Technology Professor 3.24 President, Buk bhora Baor (oxbow lake) President 3.25 Jessore Fish Market Fish wholesaler 3.26 Fingerling seller, Fish fingerling Market Fingerling seller 3.27 General Manager, Uttara Feed (Venkey) GM of Feed Mill 3.28 CFC owner CFC owner 4 Faridpur Hub 4.1 Project Manager Project staff 4.2 District Fisheries Officer DFO 4.3 Hatchery owner Carp & tilapia hatchery owner 4.4 CFC owner LSP and CFC owner 4.5 Nurserer Quality Seed Group leader and nurserer 4.6 Patilwala Fry Trader 76 # Name Designation Who are the Stakeholders 4.7 Inputs supplier LSP Association member & inputs supplier 4.8 SIS hatchery owner and carp brood supplier SIS seed, carp seed producer and wild brood supplier 77 ANNEX VII: Evaluation Team Disclosures 78 79 80 81 82 83 ANNEX VIII: Evaluation Team Composition Dr. John Hargreaves, Team Leader; Dr. Hargreaves served as the team leader and evaluation specialist during the evaluation. He led the preparation and finalization of the evaluation design and coordinated the process of assembling the final findings and recommendations of the evaluation. He has earned a Ph.D. in Aquaculture from Louisiana State University. He also holds a MS in Fisheries and Allied Aquacultures. Dr. Hargreaves has over 35 years of experience in evaluations, assessments, technical support, and training in aquaculture and fisheries management, value chains, best practices, assessment, and training. In 2012, he conducted a USAID-funded evaluation of water use and efficiency in fish farming in Iraq, and assessed the market potential for commercial aquaculture for the USAID-funded COMPETE project. More recently he conducted an evaluation of the seafood certification system benchmarking tool for the Global Sustainable Seafood Initiative. Before that, he evaluated the design and developed a management plan for a commercial tilapia farm in Guatemala. Dr. Hargreaves has demonstrated experience leading teams as well as workshops and trainings. He has conducted numerous assessments of aquaculture programming in many countries for USAID-funded projects as well as the private sector. He has frequently designed, tested, evaluated, and recommended improvements to management, systems, techniques, and facilities for fish hatcheries and farms. Additionally, he has conducted troubleshooting of processes and facilities to improve production. For the last three years he has supported dozens of aquaculture activities in Central and South America on behalf of the US Soybean Export Council, focusing on improving production efficiency, developing commercial aquaculture, improving seed production, training production managers and staff, and building institutional capacity for aquaculture development. He has provided technical assistance and training for farmers and growers of fish and shrimp, and improving nursery and hatchery systems and production facilities. Dr. Hargreaves has supported USAID-funded projects in Iraq (Inma Agribusiness Program), Afghanistan (ASMED, IDEA-NEW, AAEP II projects), the West Bank (Compete project), and multiple Farmer-to-Farmer Programs in Haiti and Guyana. Dr. Md. Abdul Hossain, Aquaculture Specialist Dr, Hossain served on the evaluation as the Aquaculture specialist and played a key role in evaluation planning as well as writing and contributing to key sections of the report. He holds a PhD in Aquaculture Fish Nutrition from Kysuhu University in Japan. He also partook in a Post-doctoral study in coastal biodiversity management at the University of Rhode Island, USA. Dr. Hossain holds two Master of Science degrees, one in Aquaculture and one in Fisheries Biology and Limnology. Dr. Hossain is a Bangladeshi national with 35 years of aquaculture experience and has substantial experience in the development, management, and evaluation of aquaculture projects. He has conducted, market studies and sub-sector studies in both fisheries and aquaculture. He is an expert in sub-sector mapping, production chain, supply chain and value chain mapping, and constraint analysis to devise appropriate interventions to address sector constraints that hinder growth and development. He also worked with marginal wetland and coastal dwellers for the improvement of their livelihoods and food security and to shift them from subsistence living on open water fisheries to managed small scale aquaculture communities. As Fisheries Specialist, he coordinated all aquaculture/fisheries based activities for a project implemented by the Bangladesh Development Society. Early in his career, he served as Chief Scientific Officer and initiated, established, and headed a coastal and marine research station where he coordinated and guided all aquaculture and aqua-farming research activities. He managed the USAID supported ATDP-II project as Team Leader, and also served as Team Leader for the USAID funded PRICE project and drafted a framework for wetland, coastal and marine protected areas in Bangladesh. Dr. Hossain also has 15 years of volunteering experience with Asiatic Society of Bangladesh to formulate Banglapedia and the Encyclopedia of Fauna and Flora of Bangladesh. During the last twelve years, Dr. Abul Hossain has worked actively for aquatic resource development to enhance income, jobs and investments as a prime target. Dr. Md. Monirul Islam, Nutrition Specialist; Dr. Islam served as the nutrition specialist on the evaluation team. He contributed to the team by assessing the nutritional impacts on beneficiaries, and focusing on nutrition related findings and analysis. Dr. Islam holds a PhD in Nutrition from Newcastle University in United Kingdom. He also holds a Master of Science degree in Aquaculture as well as a Bachelor of Science degree in Fisheries from Bangladesh Agricultural University in Bangladesh. Dr. Md. 84 Monirul Islam is a Bangladeshi national with substantial experience integrating nutrition components into development projects and programs. He has more than 25 years of experience working with Bangladesh Fisheries Research Institute (BFRI) and Bangladesh Agricultural Research Council (BARC). He is at present working as the Director of Nutrition at BARC, where he is engaged in the dissemination of nutrition information to different stakeholders. He has significant experience in community based nutrition education and behavior change communication (BCC), capacity building on food safety and quality, and mitigation measures of contaminants and adulterants in the food chain. Additionally, Dr. Islam is a reviewer of several scientific journals on agriculture research, fisheries research, and zoology, and has prepared and delivered multiple trainings on various nutrition and food safety topics. Dr. Islam has been engaged in different dimensions of Bangladesh’s nutrition such as the monitoring and evaluation (M&E) of various research projects, conducting research programs with the National Agricultural Research System (NARS) Institutes as well as the Bangladesh Council of Scientific and Industrial Research (BCSIR) and various universities. He has strong experience with shrimp & fish nutrition, heavy metals, water and industrial pollution, coastal and freshwater fisheries programs, biodiversity, livelihood and socio-economic studies, environmental issues, the role of children and women in the fisheries sector, and with the planning, implementation and M&E of different research programs. Dr. Islam has experience with USAID regulations and systems through his experience working as Research Supervisor/Shrimp-Fish Expert funded in part by USAID’s Food for Peace (FFP) office as well as his experience working as a Fish Nutrition Consultant with Winrock International, a prominent international NGO contracting with USAID in Bangladesh. 85 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523