Foreign Agricultural Service The US Department of Agriculture CAMBODIA Food for Progress Project Midterm Evaluation 8/31/2021 DISCLAIMER: This publication was produced at the request of the United States Department of Agriculture. It was prepared by an independent third-party evaluation firm. The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Mid-term Evaluation of the CAST Cambodia Program: Food for Progress Agreement Number: FCC-442-2018/002-00 Funding Year: Fiscal Year 2021 Evaluation Duration: February-August 2021 Implemented by: American Soybean Association (ASA) Evaluation Authored by: Rapid Asia Co., Ltd Daniel Lindgren, Toru Hisada and Jacob Bogart 3 Acknowledgements The mid-term evaluation (MTE) of the Commercialization of Aquaculture for Sustainable Trade (CAST) Cambodia would not have been possible without the contributions from several parties during project planning, data collection, analysis and report preparation. Rapid Asia would like to acknowledge several organizations, groups and individuals whose contributions were instrumental in making the MTE a success. Rapid Asia would like to acknowledge the contributions from CAST partners, stakeholders and beneficiaries, who took part in the key informant interviews (KIIs) conducted as part of the MTE. They include World Vision International (WVI), Kansas State University (KSU), Auburn University (AU), Can Tho University (CTU), EU CAP Fish, micro-finance institutions (MFIs), the United States Department of Agriculture (USDA), Cambodian Ministry of Agriculture, Forestry and Fisheries (MAFF) and its Fishery Administration (FiA) as well as some of the CAST beneficiaries. We want to express appreciation to American Soybean Association (ASA), who entrusted the company to conduct the MTE. Special thanks go to James Bernhardt, World Initiative for Soy in Human Health (WISHH)-Director of Monitoring & Evaluation (M&E) of ASA. Their support and inputs during the MTE have been most appreciated. The MTE would not have been possible without the support of the CAST team in Phnom Penh, Cambodia. We want to express our sincere gratitude and appreciation to Jim Hershey, Sambath Sak, Len Rodgers, Tola Heng, and Sar Sanphirom for their help during data collection and invaluable input on tools and reports. Appreciation goes to BMRS Asia Cambodia, who carried out the data collection and our aquaculture expert Soly Seak in Cambodia for his technical advice and inputs. The successful completion of the MTE would not have been possible without these contributions. Rapid Asia also wishes to acknowledge the aquaculture supply chain actors, including pond and cage fish producers, hatcheries/nurseries, processors, collectors/wholesalers, feed mills and feed distributors, who took time out of their busy schedules to participate in the phone survey. Rapid Asia Co., Ltd Bangkok, Thailand August 2021 4 Executive Summary Project background and purpose ASA/WISHH, in partnership with AU, KSU and WVI, is implementing a five-year project on the Commercialization of Aquaculture for Sustainable Trade (CAST) Cambodia. The project aims to increase agriculture productivity in the Cambodian freshwater aquaculture industry through improved inputs and practices and increase trade in aquaculture by volume and value. CAST’s key targets are local supply chain actors, including hatcheries/nurseries, cage and pond producers, processors, wholesalers/collectors, feed mills and feed distributors. The present study is a mid-term evaluation (MTE) of CAST. During the project’s second year of implementation, this exercise was conducted and aimed to measure the project's progress to date to relay this information to the project’s implementing agency (PIA), ASA/WISHH and its donor, the United States Department of Agriculture (USDA). Specifically, the MTE seeks to assess the project's relevance, provide insight into its effectiveness, document lessons learned, determine sustainability, and discuss and recommend course corrections. Finally, the MTE also highlighted the strengths and weaknesses of CAST and made recommendations for the next steps. Evaluation questions, design, methods and limitations The MTE was carried out using a mixed-methods approach. During the inception stage, a desk review was carried out, reviewing project-related documents followed by key informant interviews (KIIs) done with CAST staff, partners, stakeholders, beneficiaries and donors. At the same time, a Data Quality Assessment (DQA) was done, interviewing key CAST M&E staff and reviewing the data collection plans and other relevant documents. The results from the KIIs helped inform the development of the questionnaire for the MTE survey, which took place next. The MTE survey covered all supply chain actors, including hatcheries and nurseries, fish producers, wholesalers and collectors, processors, feed mills and feed distributors. Due to the pandemic situation in Cambodia, which was escalating during the MTE, data collection for the MTE survey was done using a computer-assisted personal interview (CAPI) system but executed by phone from a target respondent list provided by CAST. Findings and Conclusions The findings and conclusions are based on the project’s stated goals and activities as reflected in the MTE. CAST participants reported improved management practices. All hatcheries and nurseries and 85% of the producers are now using pellet feed. Half of the hatcheries and nurseries have fenced ponds and a water well, and 60% of producers have fenced ponds, and 42% have a water well. However, just over half measure the feed they give each day and around two-thirds record how much feed they use. The results show that whilst most of the actors have heard of the feed conversion ratio (FCR), only feed mills know how the FCR is calculated and how the ratio should be interpreted. Producers, in particular, show a significant knowledge gap in this area. The improved management practices result from beneficiaries attending and receiving technical assistance, which has been rather extensive, reaching between 75 and 100% of all beneficiaries who have signed up with CAST. Over four in five beneficiaries from hatcheries/nurseries, feed mills, and feed distributors said they had shared what they had learned with others. These results indicate that some benefits of the program may have reached beyond the immediate participants. 5 Since joining CAST, one out of 10 hatcheries/nurseries that took part in the MTE survey reported adding a total of five new ponds for a total increase of 3,200 square meters. Another 22 producers added a total of 46 new ponds with an estimated 97,320 square meters.1 Save for one producer, all of them attributed the increase to being part of the CAST program. Two hatchery/nurseries added new cages totalling 2,516 cubic meters. Another two producers added 36 square meters of cages. Regarding prices and sales, due partly to the spread of COVID-19 and related travel restrictions, a rise in imported fish from Vietnam, and a loss of local customers, 67% of wholesalers and collectors reported a decline in local aquaculture purchases in the reporting period. Whilst some actors saw an average increase, there is a strong indication that local sales have declined. Indeed, 20% of hatcheries and nurseries experienced an average 35% drop in the sales price, and 55% of producers saw a price drop of 20%. Overall, beneficiaries involved with CAST agree that the program has made a difference. For example, 96% of participants agreed that CAST teaches valuable skills, while 90% thought involvement with CAST would help to improve their future living standards. The majority (91%) also said the wellbeing of their family had improved. They also felt the program has contributed to improving the quality of aquaculture in Cambodia (96%). Recommendations Communication and engagement: - Aquaculture supply chain actors need to engage more to share information through provincial and regional workshops. - CAST should review the progress made to date in light of COVID-19 and query whether it should update activities and target groups to make the program more relevant. Supply chain capacity development: - Aquaculture farmers need increased access to low-interest rate loans to compete with imports from neighboring countries. - Production of local fingerlings needs to be enhanced further. Fingerlings in Cambodia are relatively expensive compared to neighboring countries, Vietnam in particular. - The handling and transportation of fingerlings need to be improved. Farmers lose significant quantities of fingerlings between leaving the hatcheries and entering the pond. Market development: - CAST should review its market activities—including market information, commercial visits with distributors, business dialogues among buyers and producers—and determine how these activities can be strengthened or scaled up. - In light of COVID-19 and the reduction in the price of fish, CAST should conduct a market or consumer demand study after the pandemic is under control. Enhancing understanding: - CAST could facilitate local and international study and learning tours to appropriate countries for selected Cambodian aquaculture producers who can serve as influencers for other producers. - CAST should continue to expand local extension workers who can support project beneficiaries by providing regular coaching and technical assistance. Sustainability: - CAST should review its exit strategy in line with the goal of the CAA being able to stand on its own after the project’s completion. 1 . Number of ponds and square meters refer to all ponds hatcheries and producers have added since joining CAST. 6 Acronyms Acronym Full term ARDB Agriculture & Rural Development Bank ASA American Soybean Association AU Auburn University CAA Cambodian Aquaculturist Association CAPI Computer-assisted Personal Interview CAST Commercialization of Aquaculture for Sustainable Trade CBO Community-based organization CCC Commodity Credit Cooperation CE SAIN Center of Excellence for Sustainable Agricultural Intensification and Nutrition CI Custom Indicator CTU Can Tho University DAP Data Analysis Plan DCM Data Collection Manual DQA Data Quality Assessment FiA Fishery Administration FCF Finite correction factor GAqP Good Aquaculture Practice IPTT Indicator Performance Tracking Table KII Key Informant Interview KSU Kansus State University MAFF Ministry of Agriculture, Forestry and Fisheries MDG Millennium Development Goal MFI Microfinance Institution (MFI) MTE Mid-term evaluation M&E Monitoring and Evaluation NGO Non-governmental organization NSDP National Strategic Development Plan PIA Project Implementing Agency PMP Performance Monitoring Plan SI Standard Indicator SME Small and Medium-sized enterprises SPF Strategic Planning Framework for Fisheries SOP Standard Operating Procedure SPS Sanitary and Phytosanitary TA Technical Assistance ToT Training of Trainer USDA United States Department of Agriculture WISHH World Initiative for Soy in Human Health WVI World Vision International 7 Table of Contents Acknowledgements...................................................................................................................... 3 Executive Summary ...................................................................................................................... 4 Acronyms .................................................................................................................................... 6 Table of Contents ......................................................................................................................... 7 List of Tables ................................................................................................................................ 8 List of Figures ............................................................................................................................... 8 Introduction ................................................................................................................................ 9 1.1. Project Background ................................................................................................................. 9 1.2. Project Description and Objectives ....................................................................................... 10 1.3. Project indicators .................................................................................................................. 12 1.4. Results Framework................................................................................................................ 13 1.5. Purpose of the Evaluation ..................................................................................................... 14 2. Evaluation Design and Methodology ................................................................................... 14 2.1. Evaluation Design .................................................................................................................. 14 2.2. Evaluation Questions ............................................................................................................ 15 2.3. Sampling Methods ................................................................................................................ 15 2.4. Data Collection Methods ...................................................................................................... 16 2.5. Data analysis ......................................................................................................................... 20 2.6. Data quality assessment (DQA)............................................................................................. 21 2.7. Limitations............................................................................................................................. 22 3. Findings .............................................................................................................................. 23 3.1 Findings from the desk review .................................................................................................... 23 3.2 Project progress on standard and custom indicators ................................................................. 23 3.3 Evaluation Question Results ....................................................................................................... 31 1. Conclusions ........................................................................................................................ 56 2. Recommendations for CAST ................................................................................................ 57 Annexes..................................................................................................................................... 59 Annex 1: Data quality assessment (DQA) ......................................................................................... 59 Annex 2: Other results from MTE survey .......................................................................................... 72 Annex 3: Theory of change ............................................................................................................... 78 Annex 4: Survey questionnaire ......................................................................................................... 79 Annex 5: KII moderator guide ........................................................................................................... 95 Annex 6: DQA moderator guide ........................................................................................................ 99 Annex 7: KII moderator training agenda ......................................................................................... 114 Annex 8: Survey enumerator training agenda ................................................................................ 115 Annex 9: Bibliography ..................................................................................................................... 116 Annex 10: Terms of reference......................................................................................................... 118 8 List of Tables Table 1: Standard and custom indicators ............................................................................................. 12 Table 2: CAST activities and their link to the results framework .......................................................... 13 Table 3: KII participants ........................................................................................................................ 16 Table 4: Phone survey outcome ........................................................................................................... 18 Table 5: Response rate and sampling error at the individual level ...................................................... 18 Table 6: Response rate and sampling error at the business level ........................................................ 19 Table 7: Beneficiary profile .................................................................................................................. 19 Table 8: DQA results.............................................................................................................................. 21 Table 9: Results for standard and custom indicators ........................................................................... 24 Table 10: Fish species produced ........................................................................................................... 26 Table 11: Progress on training development topics ............................................................................. 27 Table 12: Feed conversion ratio ............................................................................................................ 28 Table 13: Fingerling quality ................................................................................................................... 29 Table 14: Services received from CAST ................................................................................................. 30 Table 15: CAST alignment with the National Strategic Plan ................................................................. 32 Table 16: Topics covered in training and during TA .............................................................................. 39 Table 17: Change in beneficiaries’ lives as a result of CAST (all participants) ...................................... 52 List of Figures Figure 1: Change in aquaculture fish purchase last production cycle .................................................. 25 Figure 2: Change in sales price per kilo ................................................................................................. 26 Figure 3: Technical support feed mills and feed distributors provide to customers ............................ 30 Figure 4: Received training or TA provided by CAST ............................................................................. 37 Figure 5: Applied learning from training or TA provided by CAST (All beneficiary types) .................... 37 Figure 6: Can do work more effectively as a result of CAST support .................................................... 41 Figure 7: Change in aquaculture sales price ......................................................................................... 46 Figure 8: Have shared learnings from CAST with other farmers (All beneficiary types) ...................... 48 Figure 9: Added new ponds since joining CAST .................................................................................... 49 Figure 10: Added new cages since joining CAST (Producer, Hatchery/Nursery) .................................. 49 Figure 11: Change in fish production (Producer) .................................................................................. 50 Figure 12: Change in fish production (Hatchery/Nursery) .................................................................... 51 Figure 13: Last production cycle (Producer, Hatchery/Nursery) .......................................................... 51 9 Introduction 1.1. Project Background The aquaculture industry is an essential source of food security and economic livelihood in Cambodia. Compared to other more expensive protein sources, fish is an accessible food staple for Cambodian diets and a healthy one. Increased fish consumption is linked to lower rates of infant mortality and disability.2 The sale of fish has also been reported as the second most important source of income for Cambodians,3 supplementing overall familial incomes and investments and increasing school attendance rates for children.4 Furthermore, it is a fast-growing industry, averaging a growth rate of over 20% from 2003 to 2016.5 Despite its importance in Cambodian life and its consistent growth, the aquaculture industry remains in its "infant stage of development" and faces challenges that limit production and further expansion.6 These challenges are: 1. Low productivity of commercial aquaculture operations; 2. Lack of high-quality inputs (such as feed and seed) in the marketplace; 3. Weak market linkages for quality Cambodian-sourced fish; 4. Underdeveloped small and medium enterprises in aquaculture and supporting industries; and 5. Weak supporting policy environment.7 However, with potential improvements to technology, market access, and education, there remains considerable scope for expanding Cambodian aquaculture.8 2 Olaganathan, R. and Mun, A. T. K. (2017) ‘Impact of aquaculture on the livelihoods and food security of rural communities’, International Journal of Fisheries and Aquatic Studies, 5(2), p. 281. 3 Richardson, R. B. and Suvedi, M. (2018) ‘Assessing the Potential for Small-Scale Aquaculture in Cambodia’, Environments, 5(76), p. 5. 4 Richardson, R., Sudevi, M., Kaplowitz, M. and Mith, S. (2017). Cambodia HARVEST Aquaculture Program Evaluation: Feed the Future - Cambodia Impact Evaluation and Strengthening of Local Capacity to Monitor and Evaluate Food Security Programs. [online] Cambodia: Feed the Future, pp. 55–95. Available at: https://pdf.usaid.gov/pdf_docs/PA00N42Q.pdf#page=62 [Accessed 23rd November 2020], p. 70. 5 Joffre, O., Pant, J., Somony, T., Chantrea, B. and Viseth, H. (2019). Transforming aquaculture in Cambodia through introduction of improved tilapia. [online] Penang: WorldFish, pp. 1–20. Available at: https://digitalarchive.worldfishcenter.org/bitstream/handle/20.500.12348/2663/2019-03.pdf?sequence=2 [Accessed 23rd November 2020], p. 5; Leakhena, C., Viseth, H., Somony, T., Da, C., Samnang, P. and Bunly, C. (2018) ‘Aiming for Improved Gender Equity in Cambodian Aquaculture’, Fish for the People, 16(2), p. 26. 6 Ouch, L. (2015). Current Status of Sustainable Aquaculture in Cambodia. [online] Bangkok: Southeast Asian Fisheries Development Center, pp. 1–14. Available at: https://repository.seafdec.org.ph/bitstream/handle/10862/2761/LangO2015.pdf?sequence=1&isAllowed=y [Accessed 23rd November 2020], p. 28. 7 American Soybean Association & World Initiative for Soy in Human Health. (2018). Commercialization of Aquaculture for Sustainable Trade (CAST) - Cambodia: Strategic Analysis (draft), p. 5 (hereinafter “CAST – Strategic Analysis.”) 8 Manet, S. (2019) 'With aquaculture on the rise, fish production up 6 pct in 2018' [online]. Khmer Times. Available from: https://www.khmertimeskh.com/575673/with-aquaculture-on-the-rise-fish-production-up-6-pct-in-2018/ [Accessed 23rd November 2020]. 10 1.2. Project Description and Objectives The Commercialization of Aquaculture for Sustainable Trade (CAST): Cambodia aims to address the aforementioned shortcomings by increasing agriculture productivity in the Cambodian freshwater aquaculture industry through improved inputs and practices and increasing trade in aquaculture by volume and value. The key actors and businesses the project seeks to reach are local supply chain actors, including hatcheries/nurseries, cage and pond producers, processors, wholesalers/collectors, feed mills and feed distributors in Cambodia. CAST is designed for implementation over five years with a total budget of $15.5 million in monetization proceeds plus $1.6 million in Commodity Credit Cooperation (CCC) funds. CAST is led by the American Soybean Association’s (ASA) World Initiative for Soy in Human Health (WISHH). It operates in partnership with Auburn University (AU), Kansas State University (KSU) and World Vision International (WVI). CAST’s overall strategy focuses on developing aquaculture enterprises, especially those of medium and larger-sized enterprises (in Cambodian terms), for participation in mid-level to premium markets as a driver for Cambodia’s freshwater aquaculture industry. SMEs have the production capacity to supply fish to commercial markets and to drive demand for high-quality inputs. In addition, CAST hopes to link successful SMEs with entrepreneurial, high-input smallholder farmers to better connect these farmers to farm inputs, services, and markets. In practical terms, CAST seeks to increase domestic aquaculture productivity and capacity to build domestic demand and capture at least 25% of growth for Cambodia-grown, quality-assured fish that achieve price premiums in the marketplace. This foundation will catalyze Cambodian aquaculture on a path toward more significant market share capture in existing and future domestic and regional premium markets that will increase net incomes for sector stakeholders throughout the value chain.9 CAST conducts ten activities to achieve these goals, including enrolling and providing training and technical assistance to 900 aquaculture producers working at 600 businesses across six Cambodian provinces (e.g., Battambang, Kampong Cham, Kampong Thom, Kandal, Pursat and Siem Reap) and greater Phnom Penh. The project also works with 14 hatcheries/nurseries, 12 feed distributors, and five feed mills and will provide business development training and financing support to 117 post￾production actors.10 Other CAST activities to support SMEs include:11  To work with retailers and wholesalers to strengthen aquaculture SME participation in their supply chains. 9 American Soybean Association & World Initiative for Soy in Human Health, Midterm Evaluation (Mte) For C: Request For Proposals, pp. 2- 5. 10 American Soybean Association & World Initiative for Soy in Human Health. (2020.) Midterm Evaluation (MTE) For Cast: Request For Proposals. [Online]. Available at: http://www.usaedc.org/media/2385/cast-cambodia-mte-rfp_110920.pdf. 11 CAST – Strategic Analysis, p. 21. 11  To support feed mills and hatcheries to build their supply chains and grow their customer base.  To promote professional development, technology transfer and business support services.  To facilitate a policy environment that is responsive to SME needs.  To develop a Cambodian national aquaculture association to promote Cambodia-produced fish, serve as a private sector voice on policy issues, and facilitate learning opportunities for professionals in the sector.  To unlock financing to invest in growth. CAST is a market-driven project that selects species for production and specific beneficiaries for participation based on market needs. All strategies and interventions were developed and implemented with a capacity-building, commercially sustainable exit strategy in mind. CAST’s intervention strategy is designed around four key levers of change, including:12 1. Market Linkages: Improved availability and accessibility of quality inputs to increase commercial productivity; better access to mid-value and premium markets for commercial aquaculture; 2. Knowledge Platforms: Access to information and capacity building for all stakeholders and beneficiaries throughout the value chain from input supplier to producer to consumer; 3. Collective Action: Improved transactional efficiencies at all levels; and 4. Enabling Environments: Actor coordination and strength in partnerships through synergies and assets associated with partners. CAST is designed to target semi-intensive or intensive mid- and large-sized pond and cage aquaculture systems producing highly demanded species (e.g., striped snakehead, clarias, pangasius, tilapia). CAST focuses its production and enterprise development activities with market development initiatives focused on larger urban markets (e.g., Phnom Penh, Siem Reap) with growing mid- and high-end consumer market segments willing to pay quality premiums. CAST works with companies and partners that service or source from these producers regardless of location. The CAST engages smallholders working in high-input pond culture as a primary livelihood objective for focal farmer groups. CAST practices adaptive management throughout the project, whereby stakeholders and project partners and other feedback mechanisms provide feedback on the project’s implementation to guide enhancements and course corrections. Finally, CAST seeks to ensure that the continued growth of the aquaculture industry in Cambodia is sustainable. Current aquaculture practices put unsustainable pressures on wild fisheries for feed and seed. CAST connects Cambodian farmers with U.S. sourced soybean-based pellet feeds to use instead 12 Ibid. 12 of wild fish. Soybean products provide an affordable source of protein that can replace fish protein to ensure lower production costs, sustainable wild fisheries, good water quality, and affordable aquatic￾source proteins. 1.3. Project indicators The MTE included an update of CAST’s standard and custom indicators shown in Table 1 below. Updated information on the indicators came mainly from data already collected by CAST through the Open Data Kit (ODK) system. The ODK data had recently been updated in March 2021, and it was resolved that the ODK data would be more accurate than what a phone survey may be able to achieve. Because the MTE survey was conducted via phone, interviewing time was limited. Also, due to the pandemic situation, it was determined that any significant changes in the key indicators were unlikely. Instead, it was decided that the MTE survey should mainly focus on output activities undertaken by CAST and would be more helpful to determine if the project was on track. Table 1: Standard and custom indicators Standard Indicators (SIs) # Indicator 1 Yield of targeted agricultural commodities. 2 Number of hectares under improved management practices or technologies that promote improved climate risk reduction and/or natural resources management. 3 Number of hectares under improved management practices or technologies. 4 Number of individuals in the agriculture system who have applied improved management practices or technologies. 12 Number of organizations with increased performance with USDA assistance. 13 Number of public-private partnerships formed as a result of USDA assistance. 18 Value of annual sales of farms and firms. 19 Volume of commodities (metric tons) sold by farms and firms. 21 Number of individuals who have received short-term agricultural sector productivity or food security training as a result of USDA assistance. 22 Number of individuals participating in USDA food security programs. Custom Indicators (CIs) # Indicator 1 Number of private enterprises, producer organizations, water user organizations, women's groups, trade and business associations, and community-based organizations (CBOs) who received short-term agricultural sector productivity, food security or other training as a result of USDA assistance. 2 Area of ponds (cubic meters) under improved management practices or techniques with USDA assistance. 6 Number of farms that increased the number, size, or amount of ponds, cages, or equipment with USDA assistance. 13 1.4. Results Framework CAST’s theory of change (Annex 3) is that the Cambodian aquaculture industry will grow in size, profitability, and become more sustainable. And assuming relevant actors have access to better inputs (such as soybean protein-fortified feed), farm practices, markets, and financing, the aquaculture sector will grow at scale to meet the growing demand for fish in Cambodia. CAST’s theory of change is an amended version of the USDA template theory of change. CAST’s theory of change is based on the idea that aquaculture SMEs are best placed to drive commercial growth in the aquaculture sector to produce a consistent supply of quality fish. Table 2 provides the results framework based on the project’s activities: Table 2: CAST activities and their link to the results framework Project activities Link to the results framework Activity 1: Capacity building: Agricultural extension agents/services Access to training and level of participation Activity 2: Inputs: Develop agro-dealers and other input suppliers Product quality, marketing, adoption of standards, leverage of private sector resources Activity 3: Training: Improved farm management Improved linkages between buyers and sellers, access to market information Activity 4: Training: Demonstration plots Access to training and level of participation Activity 5: Training: Sanitary and phytosanitary standards (SPS) Product quality, adoption of standards, use of post-production processing and handling, government institution capacity, policy and regulation Activity 6: Capacity Building Product quality, the efficiency of post-production processes, marketing, market linkages, trade infrastructure, adoption of standards, use of post￾production processing and handling, post-harvest infrastructure, use of financial services, leverage of private sector resources Activity 7: Capacity Building: Producer groups/cooperatives Product quality, marketing, market linkages, management of buyer/seller groups, adoption of standards, government institution capacity, policy and regulation, the capacity of trade organizations, leverage of private sector resources Activity 8: Training: Improved marketing and branding The efficiency of post-production processes, marketing, market linkages, trade infrastructure, management of buyer/seller groups, adoption of standards, use of post-production processing and handling, use of financial services, government institution capacity, access to market information, leverage of private sector resources Activity 9: Market Access: Facilitate a traceability system for domestic aquaculture Product quality, marketing, management of buyer/seller groups, adoption of standards, use of post-production processing and handling, government institution capacity, policy and regulation, access to market information Activity 10: Financial Services: Provide SME finance, technical assistance loans and grants The efficiency of post-production processes, trade infrastructure, adoption of standards, use of post-production processing and handling, post-harvest infrastructure, use of financial services, leverage of private sector resources 14 1.5. Purpose of the Evaluation The present study is a mid-term evaluation (MTE) of the five-year project, conducted during its third year. The purpose of the evaluation is to measure the project's progress to date and to relay this information to the project’s implementing agency (PIA), ASA/WISHH and its donor, the United States Department of Agriculture (USDA). Specifically, the MTE seeks to provide meaningful feedback about what progress has been made towards reaching performance indicator targets, assessing the project's relevance, providing insight about its effectiveness, documenting lessons learned, assessing sustainability, and discussing and recommending course corrections. Finally, the MTE shall also highlight the strengths and weaknesses of CAST and make recommendations for the next steps. The CAST Strategic Analysis explains that “opportunities for value chain upgrading that further enable producers to capture price premiums and to enter in higher-value markets will be a cornerstone of early-project adaptive management through the comprehensive value chain analysis.”13 The present MTE will contribute to the Learning Agenda by reviewing the success of these early efforts to build market linkages via project activities. 2. Evaluation Design and Methodology 2.1. Evaluation Design The MTE was carried out using a mixed-methods approach. During the inception stage, a desk review was carried out, reviewing project-related documents, data and reports. The list of documents reviewed is listed in Annex 9. An inception workshop and several meetings were also held to obtain inputs and buy-in from various project custodians. Because of the pandemic situation, CAST had faced many challenges, and a key focus for the initial discussion was how the MTE could best contribute to the project. The final MTE plan was recorded in an inception report. Following the inception stage, key informant interviews (KIIs) were done with CAST staff, partners, stakeholders, beneficiaries and donors. At the same time, a Data Quality Assessment (DQA) was done to evaluate the M&E protocols established under CAST. The DQA included interviews with CAST M&E staff and reviewing relevant documents, including the Data Analysis Plan (DAP), Data Collection Manual (DCM), indicator performance tracking table (IPTT) and performance monitoring plan (PMP). The results from the KIIs helped inform the development of the questionnaire for the MTE survey, which took place next. The MTE survey covered all supply chain actors, including hatcheries and nurseries, pond and cage fish producers, wholesalers and collectors, processors, feed mills and feed distributors. Due to the pandemic situation in Cambodia, which was escalating during the MTE, data collection for the MTE survey was executed by phone from a target respondent list provided by CAST. During the phone interview, answers were recorded on computer tablets using CAPI.14 13 CAST Strategic Analysis, p. 21. 14 Computer aided personal interviewing. The system used was Survey Solutions developed by the World Bank Group. 15 2.2. Evaluation Questions The MTE was conducted to answer the following set of evaluation questions. Relevance  To what extent is the project aligned with national policies and strategies for aquaculture and feed production?  How relevant is the project to the stated and immediate needs of stakeholders and beneficiaries? Effectiveness  To what extent is the project producing beneficial results (outputs, outcomes) and meeting its objectives?  What, if any, unintended consequences to the aquaculture sector can be attributed to the project? Efficiency  To what extent is the relationship between inputs and outputs timely, cost-effective and to the expected standards?  To what extent has the project been efficiently implemented and managed (including financial review and audit information), as appropriate? Impact  Has the project contributed to short- and medium-term socio-economic improvements for aquaculture producers, hatcheries and nurseries, and feed mills?  What difference has the project made in the lives of beneficiaries? Sustainability  To what extent has the project fostered ownership by local and regional partners (e.g., CE SAIN, CTU)? To what extent has the project established/enhanced capacity, processes and local systems that are likely to be sustained? 2.3. Sampling Methods The KIIs were carried out following the agreed implementation plan. Using a purposive selection method, participants were selected from CAST's target list of potential respondents. To ensure high levels of participation, CAST provided an endorsement letter explaining the purpose of the MTE and sent out a communique to all potential participants to encourage them to participate. Potential participants were first contacted by phone to set an appointment for the interview, after which they were sent the endorsement letter. Participation overall was good, with very few refusals or non￾response contacts. The MTE survey was carried out in the form of a census, approaching all beneficiaries across the supply chain that had signed up with the CAST program from September 2018 to December 2020. Names and contact phone numbers were obtained from CAST’s ODK system. The list included both business and 16 individual supply chain contacts and meant some businesses clusters were present. The list was shared with CAST to verify who the unique business contacts were, and a unique identifier was added to the contact list that would be recorded as part of the survey. This was necessary to ensure the business level and individual level responses could be distinguished. 2.4. Data Collection Methods 2.4.1 Enumerator and moderator training Rapid Asia organized training online for the field data collection team. In total, eight MTE survey enumerators and two local KII moderators were trained. The training for the KII moderators took place on March 31, 2021, and the training for the MTE survey enumerators took place on May 14, 2021. Both the MTE survey enumerators and KII moderators received detailed training via Zoom that included an overview of CAST shared by CAST staff. The enumerators and moderators were also trained to execute the questionnaire and discussion guide to ensure familiarity and understanding. In particular, the KII moderators received training on ethical guidelines and how to use the informed consent form. For their part, the enumerators focused primarily on the questionnaire, including how to use CAPI on a table, the calling process, and how to navigate the questions and ensure completeness. The enumerator supervisors were also trained on how to conduct quality control checks. Following the training, the MTE survey was pre-tested with ten respondents before continuing fieldwork. The participants shared feedback and lessons learned from the training and exercise. The pretest data were checked to ensure the scripting of the CAPI survey was correct and that respondents were able to answer questions over the phone. Since no major issues were found, the survey continued, and the data collected from the pretest was retained. 2.4.2 KII with CAST staff, partners, beneficiaries, stakeholders and donor As shown in Table 3 below, a total of 27 KIIs were carried out with ASA/CAST staff, partners, beneficiaries, stakeholders and donor representatives. Five experienced moderators were deployed to conduct KIIs, including three staff members from Rapid Asia who conducted KIIs for whom English was the preferred language. The remaining KIIs were done in Khmer by two local moderators. A moderator guide was developed for the KII (Annex 5). Data collection for the KII was carried out over the course of one month, from the end of March to the end of April 2021. Table 3: KII participants CAST n CAST staff 6 CAST partner World Vision International (WVI) 2 Kansas State University (KSU) 3 Auburn University (AU) 1 Can Tho University (CTU) 1 EU CAP Fish 2 CAST Beneficiaries 17 Supply chain representatives15 6 Financial institution Chamroen MFI 1 AMK MFI 1 Government organization United States Department of Agriculture (USDA) 1 MAFF/FiA 3 Total 27 Due to the COVID-19 situation, all KIIs were done via phone or Skype. Unlike surveys with farmers, KIIs are most often conducted indoors, where COVID-19 can spread more easily. Some participants were from overseas and were interviewed by senior Rapid Asia staff. Hence, a uniform phone survey was the safest and best approach to use. CAST provided a contact list, and all participants were initially contacted by email to encourage their participation. The field team subsequently contacted them to set an appointment for the phone interview. All interviews were recorded with an MP3 audio recoder for quality control purposes, and answers were translated into English and detailed in a summary template. 2.4.3 Survey with supply chain actors The MTE survey questionnaire was developed in collaboration with CAST, and once finalized, it was translated, and the translation was checked for accuracy before scripting and uploading on the CAPI system. In addition, a test link was set up so that the online MTE survey could undergo testing and ensure that it was free from errors. Using CAPI, hosted on a secure cloud server, data collection was done via a phone survey between 17 May and 4 June 2021. Eight experienced enumerators were deployed for the MTE survey together with one field supervisor and one audit staff member. Both Rapid Asia and the supervisor had the ability to monitor interviews in real-time. The supervisor assisted by checking completed interviews before submitting them to Rapid Asia, who conducted additional checks of the data collected to ensure it was consistent and complete. In case of inconsistencies, the interview was sent back to the enumerators who had to follow up with the respondent. Data collected was, in most cases, of high quality, but there were some cases of outliers and inconsistent answers that needed to be confirmed. In addition, an independent audit team did random callbacks via phone to audit 20% of all completed interviews. The audit includes making sure an interview took place with the intended respondent and cross-referencing answers to selected questions. Attempts were made to contact all supply chain actors under CAST, which included all beneficiaries that were enrolled and received services by the end of December 2020. Each respondent on the contact list was contacted up to eight times to minimize the non-response rate as much as possible. Contact outcomes were recorded and monitored on a tracking sheet and progress was reported to CAST on a regular basis. Several numbers were found to be incorrect, or the beneficiary had changed their SIM card. CAST was able to track down some of the beneficiaries and update their contact details. Relatively few (9%) of respondents refused to participate in the MTE survey, and the main reason was 15 Beneficiaries included two distributors, one feed mill, one feed distributor, and two input providers and CAA members. 18 that they were busy or a person in the same business cluster had already been interviewed.16 The outcome of the MTE survey is shown in Table 4 below. Six beneficiaries included in the beneficiary contact list provided by CAST had ceased their aquaculture business or became CAST staff. Hence, they were not eligible for an interview. The final individual response rate was 71% and business response rate was 78%. These are regarded as a strong result given the limitations around doing a phone survey from a finite contact list. Table 4: Phone survey outcome Survey outcome No. % Original database 449 - Ineligible records17 6 - Total contact database 443 100 Incorrect numbers 31 7 Refusals 38 9 Non-response 61 14 Successful interviews 313 71 Overall, the MTE survey managed to obtain solid representation from all beneficiary groups across the supply chain and across the different provinces in which CAST is operating. Since the analysis had to be done for each value chain actor separately, and some target groups are relatively small, the impact of sampling errors need to be considered. Because smaller, finite populations represent the supply chain actors, the actual sampling error is reduced significantly. It is normally assumed that the population is extremely large and that we have sampled a small part of the population. The Finite Correction Factor (FCF) was used to adjust the variance of the sampling distribution and is appropriate when more than 5% of the population is being sampled. The full questionnaire was administered to all beneficiaries. It should be noted that the total number of beneficiaries interviewed includes some beneficiaries from the same business cluster. Collecting data for the entire questionnaire from this group was seen as relevant for evaluating their appreciation of CAST training and technical support. Table 5 below shows the individual-level response and will be applicable for results that deal with training and technical support services provided by CAST. The overall response rate was 71% but varied across value chain actors. The resulting maximum sampling error ranged from a low 3.7% for producers up to 16.5% for hatchery/nursery.18 Table 5: Response rate and sampling error at the individual level Value chain actor Individual level Contact list (n) Response (n) Response rate (%) Maximum error (%) Hatchery/ nursery 20 13 65% 16.5% 16 The contact database included several beneficiaries who worked for the same business. CAST requested they be interviewed as well and upon completion of data collection, CAST identified who the main respondent should be from each business cluster. 17 Six beneficiaries included in the beneficiary contact list provided by CAST subsequently ceased their aquaculture business or joined the CAST. Hence, they were not eligible for an interview. 18 Calculated at the 50% level. See: *Source: https://www.sphanalytics.com/sample-error-calculator/ 19 Producer 295 208 71% 3.7% Wholesaler/ collector 39 28 72% 10.0% Processor 27 17 63% 14.7% Feed mill 50 37 74% 8.3% Feed distributor 12 10 83% 13.2% Total / average 443 313 71% 3.0% When estimating progress on key indicators and related results, only unique businesses should be considered. Table 6 shows the business level response across value chain actors. There were responses from all project beneficiaries for feed mills and feed distributors resulting in a sampling error of zero. Using the FCF, as explained above, producers have a low error at just over 3%, and the remaining actors were higher at around 10%. Table 6: Response rate and sampling error at the business level Value chain actor Business level Contact list (n) Response (n) Response rate (%) Maximum error (%) Hatchery/ nursery 11 10 91 9.8 Producer 263 199 76 3.4 Wholesaler/ collector 25 18 72 12.5 Processor 16 14 88 9.6 Feed mill 6 6 100 0.0 Feed distributor 8 8 100 0.0 Total 329 255 78 2.9 The respondent profile is shown in Table 7 below. Based on the contact list provided by CAST, most supply chain actors are male and aged around 40 years.19 Apart from the six feed mills located in Phnom Penh, beneficiaries are spread across the six provinces and greater Phnom Penh, areas in which the CAST program is operating. Table 7: Beneficiary profile Demographics Base: all respondents Wholesaler/ Collector n=18 (%) Processor n=14 (%) Hatchery/ nursery n=10 (%) Producer n=199 (%) Feed mill n=6 (%) Feed distributor n=8 (%) Sex Male 61 36 80 83 100 100 19 Freshwater fish production enterprises are typically family businesses, traditionally led by the husband/father. CAST’s preliminary and subsequent early contact with the farmers almost universally has funneled through the husband or father. 20 Female 39 64 20 17 - - Region Phnom Penh 17 7 10 1 100 - Kandal - - - 24 - - Kampong Cham 44 7 10 25 - 50 Kampong Thom 17 29 - 13 - - Siem Reap 6 36 30 13 - 13 Battambang 6 7 40 11 - 38 Pursat 11 14 10 16 - - Age Average age (years) 44 39 46 46 37 45 2.5. Data analysis The quantitative data was continually checked during data collection on the CAPI system, and the final data set was cleaned to weed out outliers and back-code some of the open-ended responses. Some respondents were also re-contacted to confirm unclear or inconsistent responses. Data analysis was performed using SPSS, and the results were compiled in the form of charts and tables, used throughout this report and in Annex 2. Most of the analysis was in the form of frequency counts and cross-tabulation. Analysis was consistently performed across the six value chain actor segments. This was necessary as many of the questions in the questionnaire were value chain actor specific. Due to the limited sample size for most of the value chain segments (see Table 6), there were limitations in performing further sub-segment analysis beyond the six value chain segments. Hence, most of the analysis is based on frequency counts. Producers had a larger sample, and it was possible to perform Chi-Square analysis across all relevant variables in the data set to determine any significant differences. The qualitative interviews were reviewed for clarity and completeness before beginning analysis. Internal triangulation was first done by examining the responses from each interview and extracting relevant findings that several stakeholders consistently mentioned. The findings formed a comprehensive view of the perspectives and perceptions held by the KII participants. When a mixed method is used, triangulation becomes vital to ensure results and recommendations are consistent and supported by relevant data sources. In this case, the different data sources were also subject to different data collection methods and meant the 'multilevel combinations method' for triangulation was used.20 First, data from the desk review, phone survey and KII were analyzed separately, and key findings were identified and agreed upon. Second, all key findings were linked back to set evaluation questions and indicators. Finally, triangulation was carried out by examining key findings across different information sources. Key findings found to be supported by more than one source were included. Findings not supported but not contradicted were included if the source is seen to be reliable. 20 USAID, Conducting mixed-method evaluations, Technical Note, Washington, D.C., June 2013. 21 2.6. Data quality assessment (DQA) A data quality assessment (DQA) was carried out as part of the MTE. Purposes of the DQA were to (i) evaluate key M&E functional areas (e.g., data entry protocols, indicator definitions, reporting tools) for CAST; (ii) provide CAST with a better sense of where improvements could be made (e.g., filing practices, reporting timelines, data verification); and (iii) help donors, CAST staff and implementing partners understand the strengths and weaknesses of the data collection processes that influence management decision-making and are integral to the project. The DQA was done by reviewing CAST’s M&E related documents, including the Data Collection Manual (DCM), Data Analysis Plan (DAP), Performance Monitoring Plan (PMP) and Indicator Performance Tracking Table (IPTT). In addition, interviews were carried out with four of CAST’s M&E staff responsible for collecting, processing, and reporting on project data. The DQA guidelines, including the five data quality standards, which are “Validity,” “Reliability,” “Timeliness,” “Precision,” and “Integrity,” were used to determine the extent to which written data collection procedures were in line with their implementation in practice. As part of the DQA, the impact COVID-19 has had on the CAST was also considered. The focus of the DQA was around six standard data collection processes adopted for the project, including (i) new beneficiary enrollment; (ii) training/workshop attendance; (iii) technology adoption; (iv) sale records; (v) financial service records; and (vi) technical assistance records. Each of the six data collection processes was assessed against the five data quality standards. A moderator guide was developed around the DQA’s five data quality standards (Annex 6). Following the review of relevant M&E documents and the four staff interviews, a three-point scale assessment (None/limited: 1; Partially: 2; Mostly/fully: 3) was adopted to score each of the five data quality standards. Following is the summary of DQA results. The full DQA report is found in Annex 1. Table 8 below shows a summary of the results. Table 8: DQA results21 Data quality standard/ data collection process Validity Reliability Timeliness Precision Integrity New beneficiary enrollment 3.0 3.0 2.0 2.5 2.5 Training/workshop attendance 3.0 2.5 3.0 3.0 3.0 Technology adoption 2.5 3.0 2.0 3.0 3.0 Sale records 2.5 3.0 2.0 2.5 3.0 Financial service records 3.0 2.5 3.0 3.0 3.0 Technical assistance records 3.0 3.0 3.0 3.0 3.0 21 Color legend: Green, Mostly or fully; Amber, Partially. 22 2.7. Limitations Factors presented as limitations to the execution of the MTE include the following.  The MTE focused on the supply side and not did not directly focus on the demand side. Some producers have expanded their production facilities as a result of being part of CAST. Whilst that is a good development, farmers face reductions in the market price for fish due to cheaper imports. CAST implements activities related to market development, but these have not been evaluated as part of the MTE. To emphasize this gap, they are recognized in the MTE report as well as in the recommendations.  Due to the outbreak of COVID-19 and related travel restrictions in Cambodia, all data collection had to be conducted virtually via phone or online communication tools and included the MTE survey with beneficiaries and KII with project staff, partners, stakeholders and beneficiaries. A phone survey was not ideal, especially for the MTE survey, which had to be streamlined and simplified to accommodate a shorter interview time. Hence, primary data for most of the indicators could not be collected as initially planned. This was not seen as too critical since CAST regularly collects this data from beneficiaries, and reasonably updated information could therefore be obtained from their ODK database.  In addition to producing aggregated data for the standard and custom indicators, the idea was to merge the ODK data with the final data collected in the field. This would allow for additional analysis to be performed. However, upon review, this proved not to be possible. The statistical package used (i.e. SPSS) does not have the ability to process multiple databases in the same way that other systems can. This system incompatibility was not anticipated and was unfortunate. A discussion was held with the CAST team to see if merging the data in the ODK system was possible. However, the collaboratory effort required proved to be too time￾consuming.  During analysis, the best effort was made to identify differences between sub-groups within each of the six supply actor business segments. Dealing with mostly nominal data, Chi-Square analysis was performed. However, all supply actor segments had a relatively small sample base with fewer than 30 respondents apart from producers. Because of this, it was not possible to prove any significant differences between sub-segments. That is not to say differences don’t exist, but rather, the statistical power to prove any differences was limited by the small sample size.  The KII list was adjusted to include stakeholders actively involved in the CAST program (e.g., WVI, KSU, AU, CTU, EU CAP Fish, MFIs, USDA and MAFF). Other parties who have not yet been involved (e.g., NARDI and IFREDI) were not included. This means the KII results might not be fully inclusive and representative. However, it was felt that talking to those who had been involved with the CAST program would provide more meaningful feedback for the MTE.  Training on the SPS standards has been delayed. Because of this, Custom #5, the number of market actors selling fish compliant with national SPS standards or using an independent quality seal, was omitted from the MTE. 23 3. Findings 3.1 Findings from the desk review The desk review provided a number of lessons learned and contributed to the MTE in the following ways. a. Using a phone data collection methodology highlighted the need for streamlining, and it was decided the survey should focus on output and process indicators. b. The calculation of standard indicators 2, 3, and 4 in the baseline survey, had to be changed since the focus on practices and technologies under CAST has since been modified. c. Technical input (confirmed with the appointed aquaculture expert in Cambodia) from various reports informed the inception report and development of tools. d. An extensive part of the desk review was devoted to the DQA, for which several key documents had to be reviewed to prepare a suitable tool. e. Fish imports from Vietnam stood out as a challenge for the aquaculture industry in Cambodia and highlighted the need to improve farming practices and determine how CAST has contributed. f. Several sources were also identified and have been referenced in this report. Since its inception, CAST has experienced several constraints, notably the outbreak of the COVID-19 pandemic, budget reduction (nearly USD 2 million), and the US government's restrictions on working with the Cambodian government agencies.22 These resulted in project delays and some components, such as inputs provision, training and other capacity-building activities. This, in turn, impacted the MTE in the following ways. a. All the activities associated with the MTE were conducted virtually via phone or online communication tools. This included (i) inception workshop; (ii) enumerator/moderator trainings; (iii) KIIs; (iv) MTE survey; and (iv) validation workshop. b. The KII moderator guide and MTE survey questionnaire were adjusted to accommodate remote data collection by phone. They also included some questions related to COVID-19 and its impact on their business operations. c. The list of stakeholders and partners for KIIs were adjusted to include those actively involved in the CAST. Other parties who have not yet been involved (e.g., NARDI and IFREDI) were excluded. d. Training on SPS standards has been delayed. As such, custom indicator #5 (the number of market actors selling fish compliant with national SPS standards or using an independent quality seal) was omitted from the MTE. 3.2 Project progress on standard and custom indicators This part of the MTE evaluates how CAST performed against set standard indicators (SI) and custom indicators (CI). The initial plan was for the survey to focus on the set indicators alone. However, after review during the inception stage, the questions required to collect data for many indicators were deemed not suitable for a phone survey.23 Because a phone survey methodology had to be used due to the escalating pandemic situation in Cambodia, the interview could also not be too long. Hence, there were limitations to the amount of information that could be collected. 22 According to CAST staff interviewed, the CAST program originally planned to work directly with the Government entities NARDI and IFREDI. However, US government policy was issued that prevents CAST from directly support the work of host government institutions. 23 For example, collecting information on pond size, production, and sales data. 24 After consultation with CAST, it was decided that the MTE survey should focus on activities that CAST had been working on to date. Many activities were affected by the pandemic, and implementation plans had to be adjusted. Hence, understanding how the project has performed in this challenging environment was seen to be more useful and informative and would bring more value to the program moving forward. Following this new approach, it was further agreed that data for most of the set indicators could be sourced from the ODK system. The relevant indicators would still be reasonably recent as they had been updated in the 2021 April CAST semi-annual report. The April data was based on all farmers under CAST and more complete than what could have been achieved through a phone survey. Table 9 below shows the aggregated results for the set standard and custom indicators. The base for the indicators is represented by the 329 actors (including 443 individuals) who had registered with CAST. Baseline values, where applicable, and program targets have also been included. Table 9: Results for standard and custom indicators Standard Indicators August 2019 April 2021 2023 # Indicator Baseline24 Result25 Target26 1 Yield (Kg/Hectare) of targeted agricultural commodities. 47,700 130,737 603,600 2 Hectares under improved management practices or technologies that promote improved climate risk reduction and/or natural resources management. Not applicable 162 245 3 Number of hectares under improved management practices or technologies. Not applicable 162 350 4 Number of individuals in the agriculture system who have applied improved management practices or technologies. Not applicable 44 777 12 Number of organizations with increased performance with USDA assistance.27 0 0 363 13 Number of public-private partnerships formed as a result of USDA assistance. 0 12 10 18 Value of annual sales (USD) of farms and firms. Not applicable 19,208,908 497,318 19 Volume of commodities (metric tons) sold by farms and firms. Not applicable 11,187 40,201 21 Number of individuals who have received short-term agricultural sector productivity or food security training as a result of USDA assistance. 0 416 1,110 22 Number of individuals participating in USDA food security programs. 0 416 1,123 24 CAST Cambodia Baseline study report, August 2019. Because the target beneficiaries changed after the basline study was done, some indicators are not comparable. 25 Standard indicators 1,2,3,4,12,13,18,19 were obtained from CAST Semi-annual report (SAR) April 2021. Standard indicators 21 and 22, and custom inidicators 1 and 6 are based on results from the MTE survey. 26 CAST IPTT, New Life of Project Target 27 This indicator is based on adoption of at least two technologies or practices, having a clear farm plan and a yield increase of at least 20%. With an overall market decline this indicator has remained at zero. 25 Custom Indicators # Indicator 1 Number of actors28 who received short-term agricultural sector productivity, food security or other training as a result of USDA assistance. 0 310 740 2 Area of ponds (cubic meters) under improved management practices or techniques with USDA assistance. Not applicable 162,00029 3,502,240 6 Number of farms that increased the number, size, or amount of ponds, cages, or equipment with USDA assistance. Note: see also figure 9 and 10, section 3.3 D Impact 0 24 207 3.1.1 Production and value of aquaculture products This section concerns Standard #1 (yield), Standard #18 (value of sales) and Standard #19 (volume of sales). To add context, value chain actors were asked in the MTE survey how fish production, sales and sales prices had changed in the last production cycle. Due to the loss of customers and the impact of COVID-19, two thirds (67%) of wholesalers and collectors reported a decline in Cambodian aquaculture purchases with an average decrease of 94 Tons (Figure 1). An average negative result was also reported by 64% of processors, with an average decline of 22 tons. Whilst there were one or two actors who saw an increase on average, overall, the data indicates that purchases of Cambodian aquaculture declined. The MTE survey did not ask specifically why wholesalers and processors had lost customers, and only some had increases in purchases. But based on interviews with the stakeholders, there was a consensus that the main challenge is cheaper imports from Vietnam. Figure 1: Change in aquaculture fish purchase last production cycle Producers within CAST did not experience an equally drastic decline in production ( 28 Private enterprises, producer organizations. 29 Additional ponds and cages added since joining CAST was asked in the MTE survey. The additional volume is assumed to be included here since the ODK data was updated in April 2021. 26 Figure 11: Change in fish production (Producer)). However, even if producers within CAST maintained their production levels, 20% of hatcheries and nurseries experienced an average 35% drop in the sales price, and 55% of producers saw a price drop of 20%, shown in Figure 2. The MTE survey did not specifically ask why the price had declined but based on interviews with various stakeholders, many attributed this decline to competition from Vietnam. Due to larger volumes and higher efficiency, Vietnam can export what they produce at a lower price than local producers in Cambodia. Figure 2: Change in sales price per kilo Pangasius is the most popular fish species produced by producers with 81%, followed by Giant Snakehead with 26% and Silver barb with 18% (Table 10). In contrast, hatcheries and nurseries produce mostly Silver Barb (90%), Clarias (80%) and Tilapia (60%). Keeping in mind that hatcheries and nurseries only represented 10 businesses, it is unclear whether this disconnect represents an imbalance in the market or if the difference reflects the market when taking all suppliers (including imports) and producers into account. Additional analysis was done with producers to determine a relationship between the type of fish species produced and the change in sales price. No significant relationship was found for most fish species, but there was an indication that those growing Giant Snakehead may have seen a more substantial price decrease of 71%. Table 10: Fish species produced Fish species produced Base: all respondents Hatchery/ nursery n=10 (%) Producer n=199 (%) Silver Barb 90 18 Clarias (walking catfish) 80 15 Tilapia 60 9 Pangasius 20 81 Annabas 20 4 Carp species 10 5 Giant Snakehead - 26 Channa striata - 6 Other - 10 27 3.1.2 Management practices Improved management practices are covered under Standard #2, Standard #3, Standard #4 and Custom #2. It should be noted that Standards #2 and #3 are measured in the same way, and Custom #2 is similar to Standard #3 but uses cubic meters instead of hectares. These indicators are the result of all designed techniques/technologies being climate change adoption related. Improving management practices has been a strong focus under the CAST program. Since its inception, CAST has identified 17 key practice domains that have become the focus of the program. At the time of the MTE survey, CAST had provided support in all but four areas, as shown in Table 11 below. Table 11: Progress on training development topics No. Training topic Training provided 1 Carrying Capacity Yes 2 Feed Conversion Ratio (FCR) Yes 3 Feeds and Feeding & Feed Storage Yes 4 Pond Preparation Yes 5 Fingerling Quality Yes 6 Fingerling Handling, Transport and Stocking Procedures Yes 7 Water Management & Water Budgets Yes 8 Water Quality Yes 9 Aeration Yes 10 Disease Prevention / Biosecurity No 11 Farm Management Yes 12 Record Keeping Yes 13 Farm Enterprise Planning Yes 14 Cost of Production Yes 15 Pond Construction No 16 Cage Farm Design No 17 Cage / IPRS Aquaculture No Many hatcheries and nurseries, and producers, are affected by floods and drought. To mitigate climate change, half of the hatcheries and nurseries have fenced ponds and a water well, and 60% of producers have fenced ponds, and 42% have water well. Around one in six producers use fast-growing fish or target small harvest sizes to mitigate climate change. Most hatcheries and nurseries use burnt lime or chemicals to deal with predators (at least sometimes). Additionally, 40% do not have a target date for when fingerlings will be ready. Fewer producers use burnt lime or chemicals, and 82% knew their production cost, and 65% kept records of their fish production. However, 80% said they know the production cost of fingerlings, and 70% kept records of fingerling production. 28 Feed management is critical for effective aquaculture. All hatcheries and nurseries and 85% of the producers are now using pellet feed. The use of pellet feed varied to some extent. Producers of Giant Snakehead were less likely to use pellet feed, and producers of Pangasius were more likely to use it. However, just over half measure the feed they give each day and around two-thirds record how much feed they use. One of the major concerns fish producers have is the cost of feed, especially commercially produced feed, which tends to be expensive. Therefore, understanding how much feed to give is paramount, and the feed conversion ratio (FCR) is an effective tool to ensure the right amount of feed is given at any given time. The MTE survey asked several questions about the FCR to determine the extent to which value chain actors understand its usage. The results are summarized in Table 12 below. The results show that whilst most of the actors have heard of the FCR, only feed mills know how the FCR is calculated and how the ratio should be interpreted. Producers, in particular, show a significant knowledge gap in this area. Table 12: Feed conversion ratio FCR knowledge Base: all respondents Hatchery/ nursery n=10 (%) Producer n=199 (%) Feed mill n=6 (%) Feed distributor n=8 (%) Heard of FCR Yes 100 75 100 63 No - 25 - 37 Know how the FCR is calculated Feed weight / fish weight gain 30 11 83 - Don’t know 70 89 17 100 Know that a low FCR is usually better Yes 30 5 83 20 No 70 95 17 80 Water quality is another area where CAST is working to improve knowledge and practice. Active fish, green-colored water, algal boom and clear water are typical signs of good water quality. Green colored water is a good indicator to ensure water quality is good, and this will allow phytoplankton to grow. Phytoplankton is the feed for fish. Most hatcheries and nurseries and producers tend to rely on one of the signs rather than multiple ones. Most hatcheries and nurseries (70%) look to see if the fish is active, and producers (76%) rely mostly on whether the water is clear. This question was also asked to feed mills and feed distributors. The most common response from feed mills was algal bloom (50%), whereas half the feed distributors (50%) said they did not know (See also Annex 2). With respect to preventing water turbidity, several techniques can be used, including preventing erosion, adding water or manure to the pond. Again, hatcheries and nurseries and producers may, on average, use one technique that they are familiar with but seldom more than that, and 45% of producers and 20% of hatcheries and nurseries did not know of any technique (see also Annex 2). The MTE survey did not elaborate on why this was the case. 29 Fingerling quality has become a focus area for CAST, given the need to ramp up fingerling production and quality in Cambodia. Fingerling quality is determined by several factors including size, good appearance, fingerlings being similar in size, same species and same age (Table 13). Feed mills had an excellent understanding of this, followed by hatcheries and nurseries. Producers mostly rely on fingerling appearance. Table 13: Fingerling quality Determine fingerling quality Base: all respondents Hatchery/ nursery n=10 (%) Producer n=199 (%) Feed mill n=6 (%) Feed distributor n=8 (%) Big size, 5 grams or more 60 31 50 - Good appearance 90 76 100 13 Similar in size 40 38 100 - Same species 40 26 67 - Same age 20 10 100 - None - 7 - 87 Average knowledge30 2.5 1.8 4.2 0.1 Most hatcheries and nurseries consider water PH level, water temperature and fingerling size when stocking fingerlings. Most hatcheries and nurseries were aware of how many fingerlings they stock per square meter of the pond (80%). In addition, 80% also said they knew the survival rate of the fingerlings, which was 77% on average, which indicates there is room for improvement. On the other hand, fewer producers knew how many fingerlings they stocked per square meter (37%), and 25% said they had changed how many fingerlings they stocked. The transportation of fingerlings represents a critical phase in fish production and can severely impact survival rates and fish health. The vast majority (90%) of hatcheries and nurseries use plastic bags with oxygen when transporting fingerlings, and 30% using hauling tanks. 3.1.3 Training and technical support Standard #21, Standard #22 and Custom #1 look at the number of individuals and businesses that received short-term agricultural sector productivity or food security training or technical support. The MTE survey asked whether participants had received training or technical support, and means results are more closely aligned with Standard #22. In total, 294 individuals (94%) and 240 businesses (94%) had received training or technical support from CAST, representing the vast majority of those who have signed up with the program. The training covered 21 topics, ranging from entrepreneurship to operations management, as can been seen in Table 16 on page 39. As covered in subsequent chapters, the training has been well received, resulting in beneficiaries being able to do their work more effectively, and most have also shared what they learned with other farmers. 30 Average knowledge is calculated by adding the percentages in each column and dividing by 100. 30 To extend the reach of the technical support provided, CAST worked with feed mills and feed distributors. These actors were trained so that they, in turn, could provide better support to the farmers they served. Figure 3 below shows the support feed mills and feed distributors have provided to their customers. The feed mills appear to have a better capacity to provide support than feed distributors, who mainly do feed inspections, teach customers how to use feeding tables, and store feed. Figure 3: Technical support feed mills and feed distributors provide to customers Beyond the technical training, CAST has also supported individual beneficiaries in other ways, as shown in Table 14 below. Technical assistance has been rather extensive, reaching between 75 and 100% of all beneficiaries. Support services have also been included in public events, workshops, publications, online, and loans. Table 14: Services received from CAST Services Base: all participants Hatchery/ nursery n=13 (%) Producer n=208 (%) Wholesaler/ collector n=28 (%) Processor n=17 (%) Feed mill n=37 (%) Feed distributor n=10 (%) Technical assistance 92 85 75 82 100 90 Public event 69 37 18 6 19 20 Meeting or workshop 85 69 64 65 51 80 Publication 85 53 54 41 65 40 On-line 46 37 50 47 49 50 Loan 15 19 29 12 16 10 Other (i.e., grant or not receive any service from CAST) - 3 7 - - - 31 In addition, WVI via CAST has provided market support for project beneficiaries. In particular, WVI has supported the establishment of market linkages with key distributors such as Unica enterprise, Davane and various cooperatives, such as organic cooperatives.31 WVI has also engaged in a consumer awareness campaign targeted at the general Cambodian public. In particular, the campaign aimed to inform Cambodian consumers about the quality and safety of aquaculture fish and sanitation standards and practices. 3.3 Evaluation Question Results Information was gathered through KIIs and the MTE survey regarding the a) relevance, b) effectiveness, c) efficiency, d) impact and e) sustainability of the project. In addition, the analysis examined the factors that contribute to and inhibited the success of the project to date. Please note that all results in this section have been thoroughly analyzed and triangulated. Quotes presented by individuals should be regarded as representative of a broader view held by multiple stakeholders. A. Relevance A.1 To what extent is the project aligned with national policies and strategies for aquaculture and feed production. The Cambodian government adopted the National Strategic Plan for Aquaculture Development in Cambodia in 2016, with an implementation period of 15 years.32 The plan is implemented primarily by the Ministry of Agriculture, Forestry and Fisheries (MAFF), its Fishery Administration (FiA) and other associated organizations (e.g., Inland Fisheries Research and Development Institute). The plan seeks to create “a commercially viable and environmentally sustainable aquaculture sector contributing to food security and nutrition, socioeconomic development, GDP and export earnings” in Cambodia. As such, it prioritizes aquaculture production and market development as key components. In particular, the plan lays out steps for improving aquaculture fishery and fingerling production, SPS and traceability standards, and access to markets and finance. The Cambodian government also recently established a loan scheme via Agriculture & Rural Development Bank (ARDB), which provides low-interest loans to aquaculture farmers. The objectives of the National Strategic Plan are to: a. Increase access to high-quality seed for a range of species in demand in local, regional and global markets; 31 Key informant interview, April 10, 2021. 32 Royal Government of Cambodia Ministry of Agriculture, Forestry and Fisheries. National Strategic Plan for Aquaculture Development in Cambodia, 2016 to 2030. 32 b. Increase access to sufficient and consistent supplies of high-quality water and to reduce flood risks; c. Ensure widespread availability and efficient use of sustainably sourced, reasonably priced, high-quality feed suitable for a range of species; d. Improve efficiency, profitability and sustainability of aquaculture production through increased knowledge, skill and organization; e. Maintain environmental quality and minimize loss from disease; f. Increase the quality and value of production; and g. Facilitate access to credit as appropriate to the needs, potential and risks associated with aquaculture development.33 CAST aligns with the objectives of the National Strategic Plan for Aquaculture Development in Cambodia. For example, most CAST activities directly or indirectly contribute to six of the seven plan objectives, as shown in Table 15 below. A CAST administrator explained the level of synergy as follows: “The activities, objectives and outcomes of the [CAST] project are 90% aligned with the strategic plans and policies.” (ASA/CAST staff, Phnom Penh) In particular, CAST supports the development of aquaculture businesses from small to large to increase aquaculture fishery production, one of the goals of the National Plan. Additionally, the Cambodian government has sought to stamp fishes that meet quality standards to ensure quality and transparency. CAST aims to introduce this concept as part of the project. Finally, the Cambodian government wants to use hatcheries as training sites for commercial aquaculture farmers. This goal is also aligned with CAST’s activities: enlisting public and private hatcheries as critical players to help train production level supply chain actors to facilitate the extension of the industry. Table 15: CAST alignment with the National Strategic Plan Project activities Objectives of the National Strategic Plan for Aquaculture Development in Cambodia Activity 1: Capacity building: Agricultural extension agents/services Objective 4) to improve efficiency, profitability and sustainability of aquaculture production through increased knowledge, skill and organization Activity 2: Inputs: Develop agro￾dealers and other input suppliers Objective 1) to increase access to high-quality seed for a range of species in demand in local, regional and global markets; Objective 3) to ensure widespread availability and efficient use of sustainably sourced, reasonably-priced, high-quality feed suitable for a range of species Activity 3: Training: Improved farm management Objective 1) to increase access to high-quality seed for a range of species in demand in local, regional and global markets; Objective 3) to ensure widespread availability and efficient use of sustainably sourced, reasonably-priced, high-quality feed suitable for a range of species; 33 Ibid 33 Objective 4) to improve efficiency, profitability and sustainability of aquaculture production through increased knowledge, skill and organization Activity 4: Training: Demonstration plots Objective 1) to increase access to high-quality seed for a range of species in demand in local, regional and global markets Activity 5: Training: Sanitary and phytosanitary standards (SPS) Objective 6) to increase the quality and value of production Activity 6: Capacity Building Objective 4) to Improve efficiency, profitability and sustainability of aquaculture production through increased knowledge, skill and organization Activity 7: Capacity Building: Producer groups/cooperatives No specific objective, but the National Plan does note the importance of working with farmer/trader networks, groups, and associations.34 Activity 8: Training: Improved marketing and branding No specific objective. Activity 9: Market Access: Facilitate a traceability system for domestic aquaculture No specific objective. Activity 10: Financial Services: Provide SME finance, technical assistance loans and grants Objective 7) to facilitate access to credit as appropriate to the needs, potential and risks associated with aquaculture development 34 Plan of Operation and Activities, p. 13. 34 A.2 How relevant is the project to the stated and immediate needs of stakeholders and beneficiaries? The Cambodian stakeholders and beneficiaries of CAST interviewed reported four related immediate needs: increased technical knowledge, financing, government protection from imports, and a change in the culture around aquaculture. According to these actors, the CAST program is, for the most part, relevant to these needs but could do more to meet them. First, many stakeholders interviewed agreed that a lack of technical skills and knowledge related to fish farming and aquaculture presented a severe problem for the industry. In particular, one CAST beneficiary interviewed explained that: “Technical skill in producing standard aquaculture products … is the priority need for the Cambodian aquaculture industry as we need to produce standard aquaculture products to sell to the market and compete with our import products, especially Vietnamese products.” (CAST beneficiary, Phnom Penh) Box 1: National Strategic Plan for Aquaculture Development in Cambodia (2016-2030) The Cambodian Government established the National Strategic Plan for Aquaculture Development in Cambodia (2016-2030) as a guide for improving the management of the aquaculture sector in Cambodia. The strategic plan considers four essential characteristics of the Cambodian aquaculture sector that includes (i) the existence of broadly favorable conditions for aquaculture development; (ii) being able to access strong national and regional markets; (iii) the existence of neighboring countries (Thailand and Vietnam) that are highly developed and efficient producers of several commodities; and (iv) the fact that much of the current aquaculture practice is highly dependent on the use of low value fish as feed inputs. The strategic plan outlines a vision and strategy for the sustainable development of the Cambodian aquaculture sector, contributing to poverty reduction, food security and GDP growth. It serves as a prospectus and framework for coordinated action by the Cambodian government, private sector actors, non-governmental organizations (NGOs) and development partners. The specific aim of the strategic plan is: “To contribute to the sustainable development of fisheries resources in Cambodia with the aim of promoting food security and socioeconomic development in the country, and more specifically to support the development of a commercially viable and environmentally sustainable private-sector and/or small-holders based aquaculture sector in the country.” The over-arching policy framework guiding development in Cambodia, and the fishery sector in particular, including the aquaculture sector, is defined by a number of inter-linked and integrated policy structures. These include the Millennium Development Goals (MDGs), the National Strategic Development Plan (NSDP) and the Strategic Planning Framework for Fisheries (SPF). The strategic plan is composed of seven strategic objectives including (i) the availability of high quality and diverse brood stock; (ii) the capacity to maximize profits from significantly seasonal markets; (iii) the conservation of resources and feed; (iv) financial and advisory to support new and existing businesses; (v) planning and management; (vi) disease status and quality of imports; and (vii) strategy implementation. These objectives and associated activities are developed to achieve a vision for the future of the Cambodian aquaculture sector the essential elements of which are sustainability, diversity and value. Source: Fishery Administration (FiA), Royal Government of Cambodia Ministry of Agriculture, Forestry and Fisheries: MAFF. (2017). National strategic plan for aquaculture development in Cambodia (2016-2030): Phnom Penh, Cambodia. 35 The problem could also be technical, and one financial institution interviewed added the following. “Even though farmers have some experience, it is still limited because they have not used the right technique so far. Therefore, to improve yields and better food safety, institutions should support them regarding farm techniques.” (Financial institution, Phnom Penh). Aquaculture farmers also need to understand intensive fish farming practices and approaches, such as calculating efficiencies, keeping records, and retaining workers, all of which requires training. Second, when aquaculture actors have the technical know-how to produce good results, one financial institution interviewed explained that a lack of finance and capital limits their ability to compete. “When they already have the technical know-how, they will also need finance. Nowadays, farmers, traders, food processors, and producers are all facing problems as they cannot access finance.” (Financial institution, Phnom Penh) Another problem frequently mentioned concerning financing is the high cost of fish feed. A government organization interviewed explained with the following comment: “The main big problems are related to fish feed because its cost is high, as production is expensive and the profit is low compared to neighboring countries.” (Government organization, Phnom Penh) Third, stakeholders and beneficiaries reported that imports from neighboring countries presented problems for local producers. According to these actors, lower input costs (such as water, electricity, and transportation) in Vietnam permit their farmers to produce and export fish and feed at a lower price than what Cambodian producers can achieve. The UN estimated that in 2015 Cambodia had a $34 million trade deficit in fish and shellfish products and the deficit for imported seed and feed was predominantly from Thailand and Vietnam.35 Several stakeholders suggested that the Cambodian government should restrict imports from neighboring countries to protect the domestic market. One CAST staff explained the difficulty that domestic producers face: “The challenge of importing fish from neighboring countries is that when they have over￾supply of fish and other meats, they will sell to Cambodia even at a loss. During the COVID￾19 pandemic, the demand for local supply decreases, so imports are higher with a lower price. There is demand from the consumer, as well as the market, but protection from the government is still limited. When Vietnam oversupplies, they export into our country, and local producers lose out.” (ASA/CAST staff, Phnom Penh) Several stakeholders interviewed expressed similar concerns as exemplified by the following comments: “I can say that there are many illegal and legal tons of fish imported from our neighboring countries every day without any intervention from our governments' side. If we want to improve our aquaculture industry quickly, we need to make sure that a law or policy needs to be implemented 100% against all fish importing from our neighboring countries.” (CAST beneficiary, Phnom Penh) 35 CAST Strategic Analysis, p. 4, fn. 13. 36 “I think imports from Vietnam are our main challenge. If Vietnam has a disaster, it will affect us.” (NGO, Phnom Penh) Staff from the CAST program corroborated these difficulties with the following explanations: “This year is a bad year for the aquaculture industry in Cambodia. Lots of fish are coming from Vietnam, and local farmers in Cambodia have not been able to sell their fish.” (ASA/CAST staff, Phnom Penh) “The aquaculture market at the regional level is dominated by neighboring countries, such as Vietnam and Thailand. Local producers have been facing this challenge of market access even before the pandemic.” (ASA/CAST staff, Phnom Penh) Towards the end of 2020, Cambodian fish markets were flooded with imported and cheaper fish from Vietnam, lowering the price of Cambodian fish and decreasing demand.36 The SAR for March 2021 explains that serious underreporting of imports had facilitated the flood of cheap fish: “MAFF announced the creation of a Cambodian Aquaculture Promotion Committee, an internal working group aiming to resolve these issues at FiA, and an anti-illegal import task force. The CAA’s efforts have started a policy process that could level the playing field for Cambodian aquaculture. This process could create an opportunity for the Cambodian aquaculture value chain to increase productivity, reduce the cost of production and create an appreciation of quality, Cambodian raised fish, all goals of both the CAST Program and the CAA.”37 Fourth, CAST administrators explained that the culture around aquaculture farming needs to be changed. According to them, farmers need to be educated on aquaculture farming. Some farmers do not consider aquaculture their main occupation and do not focus on adopting the latest technologies, techniques and practices. Instead, they view their work as simply growing fish and are unwilling to invest in new practices to improve their business. One CAST administrator explained: “The farmers think they know a lot. Many farmers just throw the feed into the pond. Farmers waste feed and also kill fish. Changing the culture and the mindset of farmers is a major challenge.” (ASA/CAST staff, Phnom Penh) CAST has sought to respond to most of these needs. CAST provides training, TA, and other services to supply chain actors in their program to help build capacity. For example, CAST provided technical support to six feed mills and eight feed distributors, covering a range of topics included in good aquaculture practices (GAqP), such as carrying capacity, farm plans, etc. As shown in Figure 4, nearly all CAST beneficiaries who participated in the MTE survey received training or TA. 36 American Soybean Association & World Initiative for Soy in Human Health, Commercialization of Aquaculture for Sustainable Trade (CAST) – Cambodia: Semi-annual Report (October 1, 2020–March 31, 2021) (2021), p. 28 [hereinafter SAR April 2021]. 37 SAR April 2021, p. 49. 37 Figure 4: Received training or TA provided by CAST The MTE survey respondents reported that CAST’s capacity-building efforts (training and TA) are relevant to their needs, as highlighted in Figure 5 below. Specifically, the vast majority (between 85 and 95%) reported applying some or most of the techniques and processes they learned in training. Across the beneficiary groups, hatchery/nursery appear to have benefited the most. Figure 5: Applied learning from training or TA provided by CAST (All beneficiary types) Among those who said they could not implement the information they learned, 37% reported it was because they did not have the resources to do so, and 33% said they simply had not yet had the time to do so. CAST has worked with microfinance institutions (MFIs) such as AMK and Chamroeun to facilitate access to financial loans for project beneficiaries in response to the lack of financing. According to the MTE survey, 32% of respondents had received loan services from CAST.38 One MFI explained that CAST enabled MFI to invest in those actors who have the capacity and willingness to scale up and expand input supply and production. CAST’s interventions have had limits to their relevance for some Cambodian actors. While many beneficiaries hoped the government and CAST would do more to restrict the imports of fish from neighboring countries, the focus of CAST is on developing the domestic supply chain and not preventing open trade. A USDA representative stated the following: 38 The total number of participants is 313 and the total number of persons who received a loan is 101 (101/313=32%). 0 62 31 8 None Some Most All Hatchery/nursery (%) Base: n=13 11 65 24 0 None Some Most All Producer (%) Base: n=191 15 77 8 0 None Some Most All Wholesaler/collector (%) Base: n=26 12 77 12 0 None Some Most All Processor (%) Base: n=17 5 81 14 0 None Some Most All Feed mill (%) Base: n=37 10 70 20 0 None Some Most All Feed distributor (%) Base: n=10 38 “There is a lot of competition for bringing in fish from neighboring countries, and CAST wants to build the domestic industry. As such, CAST needs to be focusing on where there is a gap for developing the domestic industry. CAST also needs to gather the attention of the government to be more supportive rather than passive . . . in developing the industry.” (USDA, Washington D.C.) In light of this domestic industry focus, access to the market was an issue raised by several stakeholders interviewed. The area for improvement was well captured when a CAST beneficiary recommended the following: “My suggestion for CAST in the coming years is to put more effort into the market side. If we have a good and standard fish for supply to the customers but do not have a wider market, it will cause our investors/raisers a big loss. As a result, there will not be more investors/raisers in our aquaculture industry anymore.” (CAST beneficiary, Phnom Penh) B. Effectiveness B.1 To what extent is the program producing worthwhile results (outputs, outcomes) and meeting each of its objectives? With respect to CAST producing worthwhile results, the results presented in section 3.2, project progress on standard and custom indicators, can be regarded as a complement to this section. According to CAST stakeholders, the project produces worthwhile results for project beneficiaries and meets its two main objectives: 1. Increase agricultural productivity in the freshwater aquaculture industry through improved inputs and practices; and 2. Increase Cambodian aquaculture trade by volume and value.39 However, challenges remain. The CAST plan of operations and activities lists the creation of “a Cambodian national aquaculture association” as a key step in building local capacity and information sharing across the industry, including “producers, focal farmers, input suppliers, aggregators/transporters, wholesalers, processors, and exporters.” In interviews, CAST administrators cited the creation of the Cambodian Aquaculturist Association (CAA) as a notable result of the program. The CAA has almost 300 members, and most of them are farmers beneficiaries of the CAST. According to an administrator for CAST, 150 beneficiaries signed up for the CAA in its first year. In concert with the CAA, CAST also creates other benefits for its beneficiaries. For example, CAST has helped to raise the profile and importance of the aquaculture industry. One CAST beneficiary interviewed had this to say: 39 CAST ToR, p. 3. 39 “The CAA has made the aquaculture sector get recognition from government institutions. This recognition made the investors have more voice and get attention from relevant stakeholders to their challenges and issues.” (CAST beneficiary, Phnom Penh) In addition, a Cambodian government official also agreed that the CAA contributed to raising the profile of Cambodian aquaculture. “The other one that I see as the most important is the establishment of aquaculture associations, which CAST created and organized with the Ministry of Agriculture, Forestry and Fisheries, and the Ministry of Interior. This increases the size of our voice and makes it easier to negotiate trade and sales of aquaculture products in a networked way, so it makes them stronger and easier for them to help each other in this industry.” (Government organization, Phnom Penh) Second, CAST training and TA have covered a wide range of subjects, reached hundreds of participants, improved staff retention among value chain actors, and enabled an increased adoption rate of aquaculture technologies and practices. The training and TA have taught project beneficiaries how to improve their workflow, sourcing farm input supplies, improving production outputs, and manage operations better to save costs. Table 16 below shows the proportion of supply chain actors who have received training or TA across various relevant topics. Overall, the support provided appears to be geared towards the needs of key actors in the supply chain. Hatchery/nursery, producers and feed mills have received more technical input to improve know-how and production skills. Producers may need more support on operational management, transport/handling and quality standards. For wholesaler/collector and feed distributors, the focus has been on sales, marketing, financial management, and bookkeeping. Some topics highlighted in the table were not intended topics by design. However, beneficiaries mentioned these because CAST has an open training policy in place, and anyone can attend. In addition, the results of this study have indicated that participants have shared what they have learned with others. Table 16: Topics covered in training and during TA Training or support topic Base: all participants Hatchery/ nursery n=13 (%) Producer n=208 (%) Wholesaler/ collector n=28 (%) Processor n=17 (%) Feed mill n=37 (%) Feed distributor n=10 (%) Entrepreneurship 77 64 58 88 49 70 Sales and marketing 69 68 89 88 49 100 Financial management 69 59 77 88 38 100 40 Bookkeeping 75 65 65 82 43 90 Business plan creation 77 62 73 88 51 100 Operations management 77 59 65 88 38 80 Fish stocking 92 85 15 24 78 20 How to store fish feed 77 85 12 24 87 50 Using feed table 85 87 4 18 84 40 Feed inspection 77 87 8 24 95 50 Feed conversion ratio 92 78 4 6 95 40 Carrying capacity 85 75 12 24 92 30 Fish transport/handling 77 59 31 29 41 10 Water management 92 85 12 18 84 10 Aeration 85 69 12 18 84 10 Farm management 85 76 12 24 73 20 Farm enterprise planning 77 71 8 24 60 10 Water quality 92 88 12 24 95 10 Fingerling quality 85 82 8 24 87 10 Farm record keeping 77 74 19 24 60 10 Quality standards 62 42 35 88 27 - Most stakeholders interviewed agreed the training provided by CAST has been beneficial. Explaining the subject matter of the training, one CAST partner said the following: “Training is provided directly to the farmers. In particular, on things like water quality and feed management, which are the main issues for growing fish. This helps to improve production as well as economic cost, which means getting them to improve and increase profit.” (CAST partner, the USA) There are substantial benefits for the beneficiaries who attend the training and receive TA. According to a CAST administrator, business operation costs account for around 80% of the revenue for Cambodian aquaculture farmers. However, CAST beneficiaries have learned to reduce their costs and thereby increase overall profits. Indeed, most of the MTE survey respondents reported doing their work more effectively due to CAST (Figure 6). Specifically, 100% of the feed mills and feed distributors surveyed stated that was the case, followed by 92% of hatcheries/nurseries, 82% of wholesalers/collectors, 78% of producers, and 71% of processors. 41 Figure 6: Can do work more effectively as a result of CAST support According to data provided by CAST, it provided training to 398 businesses, all full beneficiaries, from the inception of CAST in 2018 to December 30, 2020. Yet, from early 2020, COVID-19 forced CAST to reduce the number of training it offered. However, whilst the training looks to be high quality, CAST has not trained as many beneficiaries as planned.40 Instead, they started offering one-on-one technical transfers to direct beneficiaries, which led to difficulties in data collection.41 The report explains: “Due to this process of reduced beneficiaries per training, as well as an effort to catch up with training target numbers, efforts to survey beneficiaries’ retention of these techniques has lagged. The project refocused on monitoring knowledge retention and adoption during the first quarter of FY21.”42 According to a CAST administrator, “fingerling, hatcheries and feeds are improving.” In particular, the first Cambodian feed producer was introduced to the market last year.43 Additionally, CAST is linked to FiA regarding standards and GAqP. CAST has developed a curriculum, manual, and lesson plans for the extension of commercial aquaculture, which contains 17 topics in both Khmer and English.44 “One of the achievements is that the project is linked to the FiA on the questions of standards and good aquaculture practices. The project is in alignment with the curriculum. The project is trying to teach farmers what we are pushing and what aquaculture scientists recommend. We have curriculum, we have lesson plans, and these are encouraging.” (ASA/CAST staff, Phnom Penh) The research park (which is a part of CE SAIN) has verified and adopted a feeding table for the Cambodian context, a major development according to CAST staff. One of them stated: “One of the biggest benefits of CAST is the research park, which is part of CE SAIN activity. They have adopted a feeding table for the Cambodian context. A feeding table tells you how much you need to feed, etc. To me, feed is one of the most expensive things in production but crucial for making the farm profitable.” (ASA/CAST staff, St. Louis) CAST aims to “enroll and provide training and technical assistance to 900 aquaculture producers working at 600 businesses across seven Cambodian provinces” for the five-year program.45 However, 40 American Soybean Association & World Initiative for Soy in Human Health, Commercialization of Aquaculture for Sustainable Trade (CAST) – Cambodia: Semi-annual Report (April 1, 2020–September 31, 2020) (2020), p. 21 [hereinafter SAR October 2020]. 41 SAR October 2020, p. 21. 42 SAR October 2020, p. 21. 43 Key informant interview, April 5 2021. 44 Key informant interview, March 25, 2021. 45 American Soybean Association & World Initiative for Soy in Human Health, Midterm Evaluation (Mte) For CAST: Request For Proposals, p. 2. 42 CAST needs to do more to reach this goal in the remaining time while responding to the challenges presented by the COVID-19 pandemic. Finally, CAST administrators and stakeholders agreed that the benefits being created by CAST align with the overall objectives of CAST and the aquaculture industry in Cambodia. Several different KII respondents agreed with this view: “The benefits generated by the project are aligned with the goals and objectives of the aquaculture industry in Cambodia. CAST right now is dealing with the market system (not only production). The work of CAST aligns with the goals and objectives of FiA’s that has ten years’ milestone (improve production, product quality, etc.).” (ASA/CAST staff, Phnom Penh) “Benefits generated are relevant and align with the goals and objectives of the aquaculture industry in Cambodia quite well.” (CAST stakeholder, Phnom Penh) “I would say that CAST has been working with all relevant stakeholders in the aquaculture industry. To sum up, I can say that CAST’s work is aligned with the objectives of the aquaculture industry in Cambodia.” (CAST beneficiary, Phnom Penh) B.2 What, if any, unintended consequences to the aquaculture sector can be attributed to the project? CAST has had both positive and negative unintended consequences. First, the positive consequences include increasing the number of people engaged in the aquaculture business and the number of hatcheries. Furthermore, aquaculture production has also increased during the CAST program because more aquaculture farmers have developed the right technologies. Furthermore, in the last few years, several large firms have upgraded their facilities. Second, manufactured feed (e.g., commercial feed) has become more common in Cambodia due in part to an increase in imported feed. This unintended consequence is partially positive since commercial feed is more efficient, reducing farmers’ expenses and the cost of production and labor. However, the CAST and CAA team “now recognize that feed imports to Cambodia are necessary for the market and industry to grow.”46 46 SAR April 2021, p. 19. 43 The project has also had some unintended negative consequences. First, according to a CAST administrator, CAA has discussed banning imports of fish from neighboring countries with the Cambodian government to protect the Cambodian market from competition. As noted above, many of the stakeholders interviewed commented in favor of increased protection of the Cambodian market from imports. However, CAST does not support such measures and favors free trade as a general principle as a USDA program.47 Additionally, it was argued that the Cambodian aquaculture industry is growing at a rate that may be too rapid for the capacity of producers and other supply chain actors.48 One of the CAST stakeholders explained: “The downside of the development in aquaculture is that it’s growing too fast, contrasting with the capacity of the producers in the sector. They are doing too many things at the same time while the foundation is still fragile.” (CAST stakeholder, Phnom Penh) 47 Key informant interview, April 6, 2021. 48 Key informant interview, April 6, 2021. Box 2: Changes in technology during the COVID-19 pandemic The economic and social disruption caused by the COVID-19 pandemic has been devastating with significant stress on the aquaculture sector, one of the most important for Cambodian livelihoods. During this difficult time, Cambodian actors, including those in the government, have increased their focus on issues facing the aquaculture sector. According to key informants, the combination of COVID-19 and the CAST program brought about heightened attention to training and technology adoption for the aquaculture industry: “COVID-19 has been a silver lining to get people together, including the government, to pay attention to issues faced by the aquaculture industry. COVID-19 seriously hampered food security, and the government realized that aquaculture is very important for food security and we had a structure and people willing to talk to the government and push them to take actions.” (ASA/CAST staff, Phnom Penh) In addition, the pandemic has also sped up the adoption of technologies. It has brought about changes in the way CAST implements its project. “In terms of the benefits provided by COVID-19, CAST technical staff spent three days (five hours/day) with technical and sales staff of feed companies on Zoom instead of being in the field to conducting one-off trainings. Indirect beneficiaries including university staff and Cambodian government officials also attended the training, Participants have had access to Khmer language PPT with Western technical experts with Cambodian project staff who help with translation and sequential interpretation and Q&A. Even our Western expert said the Q&As are extremely valuable, not only for communicating the technologies but to find out what the local reality is.” (ASA/CAST staff, Phnom Penh). “CAST is one of the biggest projects that is thought out very carefully, step-by-step. It shared advanced techniques and helped the local business. The challenges we are facing concern training methods. Due to COVID-19, people cannot travel here to receive the trainings so we have had to do remote training. Now, we have a good system for remote training in which if they have any questions, they can ask immediately.” (CAST partner, Can Tho) 44 Another CAST partner also agreed, adding that: “Aquaculture in Cambodia is growing rapidly but in a disorganized way. Many things have to be improved in every part of the value chain such as regulations about production and quality standards.” (CAST partner, Phnom Penh) The importance of ensuring that growth is sustainable was reiterated by another CAST stakeholder: “I do not see it now, but it may appear in the future. If aquaculture becomes a profit￾making industry, it will impact the environment. Everyone can dig anywhere to make ponds, so it is important to convince the Cambodian government to have a proper aquaculture policy (e.g., where people can dig ponds, handle wastewater). The government should have tools and instruments like a legal policy. These will be needed in the near future. It is time for the Cambodian government to think about it before it goes beyond its control.” (CAST stakeholder, Phnom Penh) Finally, COVID-19 interrupted the expected flow of the program, preventing international experts from travelling to Cambodia due to travel restrictions. However, the funds earmarked for travel could not be used to hire more local technical staff. One CAST administrator explained as follows: “Technical experts from Auburn University (AU) and Can Tho University (CTU) could not travel to Cambodia due to COVID-19, and we could not add local resources. Most of the money for the technical experts from outside was already contracted before COVID-19 hit the project.” (ASA/CAST staff, Phnom Penh) Box 3: The negative unintended consequence of COVID-19 The COVID-19 pandemic has severely impacted the Cambodian fishery sector, especially on the socio￾economic conditions of the stakeholders, (e.g., fishers, fish farmers, traders, as well as consumers.) The closure of food services along the supply chain puts immediate stress on them, not only on production but also in marketing and trade. Currently, the Cambodian Aquaculturist Association (CAA), an organization promoting a sustainable Cambodian aquaculture industry, has attempted to handle the difficult situation in the aquaculture market during the pandemic. The CAA called on relevant authorities and institutions to enforce compliance with fish-import regulations and facilitate transport for distributors to ensure adequate market supply of fish to reverse a recent price jump. The lockdown has kept aquaculture products from arriving at distribution warehouses in the capital, leading to delays and shortages of aquaculture products in retail market prices. The pandemic reveals Cambodia’s excessive reliance on imported fish and fisheries products. Accordingly, the CAA attempts to lessen the impact of imported fish by promoting an appropriate quota, fair competition and anti-dumping measures. The CAA has discussed the import ban with the government based on WTO laws and principles with the aim of protecting local fishes and fisheries products as well as aquaculture farmers. Such a move from the CAA has created tension among the CAA, the Cambodian government and the United States Department of Agriculture (USDA). As one of the ASA/CAST staff explained: “The CAA decided to discuss an import ban with the government. There is some political conflict with the government of Cambodia. The USDA does not have any policy to promote protectionism, and they promote the exact opposite. Their policy is more on free trade.” (ASA/CAST staff, St. Louis) Source: Thou Vireak. 2021. ‘Aquaculture group urges authorities to enforce import rules, ease transport,’ The Phnom Penh Post, April 20, 2021 < https://www.phnompenhpost.com/business/aquaculture-group-urges-authorities-enforce-import-rules-ease-transport￾issues> 45 C. Efficiency C.1 To what extent is the relationship between inputs and outputs timely, cost-effective and to the expected standards? Overall, it was generally agreed that COVID-19 has hampered the efficiency of the program and caused delays. One CAST staff reported that the program had not been implemented in a timely manner, saying: “Project components 1-4 are under the supervision of ASA/CAST experts . . . The experts could not travel to Cambodia to provide technical assistance. Therefore, with the resources and capacity of the local team, they could only provide limited support to the aquaculture manufacturing and industry.” (ASA/CAST staff, Phnom Penh) Indeed, the SAR for October 2020 reported that “COVID-19 contributed to several delays at CE SAIN’s technology parks and the pandemic impacted the implementation of some sub-activities. It was difficult to deliver training during the current reporting period, and some farmer training was delayed until the fourth quarter of FY20, thereby reducing the number of farmers trained.”49 Delays in training farmers lead to delays for farmers to implement the training in practice. Also, some training has continued but in a different format. “Since the outbreak of COVID-19, we have been trying to conduct training online, also recording some training on video that can be disseminated so that our technical team can follow up by phone. Online is not as good as hands-on, in-person training, but overall, it has covered six different topics. Everyone in the world needs time to process and adapt to new information. Online training is helpful, and follow-up will be more helpful, but it’s not really possible to do by phone.” (ASA/CAST staff, Phnom Penh) However, as highlighted in section 3.2, despite the challenges presented by the pandemic, technical training and technical support has been provided and contributed to some farmers expanding their production facilities. C.2 To what extent has the project been efficiently implemented and managed (including financial review and audit information), as appropriate? CAST’s project set-up and management have been working well. The project’s M&E staff works across all project areas. The M&E team is responsible for field data collection and technology and technique retention assessments. The project also has separate technical, associations, communications, operations, and finance and marketing teams. The coordination and communications flow through ASA headquarters in the USA ensure consistency and alignment. CAST has engaged in adaptative project management and implementation in the Cambodian aquaculture industry to effectively and efficiently implement the project. As an example of this, one CAST administrator explained: 49 SAR October 2020, p. 17. 46 “At the start of the CAST, we could not do large-scale training, so we had to focus entirely on short training duration, fewer topics and fewer farmers in the chair at a time. That was the only way to get beneficiaries in, by using different implementation tactics.” (ASA/CAST staff, Phnom Penh) Staffing has presented some problems for efficient project implementation. Overall, CAST is understaffed. In particular, the technical team has a wide range of responsibilities but remains relatively small, due in part to COVID-19. In contrast, both the marketing and associations teams are rather large and responsible for additional activities.50 One CAST administrator expressed frustration at this dynamic: “I feel that the technical team is a bit small, given the COVID-19 context. CAST was hoping to get support from AU and CTU. Yet, in the end, most of the work related to TA had to be done by our small Cambodian team. We have had to ask three to four people to do everything. It has been challenging for our local team here.” (ASA/CAST staff, Phnom Penh) While training is received well, limited resources have prevented them from reaching the target number of beneficiaries. “Regarding the trainings we have done, our staff is skilful and capable and has provided quality training. The team is supposed to do two to three training in a year but have not done so because the team is relatively small.” (ASA/CAST staff, Phnom Penh) Furthermore, there has been less participation from Cambodian aquaculture farmers in CAST activities in recent times. The price of fish has also gone down in 2021, and aquaculture farmers face challenges in selling their fish. This was confirmed by the MTE survey, where over 70% of wholesalers and collectors reported price decreases of 39% up to 46% (Figure 7). The results are also consistent with previous research reports on aquaculture.51 Figure 7: Change in aquaculture sales price In terms of CAST’s budget, while line items are fine and CAST has been able to maintain the budget, in part due to COVID-19, there has been a serious revenue shortfall since the beginning of the project. According to a memorandum on CAST’s Plan of Operations, the decrease in soybean price has led to 50 Marketing team is in charge of an additional 5 activities (Activities #5, #6, #8, #9 and #10) and holds one third of the total CAST budget. The association team is in charge of one activity (Activity 7). 51 Regional Solutions, Commcercialization of aquaculture for sustainable trade (CAST) Cambodia: value chain analysis, January 2020. Aveage increase 23 % Aveage decrease 39 % Had price decrease 72% Had price increase 17% Change in sales price: Wholesaler/collector Base: all, n=18 Aveage increase 20 % Aveage decrease 46 % Had price decrease 71% Had price increase 7% Change in sales price: Processor Base: all, n=14 47 a nearly 2 million USD budget gap for the project, which USDA has not replaced as of January 2021.52 This has forced CAST to reduce costs wherever possible to ensure liquidity.53 One CAST administrator gave this explanation: “The first few years of monitorization did not produce what we proposed because the global price of soybeans fell between 2018 to 2019. We lost 25-30% of the projected revenue.” (ASA/CAST staff, Phnom Penh) While constraints on training presented the largest challenge to project implementation, another challenge was establishing clear links between some of CAST’s activities and key project indicators. Many stakeholders indicated that it is too early to see a conversion from program activities to actual changes in some of the standard and custom indicators set for the program. One of the program partners suggested that CAST pay more attention to qualitative data and not solely relying on quantitative indicators that may be too rigid. “I suggest that CAST should pay more attention to the qualitative aspects of the project implementation in terms of data collection, which sometimes does not include the value￾added aspects of the project in the indicator. The project itself has many indicators, and the data collection system itself and how they are implementing are very rigid.” (CAST partner, Kansas) In general, however, representation by the beneficiaries, stakeholders and partners in the implementation of CAST appears to be appropriate. According to CAST administrators, the funding utilization process has been transparent and has followed the CAST financial policy and manual. D. Impact D.1 Has the project contributed to short-and medium-term socio-economic improvements for aquaculture producers, hatcheries and nurseries, and feed mills? CAST administrators, beneficiaries, and partners were unsure of the project's short-term and medium￾term socio-economic impact. One CAST administrator summarized the problem by assessing the impact of the project at this stage: “I think it is too early to say because the market is not normal because of COVID-19, which significantly delayed the new year celebration and closed the hospitality sector for a year. This has impacted consumer demand. After the economy recovers and the situation is back to normal, we may be able to compare results to baseline data.” (ASA/CAST staff, Phnom Penh) CAST has provided skills in understanding production costs, financial planning, return on investment, budgeting and sufficiency, among other topics for participants in the project. The MTE survey showed that many beneficiaries have seen it worthwhile to share learnings with other farmers, as demonstrated in Figure 8. Over four in five beneficiaries from hatchery/nurseries, feed mills, and feed distributors said they had shared what they had learned with others. These results indicate that some benefits of the program may have reached beyond the immediate participants. 52 Comments on the CAST Plan of Operations, Jan. 31, 2021, p. 1. (Note: an amendment was signed on 4/22/21 (sale expected in November 2022), which is expected to compensate) 53 Comments on the CAST Plan of Operations, Jan. 31, 2021, p. 1. 48 Figure 8: Have shared learnings from CAST with other farmers (All beneficiary types) While the program's economic impact remains challenging to measure, CAST moved forward with increasing access to finance for CAST beneficiaries despite difficulties posed by COVID-19. According to the SAR for March 2021, the pandemic slowed CAST's ability to support financing programs with a local MFI partner. Despite these difficulties, the Cambodian government opened a $50 million line of credit targeting rice, livestock, and fish via the Agriculture & Rural Development Bank (ARDB).54 All along the value chain, several CAST beneficiaries received loans from the ARDB, partly because of training and information they received from CAST.55 That loans were received was also confirmed by the MTE survey. As shown in section 3.2, several producers also expanded their production facilities as a direct result of participating in the CAST. CAST also established a relationship with Chamroeun Microfinance Plc (CMP), officially selected as the financial partner for loan disbursements and serves as CAST’s financial partner and provides a financial literacy training program to CAST beneficiaries for free.56 A technical working group (TWG) was established to oversee the disbursement of loans.57 As of October 2020, eight individuals received loans as a result of USDA assistance.58 Additionally, in the last two months of the most recent reporting period, CMP “disbursed $162,000 to seven SMEs for their business improvement. This represents 72% of the project’s annual target for FY21. Through technical assistance from CAST, another 12 SMEs can now access loans from other formal MFIs.”59 One CAST partner explained the significance of this relationship: “The most positive development [regarding loans] was the collaboration with Chamroeun Microfinance Plc. to initiate the first aquaculture financial services program, which provides loan, credit and microfinance services to the aquaculture industry.” (CAST partner, Phnom Penh) In interviews, CAST stakeholders reported positively about the increased financing. One financial institution interviewed provided details of the impacts of these loans: “We just started to work with project beneficiaries to disburse the loan. What we see is that the conditions we are offering are possible through a partnership with the different stakeholders of CAST. It allows the farmers to start upscaling their operations, which allows them to find positive income.” (Financial institution, Phnom Penh) 54 SAR October 2020, p. 29. 55 SAR October 2020, p. 28. 56 SAR October 2020, p. 24; SAR April 2021, p. 46. 57 SAR April 2021, p. 46. 58 SAR October 2020, Appendix 3, Table 10. 59 SAR April 2021, p. 47. 49 Aside from loans, CAST has had an impact in other realms. As shown in Figure 9 below, since joining CAST, one hatchery/nursery that took part in the MTE survey reported adding a total of five new ponds for a total increase of 3,200 square meters. Another 22 producers added a total of 46 new ponds with an estimated 97,320 square meters.60 Save for one producer, all of them attributed the increase to being part of the CAST program. Figure 9: Added new ponds since joining CAST In addition, since joining CAST, two hatcheries/nurseries that took part in the MTE survey added new cages totalling 2,516 cubic meters. Another two producers added 36 square meters of cages (Figure 10). All of them attributed the increase to being part of the CAST program. Chi-Square analysis61 was done to determine how CAST may have contributed to this expansion of production facilities, including ponds and cages. Several significant results were found when examining training and technical support provided by CAST. Those who added new ponds or cages had to a greater extent, received training and support on public events, publications, sales and marketing, FCR, and farm enterprise planning. They were also found to be more knowledgeable about record keeping, water quality and how to transport fingerlings. The producers also stated they increased the number of ponds and cages due to their involvement with CAST, which suggests the training and technical support provided aided in their decision to expand. Figure 10: Added new cages since joining CAST (Producer, Hatchery/Nursery) 60 One hatchery/nursery and eight producers had developed more than one new pond. Due to the limitation of using a phone survey, the size of the largest and smallest ponds were collected. The few farmers that had more than two ponds or cages were contacted separately to confirm the total size of each pond or cage. 61 Piersons Chi-Square at the 0.05 significance level. 50 The KII supported these findings, and a financial institution interviewed stated the following: “Capacity building conducted as part of the project has allowed farmers to upscale their vision, measure business plans and to take risks in a way that they can do so more confidently.” (Financial institution, Phnom Penh) Among fish producers surveyed, just over one in four producers (27%) reported a decrease in their production, averaging a 39% decrease (Figure 11). However, another quarter of producers (23%) stated they saw an increase that was, on average, 38%. The reasons provided for the decrease were varied: The most significant reason was fish mortality (40%); 15% cited price falls, 13% had lost customers, and 13% said it was due to COVID-19. These results are in line with reports from the KII, where several stakeholders mentioned the decrease in price and demand and the problems posed by drought and COVID-19 as challenges in the industry. To better understand why some producers saw an increase in production, a Chi-Square analysis62 was done. No significant relationships were found when cross-tabulating results with cast training, better farming techniques or having received technical support. The only significant variable was the geographical location, and producers in Phnom Penh were more likely to increase production, probably because of closer proximity to the market. Figure 11: Change in fish production (Producer) The situation was worse for hatchery/nursery producers, with 40% reporting an average decrease of nearly 50% but 20% reporting a production increase of 30% (Figure 12). Drought was a major factor for the decrease combined with a loss of customers and difficulty competing with imports and COVID￾19. 62 Piersons Chi-Square at the 0.05 significance level. Reason for production decrease: Producer Base: had production decrease, n=53 Aveage increase 38 % Aveage decrease 39 % Production decrease 27% Production increase 23% Cannot compete with imports 8% Lost customers 13% Drought 6% High fish mortality 40% Difficult to source fingerlings 6% Price fall 15% Other (Covid-19) 13% Change in fish production: Producer Base: all, n=199 51 Figure 12: Change in fish production (Hatchery/Nursery) On average, the hatcheries/nurseries produce fingerlings for a four-month period each year in which they have seven production cycles on average. However, the breeding of fingerlings and fish production should ideally overlap in the same period. To examine this overlap, Figure 13 shows the last production cycle reported by hatchery/nursery and producers. The production peaks don’t overlap and suggests that hatchery/nursery and producers may not be in sync. When cross￾referencing production month with production quantity for producers, there was an indication that sales slumped between October and December 2020. Several factors could explain this, but assuming cooperation within the program, the delayed production of fingerlings in 2021 compared to fish production may need to be investigated further. Figure 13: Last production cycle (Producer, Hatchery/Nursery) D.2 What difference has the project made in the lives of beneficiaries? Most stakeholders interviewed stated it was too early to say what difference the project has made in the lives of the beneficiaries. According to a CAST administrator, the Cambodian aquaculture industry is facilitated by family members who work on the farm. Therefore, it is quite difficult to calculate changes in employment rates or wages. It was further argued that some estimates indicate that the employment rate is slowly improving but data to support this notion is incomplete. Reason for production decrease: Hatchery/nursery Base: had production decrease, n=4 Aveage increase 30 % Aveage decrease 48 % Production decrease 40% Production increase 20% Cannot compete with imports 25% Lost customers 25% Drought 50% Change in fish production: Hatchery/nursery Base: all, n=10 *Note: Eight farmers did not produce fish in 2020 due to Covid-19 20 30 20 10 20 10 18 8 6 2 4 3 4 5 11 15 33 0 5 10 15 20 25 30 35 40 Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Hatchery/nursery Producer 2020 2021 Month of latest production cycle (%) Base: all hatchery/nursery, producer, n=10 / n=191* Survey period 52 Several related questions on the subject were asked in the MTE survey to gain an insight into how beneficiaries themselves see their situation. The results are shown in Table 17 below. Overall, beneficiaries involved with CAST agree that the program has made a difference. For example, 89% to 100% of participants agreed that CAST teaches useful skills, while 80% to 100% thought involvement with CAST would help to improve their future living standards. The majority (80–93%) also said the wellbeing of their family had improved. They also felt the program has contributed to improving the quality of aquaculture in Cambodia (92–100%). These results cannot be taken as representative of the entire aquaculture industry in Cambodia but reflect a shared sentiment that the outcomes of the CAST program are seen to make a difference to those involved. Table 17: Change in beneficiaries’ lives as a result of CAST (all participants) Agree that CAST … Base: all participants Hatchery /nursery n=13 (%) Producer n=208 (%) Wholesaler /collector n=28 (%) Processor n=17 (%) Feed mill n=37 (%) Feed distributor n=10 (%) Teach useful skills 100 96 89 94 100 100 Provide timely support 69 80 75 77 81 100 Motivate to recommend program to others 100 86 79 82 97 70 Improve future living standards 92 90 89 88 97 80 Improve business performance 92 92 89 88 95 80 Improve wellbeing of family 92 93 89 88 81 80 Improve quality of aquaculture in Cambodia 92 95 96 100 100 100 E. Sustainability E.1 To what extent has the project fostered ownership by local and regional partners (e.g., CE SAIN, CTU)? Many of CAST’s partners, stakeholders and beneficiaries feel they play an integral part in the project thanks to the cooperative manner in which ASA implements it. Also, CAA contributes to sustained, locally-led processes and capacity following the completion of the project. For example, CAA is being organized and operationalized with nine board of directors and 300 members, most of whom are CAST’s farmer-beneficiaries. CAA takes all the models, technical knowledge and know-how from CAST and will continue to implement activities in the future on behalf of the Cambodian aquaculture industry. As one administrator explained: 53 “CAA will carry on the work of the CAST as CAST has built the capacity and spread technical knowledge to the farmers, which will ensure continuity.” (ASA/CAST staff, Phnom Penh) Other CAST partners also agreed with this assertion, with one of them saying: “I would say that when CAST ends, the whole sector will stand on their own as we have an association and teamwork, which is working with one another very well.” (CAST beneficiary, Phnom Penh) However, for CAA to be sustainable, it must have strong local ownership and commitment. Indeed, the SAR October report notes that “A strong industry association is critical to the long-term health of the aquaculture industry and the sustainability of the project's future successes.”63 However, with only the financial support of members, CAA may not be sustainable in the long term without an independent, long-term funding source, as one of the CAST stakeholders said. “CAST should prepare well-thought-out strategies for CAA so that it can stand on its own after the completion of the project.” (CAST stakeholder, Phnom Penh) CAST beneficiaries held a similar viewpoint, as captured by the following point made by one of them: “We also see that CAA does not have any signs that it could be alive on its capacity without depending on the donation from any party. They are still uncertain. If CAST could help the association make its profit, I believe it could keep going. If it is like nowadays—where it survives with some support from the other members in this sector—it will not last.” (CAST beneficiary, Phnom Penh) E.2 To what extent has the project established/enhanced capacity, processes and local systems that are likely to be sustained? Whilst on a smaller scale than initially planned, CAST has enhanced the capacity, processes, and local systems so that they are likely to be sustained. For example, CAST has trained producers and local feed mill and import feed companies’ staffs to have the knowledge and skills to work with aquaculture farmers in the future and train new aquaculture farmers after completing the project without the assistance of CAST. CAST has also provided business development support (BDS) training to post￾harvest actors (i.e. wholesalers/collectors) on entrepreneurship, sales and marketing; financial management; bookkeeping; and business plan creation. In addition, CAST has trained local feed distributors and connected them with producers. One CAST administrator explained: “Farmers are trained on good aquaculture practices (carrying capacity, farm plans, etc.) by CAST, and they can transfer the knowledge to other farmers in the future.” (ASA/CAST staff, Phnom Penh) Due to COVID-19 travel restrictions, CAST has explored the creation of video conferencing (live) for training trainers and the creation of video recordings of targeted, short extension style training videos, particularly for farmers.64 It is expected that sharing such videos, alongside more traditional curricula and manuals, will be playing a part in making the program more sustainable going forward: 63 SAR October 2020, p. 31. 64 SAR April 2021, p. 21. 54 “Project materials, videos and manuals CAST has produced will be available in the longer term for farmers to learn. They will be handy and will also contribute to the sustainability of the project results.” (USDA, Washington D.C.) In general, CAST administrators and beneficiaries, stakeholders, and partners were optimistic that the processes put in place by CAST would continue. For example, one partner explained: “The knowledge sharing and capacity building that CAST provided and the networks and relationships that they established are going to support their work and funding. The relationship with the ministry and RUA will help to sustain CAST in the future.” (CAST partner, Phnom Penh) Another university professor agreed, saying that: “CAST has provided a lot of technical support to relevant stakeholders, so I would say that this project is moving forward if CAST is gone. Anyway, we need to ensure that our network/association has a stable connection so that everything will be moving forward.” (CAST partner, Phnom Penh) Additionally, CAST via WVI engages with MFI to provide financial products to actors in the supply chain. As a result, the KII confirmed that the MFI could offer the capital necessary for the long-term development of the supply chain. One financial institution reiterated this point, adding that their MFI’s support would continue after CAST had concluded: “From our side, the way the project is run is long-term sustainable. The project is trying to set up a system that will be there after the project concludes. As an MFI, one of the reasons we are selected is that we intend to continue to be involved in aquaculture financing even after the project.” (Financial institution, Phnom Penh) 55 Box 5: Feeding table for Cambodia One of CAST’s successes is the idea of the feeding table, initiated by the Royal University of Agriculture’s Center of Excellence on Sustainable Agricultural Intensification and Nutrition (CE SAIN) in Cambodia, one of CAST’s partners. In Cambodia, farmers often overfeed or underfeed. To assist farmers in addressing this problem, the feeding table is designed to improve feed management for farmers. The table informs the farmer of the correct amount of feeding, which helps farmers to save money and protect water from being polluted. As a consequence, farmers can make more profit and engage in sustainable aquaculture farming. This kind of table has been used in the US, P.R. China, and Viet Nam, which have more sophisticated and effective operations of aquaculture farming. “One of the biggest benefits generated by CAST derives from the research park which is part of CE SAIN. The research park has a verified/adopted feeding table for the Cambodian context. The feeding table tells you how much you need to feed, etc. All of that depends on the conditions in the pond. To me, feed is one of the most expensive things in production, more than selling the final fish itself. Making it right is crucial for making the farm profitable.” (ASA/CAST staff, St. Louis) A feeding table should be used only as a guide. Using a feeding table requires technical knowledge on aquaculture farming as well as accurate information about the number of fish they have and their size. Farmers also have to pay close attention to the amount of feed the fish consumes, the fish’s protein intake, the weight of the fish, and the temperature of the water. In this sense, well-trained feeders are more important than a feed table. However, it is undeniable that a feed table is an important instrument in assisting well-trained aquaculture farmers to increase their profits and engage in more sustainable aquaculture farming. Source: USDA, ASA, et al. (n.d.). How Much Should I Feed My Fish?. 56 1. Conclusions The following section includes concluding remarks regarding the major contributing and inhibiting factors to the success of the CAST program. 1) Major contributing factors In terms of factors that contribute to the project's success, one factor is the interest shown by the project beneficiaries. Whilst not representative of the industry, beneficiaries involved rate CAST highly and agreed that the program teaches useful skills in producing fish and selling them in Cambodia and has helped improve their future living standards. The majority also said the well-being of their family had improved. They also felt the program has contributed to improving the quality of aquaculture in Cambodia. The CAA establishment has also positively affected CAST beneficiaries and stakeholders, unifying and permitting them to share information, strategize, and engage in targeted advocacy. While CAST may not always support the advocacy aims of CAA, the fact that its members, which already number over 300, are crafting and engaging in advocacy is a positive sign that there is an interest and willingness to develop the aquaculture sector in Cambodia. CAST has developed manuals for the extension of commercial aquaculture, which contains 17 topics in both Khmer and English.65 In addition, CAST has also developed a 5-topic curriculum on business development (i.e. entrepreneurship, sales and marketing, financial management, bookkeeping and business plan creation) that is being delivered to its beneficiaries. Additionally, CAST/ASA developed and shared contact lists of hatchery farms with aquaculture producers. Furthermore, the number of people engaged in the aquaculture industry and the number of hatcheries has increased. This increase is possible because more aquaculture farmers have the appropriate technology to effectively and efficiently engage in farming. In addition, in the last few years, several large firms have upgraded their systems. Finally, manufactured feed (e.g., commercial feed) has become more common in Cambodia. This unintended consequence is positive since commercial feed is more efficient, reducing farmers’ expenses and the cost of production and labor. 2) Major inhibiting factors In terms of factors that inhibit the project's success, one key factor is that there is not a strong enough preference for locally produced fish in Cambodia due to higher retail prices. Many KII respondents confirmed that since the beginning of 2020 and through 2021, imports of fish from Vietnam had been high and local producers are forced to sell their fish at a lower price. Despite CAST’s success in increasing the capacity of producers, the market for locally produced fish has not effectively mitigated the fish imports due to relatively higher transportation and operation costs. COVID-19 has disrupted CAST’s implementation. It caused “an almost total halt to domestic travel, meetings and training activities for beneficiaries.” Due to travel restrictions, it prevented consultants and foreign-based experts from engaging in several technical activities, especially those managed by Auburn University (AU).66 65 Key informant interview, March 25, 2021. 66 SAR October 2020, p. 31. 57 Another inhibiting factor was the difficulty CAST had in staffing the program. CAST struggled to find a qualified Deputy Chief of Party, technical or local staff with the requisite private sector experience.67 Indeed, this shortfall of human resources for the technical team, in concert with a halt in short-term technical assistance travel, “impeded the project’s ability to make anticipated progress towards satisfying indicator targets for Activity #2.”68 This was rectified by CAST adding a third aquaculture specialist focusing on Activity 2 and the input dealers. Finally, CAST struggled to identify businesses with enough scale to pull the demand at the distribution level in the main cities in Cambodia. The absence of fish exporting companies also is a limiting factor in CAST’s efforts to increase the demand for local produce. To overcome this challenge, CAST will “get in contact with a high number of actors with limited purchase capacity and with diverse quality and volume requirements.”69 2. Recommendations for CAST Based on the MTE results, the following recommendations are given for course corrections and improvement in the project implementation for the remaining project duration (assuming COVID-19 does not cause further disruptions). Communication and engagement  Aquaculture supply chain actors need to engage more to share information through provincial and regional workshops. CAST, including market systems and the CAA, facilitates business dialogue between distributors, processors, and farmers, but they are not enough. There should be a higher degree of connection and engagement among CAST beneficiaries (different supply chain actors). Increased engagement will help farmers have better relationships and more accessible feed inputs and market information for fish producers. Implementation could start once the pandemic is under control.  Due to the pandemic situation, CAST has had to make several adjustments to its implementation plan. In the short term, CAST should review the progress made to date in light of COVID-19 and query whether it should update activities and target groups to make the program more relevant. Supply chain capacity development  Aquaculture farmers need increased access to low-interest rate loans to compete with imports from neighboring countries. CAST has established the first loan product for aquaculture in Cambodia, with preferential interest rates and non-collateralized up to 40K. CAST should review how it can further capitalize on this partnership model.  Production of local fingerlings needs to be enhanced further. Fingerlings in Cambodia are relatively expensive compared to neighboring countries, Vietnam in particular. Locally produced fingerlings could potentially offer better quality and could be a strategy to compete more effectively in the long term.  The handling and transportation of fingerlings need to be improved. Farmers lose significant quantities of fingerlings between leaving the hatcheries and entering the pond, as confirmed 67 American Soybean Association & World Initiative for Soy in Human Health, Commercialization of Aquaculture for Sustainable Trade (CAST) – Cambodia: Semi-annual Report (October 1, 2019–March 31, 2020) (2020), p. 40 [hereinafter SAR April 2020.] 68 SAR April 2020, p. 23. 69 SAR October 2020, p. 31. 58 by the MTE survey and the KII. Multiple factors could cause the higher mortality rate, and which specific factors to focus on could not be determined by the MTE and may warrant further investigation. Market development  CAST has managed to stimulate some farmers to increase the production facilities. But preferences for imported fish prevent many aquaculture farmers from realizing their potential gains. CAST should review its market activities—including market information, commercial visits with distributors, business dialogues among buyers and producers—and determine how these activities can be strengthened or scaled up.  While this study focused on the supply side, there is an evident need to continue promoting and raising awareness of local aquaculture fish to Cambodian consumers and influencing consumer preferences. Market studies have been undertaken to inform market campaigns. These efforts need to be reviewed and evaluated to determine campaign impact and guide future marketing activities.  In light of COVID-19 and the reduction in the price of fish, CAST should conduct a market or consumer demand study after the pandemic is under control. It can serve as an additional study, like the Consumer Preference Survey (CPS), which CAST did in 2019, and can examine market or consumer demand in the post-pandemic era.  Regarding encouraging fingerling production, since about 80% is imported from Vietnam, CAST should provide training for fingerling production that cooperates with farmers and other relevant partners to produce fingerlings that can be sold to farmers at a competitive price while still offering good quality. There is some evidence suggesting hatcheries/nurseries and producers are not in sync, both in production cycles and type of species produced, at least within the CAST beneficiary base. Enhancing understanding  In addition to current training and technical assistance, CAST could facilitate international study and learning tours to appropriate countries for selected Cambodian aquaculture producers who can serve as influencers for other producers. The learning tours would enable producers to appreciate better new technologies and techniques (COVID-19 restrictions permitting). CAST should also promote and support tours to local facilities, such as the CE SAIN’s technology park, in the same vein.  CAST should continue to expand local extension workers who can support project beneficiaries by providing regular coaching and technical assistance. Further expansion is necessary since many project partners are overseas and have not been able to come to Cambodia to provide in-person support. Sustainability  The CAA needs to be financially self-sufficient. The CAA is currently independent but supported financially by CAST. CAA must have strong local ownership and receive sustainable, long-term funding. In this regard, CAST should review its exit strategy in line with the CAA's goal to stand on its own after the project’s completion. 59 Annexes Annex 1: Data quality assessment (DQA) 1.1 Introduction A data quality assessment (DQA) was carried out as part of the MTE. The purpose of the DQA is to help donors, CAST staff and implementing partners understand the strengths and weaknesses of the data collection processes that influence management decision-making and are integral to the project. The DQA was done by reviewing CAST’s M&E related documents, including the data collection manual (DCM), the data analysis plan (DAP), the performance monitoring plan (PMP) and the indicator performance tracking table (IPTT). In addition, interviews were carried out with four of CAST’s M&E staff, who are responsible for collecting, processing, and reporting on project data. The DQA guidelines, including the five data quality standards, were adopted to determine the extent to which actual data collection practice is in line with document data collection procedures. The data quality standards are summarized below. As part of the DQA, the impact of COVID-19 on data collection processes was also considered. Validity: The data collected should clearly and accurately represent the intended result with no measurement errors. Validity was assessed by looking at how the data was collected and if any problems had been experienced. Reliability: The data should reflect stable and consistent data collection processes and analysis methods over time. Reliability was assessed by looking at standard and custom indicators and determining how easy or difficult it is to track results over time. Timeliness: Data for indicators should be available at a sound frequency, current, and timely enough to aid management decision-making. Timeliness was assessed by looking at data collection and reporting procedures and how regularly the data were collected and reported. Precision: Data should be accurate and have a sufficient level of detail to permit management decision-making. Precision was assessed by looking at the tools used to collect data and obtaining feedback from project staff regarding measures taken to ensure data accuracy. Integrity: Collected data should have safeguards to minimize the risk of transcription errors or manipulation. Integrity was assessed by looking at data collection instruments and how data are stored, shared and transferred. The focus of the DQA was around six standard data collection processes adopted for the project, including: 1. New beneficiary enrollment 2. Training/workshop attendance 3. Technology adoption 4. Sale records 5. Financial service records 60 6. Technical assistance records Each of the six data collection processes was assessed against the five data quality standards. A moderator guide was developed around the DQA’s five data quality standards (see Annex 6). Following the review of relevant M&E documents and the four staff interviews, a three-point scale assessment (none/limited, partly, mostly/fully) was adopted to score each of the five data quality standards. The scoring was done across the six data collection processes, with results presented in a spider chart. Each score represents the average of two scores. The first score was derived from examining the documentation and standard operating procedures (SOPs) in place (process component). The second score was based on assessing implementation with CAST’s M&E staff (people component). Scores are summarized as follows:  None or limited: Data quality standard component is not implemented properly (i.e., as intended) because of issues with the process itself, or CAST’s M&E staff are not familiar with the procedures (score=1).  Partly: The data quality standard component is being implemented but needs some improvement. CAST’s M&E staff do not appear to have full familiarity with the procedures, or procedural adjustments are needed (score=2).  Fully or mostly: Data quality standard component meets requirements, and CAST’s M&E staff are familiar with the procedures and implement them in good order (score=3). 1.2 New beneficiary enrollment Short screening of potential beneficiaries is carried out at first contact with them and can be in person or over the phone. The New Beneficiary Enrollment Form is used to determine if a potential beneficiary meets the eligibility criteria for the program. If the respondent is eligible, a member of the project’s M&E team contacts them to complete a Full Enrollment Form. The worsening of the COVID￾19 situation in Cambodia led to travel restrictions and meant that many short screeners and Full Enrollment Forms had been administered over the phone. All data are collected via ODK Collect, and hard copies are used as a backup if/when necessary. Later on, collected data is submitted to the ODK server. All data sets are stored on a central MIS system, managed by the project’s MIS Specialist. Standard form: B01: Short Screen Enrollment Form/B02: The New Beneficiary Enrollment Form/baseline form (off-line and hard copy) Frequency of data collection and reporting: Routine (data for Form B02 are updated every 12 months) Indicators against which data are recorded: N/A 61 Figure 1: New beneficiary enrollment performance across five data quality standards 1.2.1 Meet data quality requirements Validity: Data quality standard component meets requirements, and CAST’s M&E staff are familiar with the procedures of new beneficiary enrollment and can implement them in good order. Sound methods are used to collect new beneficiary data. The collected data measure what they are supposed to measure and fall within a plausible range. There is assurance that the data collection methods used do not produce systematically biased data. Reliability: Data quality standard component meets requirements, and CAST’s M&E staff are familiar with the procedures of new beneficiary enrollment and can implement them in good order. Collected data of new beneficiaries generates complete and consistent records each time. Also, data collection methods are documented and standardized to ensure consistency over time, except phone data collection due to COVID-19. 1.2.2 Opportunities for improvement Timeliness: Data quality standard component is being implemented, but certain procedural adjustments are needed. Data are regularly collected in order to inform project management decision￾making. CAST collects and reports the most practically available data, and data are reported as soon as they are collected. However, CAST sometimes lags behind target rates regarding new enrollment of beneficiaries because of three factors: timing, time availability and tools. When the data collection team (comprising the technical team, CAA team and FMOs) enrolls new beneficiaries in the field, the technical team and CAA team are also required to allocate time for other on-going activities at the same time (e.g., training, workshops, technical assistance). There is often insufficient time for enrolling new beneficiaries, which has occasionally slowed down the data collection process against the target rate. Also, for full enrollment of new beneficiaries, the B02 Form (i.e., Full Enrollment Form) and baseline form (for those who are not Baseline Study respondents) are used. Generally, it takes up to 30 minutes to complete the B02 form, whereas the baseline form takes over one hour. When in the field, data collection involves the baseline form, it requires a significant amount of time and often annoys 3.0 3.0 2.0 2.5 2.5 Validity Reliability Precision Timeliness Integrity New beneficiary enrollment 62 beneficiaries, making them less motivated to work with CAST. Such annoyance has also resulted in an occasional slowdown in the data collection process. In terms of data submission timing, some of CAST’s M&E staff do not submit data in accordance with the DCM’s guidelines, which state that data should be submitted within 72 hours of collection. Instead, these staff members submit data on the Thursday of the week they collected the data. Precision: Data quality standard component is being implemented but needs some improvement. The data collection methods and tools adopted for the program are fine-tuned to collect and record accurate information. However, while certain of CAST’s M&E staff are familiar with the procedures for ensuring that information is accurately recorded in the form, other staff are unaware. Also, some of CAST’s M&E staff are not aware of the procedures and safeguards adopted by the project to ensure that information is accurately recorded. Integrity: Data quality standard component is being implemented but needs some improvement. CAST adopts policy (procedures or safeguards) to minimize data transcription errors. Also, independence is ensured in key data collection management and assessment procedures. Finally, CAST adopts mechanisms to prevent unauthorized changes to data. However, while certain CAST’s M&E staff are familiar with the procedures adopted by CAST to minimize data transcription errors, other staff are not aware of such procedures. 1.2.3 Recommendations Timeliness: It is recommended that in the field data collection process should be streamlined and made more user friendly for beneficiaries who register with CAST. The streamlined process should be standardized in the next version of the DCM. This would also allow sufficient time to enroll new beneficiaries and obtain more complete data from all beneficiaries in the field. Regarding data submission timing, it is recommended that CAST should review the 72-hour time period stated in the DCM and agree and practice on a common standard in accordance with the actual situation on the ground. Precision: It is recommended that CAST’s M&E staff enhance their knowledge and understanding of the procedures and safeguards adopted by the project to ensure that information is accurately recorded by regularly organizing training/workshops for M&E staff. Integrity: It is recommended that CAST’s M&E staff enhance their knowledge and understanding of the policies (procedures or safeguards) adopted by the project to minimize data entry and transcription errors through regularly organizing training/workshops for M&E staff. 1.3 Training/workshop attendance Attendance of participants at CAST sponsored training and workshops is recorded. If new beneficiaries attend, they also need to be enrolled in the system. All data are collected via ODK Collect, and hard copies are used as a backup if/when necessary. Later on, collected data is submitted to the ODK server. All data sets are stored on a central MIS system, managed by the project’s MIS Specialist. A retention assessment is also carried out with 60% of attendees. 63 Standard form: B03: Training/Workshop Attendance Records Form (off-line and hard copy) Frequency of data collection and reporting: Routine Indicators against which data are recorded: SIs 21 and 22 and CIs 1 and 9 Figure 2: Performance of training/workshop attendance data across the five data quality standards 1.3.1 Meet data quality requirements Validity: The data quality standard component meets requirements, and CAST’s M&E staff are familiar with the procedures of collecting training/workshop attendance data and can implement them in good order. Sound methods are used to collect training/workshop attendance data. The collected data measure what they are supposed to measure and fall within a plausible range. There is assurance that the data collection methods used do not produce systematically biased data. Timeliness: The data quality standard component meets requirements, and CAST’s M&E staff are familiar with the procedures of collecting training/workshop attendance data and can implement them in good order. Data are collected regularly to inform project management decision-making. CAST collects and reports the most practically available data, and data are reported as soon as they are collected. Precision: The data quality standard component meets requirements, and CAST’s M&E staff are familiar with the procedures of collecting training/workshop attendance data and can implement them in good order. Data collection methods/tools are fine-tuned to collect and record accurate information. Integrity: Data quality standard component meets requirements, and CAST’s M&E staff are familiar with the procedures of collecting training/workshop attendance data and can implement them in good order. CAST adopts policies (procedures or safeguards) to minimize data transcription errors. Independence is ensured in key data collection management and assessment procedures. In addition, CAST adopts mechanisms to prevent unauthorized changes to data. 3.0 2.5 3.0 3.0 3.0 Validity Reliability Precision Timeliness Integrity Training/workshop Attendance 64 1.3.2 Opportunities for improvement Reliability: Data quality standard component is being implemented but some procedural adjustments are needed. The collected training/workshop attendance data generates complete and consistent results each time. Also, the adopted field data collection method is documented and standardized to ensure consistency over time. However, the adopted phone data collection method due to COVID￾1970 is not documented and standardized to ensure consistency over time. Overall: According to the DCM, the collection of training/workshop attendance data are primarily the responsibility of the technical/CAA team, and the M&E team comes as secondary and tertiary. As such, while certain of CAST’s M&E staff are familiar with the data collection process, other staff are not. 1.3.3 Recommendation Reliability: For the next version of the DCM, it is recommended that the process of collecting attendance data concerning on-line training/workshops (e.g., Zoom training/workshops) should be documented and standardized to ensure consistency over time. It is noted that these on-line training/workshops processes were still being developed at the time of the DQA. Overall: It is recommended that those who are secondary and tertiary in terms of responsibility for collecting training/workshop attendance data, including M&E staff, should also be aware of data collection processes. For this, CAST should regularly organize trainings/workshops for M&E staff to ehnace their knowledge and understanding. There is an opportunity for more synergy if all CAST’s M&E staff understand the entire data collection process, including those for whom it is not their primary responsibility. 1.4 Technology adoption Data are collected six months after the beneficiary attends the training. Data are collected from an individual or enterprise who has adopted at least one technology or technique they learned from training in their farm or business operation. All data are collected via ODK Collect, and hard copies are used as a backup if/when necessary. Later on, collected data is submitted to the ODK server. All data sets are stored on a central MIS system, managed by the project’s MIS Specialist. Standard form: B04: Techniques/Technologies Adoption Tracker (off-line and hard copy) Frequency of data collection and reporting: Six months after the beneficiary attends training on aquaculture technique/technology Indicators against which data are recorded: SIs 2, 3, and 4 and CI 2. 70 For example, when trainings/workshops are conducted via Zoom, each participant is required to enter their personal profile data (e.g., name, company, position etc) before they are allowed to on-line sessions. 65 Figure 3: Performance of technology adoption data across the five data quality standards 1.4.1 Meet data quality requirements Reliability: Data quality standard component meets requirements, and CAST’s M&E staff are familiar with the procedures of collecting technology adoption data and can implement them in good order. The collected technology adoption data generates complete and consistent records each time. Also, data collection methods being adopted are documented and standardized to ensure consistency over time. Precision: Data quality standard component meets requirements, and CAST’s M&E staff are familiar with the procedures of collecting technology adoption data and can implement them in good order. The adopted data collection methods/tools are fine-tuned to collect and record accurate information. Integrity: Data quality standard component meets requirements, and CAST’s M&E staff are familiar with the procedures of collecting technology adoption data and can implement them in good order. CAST adopts policies (procedures or safeguards) to minimize data transcription errors. Independence is ensured in key data collection management and assessment procedures. In addition, CAST adopts mechanisms to prevent unauthorized changes to data. 1.4.2 Opportunities for improvement Validity: Data quality standard component is being implemented but needs some improvement. The collected data measure what they are supposed to measure and fall within a plausible range. There is assurance that the data collection methods used do not produce systematically biased data. Sound methods are used to collect technology adoption data. However, while certain of CAST’s M&E staff are familiar with collecting technology adoption data procedures, others are not. Regarding the timing of data collection, CAST’s M&E staff have varied understandings and practices. According to the DCM, data are collected six months after a beneficiary attends aquaculture technique/technology training. However, some of CAST’s M&E staff collect data twice (six months and 12 months after the beneficiary attends a training). 2.5 3.0 2.0 3.0 3.0 Validity Reliability Precision Timeliness Integrity Technology Adoption 66 Timeliness: Data quality standard component is being implemented, but some procedural adjustments are needed. CAST collects and reports the most practically available data, and data are reported as soon as they are collected. However, regarding data collection frequency, some CAST’s M&E staff consider six months following training to be too long and suggest the frequency to be changed to three months, six months and 12 months following training. The reason for this change is because six months is long enough for some beneficiaries to forget what they learned in training. In terms of data submission timing, some of CAST’s M&E staff do not perform data submission in accordance with what is instructed in the DCM, which states that data should be submitted within 72 hours after collection. Some CAST’s M&E staff submit data on the Thursday of the week they collect data. 1.4.3 Recommendation Validity: Regarding the timing of data collection, it is recommended that CAST’s M&E staff have a common understanding and practice according to what is instructed in the DCM. If necessary, the DCM can be updated to be in line with the actual situation on the ground. Timeliness: Regarding the frequency of data collection, it is recommended that CAST’s M&E team review the DCM and update it to align with the actual situation on the ground. Regarding data submission timing, it is recommended that CAST should review the 72-hour time period stated in the DCM and agree and practice on a common standard in accordance with the actual situation on the ground. 1.5 Sales records Data collection is dependent on the producer keeping proper sales records. If proper records do not exist, the data are collected based on recall using the sale record form or Annual Results Survey. The data are used to calculate the annual value (in Riel, converted to USD) and volume (in kilograms, converted to metric tons) of incremental sales of commodities resulting from USDA investment. All data are collected via ODK Collect, and hard copies are used as a backup if/when necessary. Later on, collected data is submitted to the ODK server. All data sets are stored on a central MIS system, managed by the project’s MIS Specialist. Standard form: B05: Sales Records Form (off-line and hard copy) Frequency of data collection and reporting: Quarterly (every three months) Indicators against which data are recorded: SIs 1, 18, and 19, and CIs 4, 8, and 10 67 Figure 4: Performance of sales records data across the five data quality standards 1.5.1 Meet data quality requirements Reliability: The data quality standard component meets requirements, and CAST’s M&E staff are familiar with the procedures of collecting sales records data and can implement them in good order. The collected sales record data generates complete and consistent records each time. Also, data collection methods being adopted are documented and standardized to ensure consistency over time, except phone data collection due to COVID-19. Integrity: The data quality standard component meets requirements, and CAST’s M&E staff are familiar with the procedures of collecting sales record data and can implement them in good order. CAST adopts policies (procedures or safeguards) to minimize data transcription errors. Independence is ensured in key data collection management and assessment procedures. In addition, CAST has adopted mechanisms to prevent unauthorized changes to data. 1.5.2 Opportunities for improvement Validity: Data quality standard component is being implemented but needs some improvement. The collected data mostly measure what they are supposed to measure and fall within a plausible range. There is assurance that the data collection methods used do not produce systematically biased data. A sound research method is used to collect sales record data. However, the level of understanding of the data collection process among CAST’s M&E staff varies, and the understanding and practices of some staff regarding collecting sales records data are not in accordance with the DCM. Some staff understand that data is collected from only those beneficiaries who attend trainings organized by CAST, while others understand that data is collected from every beneficiary enrolled in CAST. Furthermore, the DCM requires staff to collect data every three months. Yet some staff understand that data is collected every six months, while others understand it as monthly. 2.5 3.0 2.0 2.5 3.0 Validity Reliability Precision Timeliness Integrity Sale records 68 Timeliness: Data quality standard component is being implemented, but some procedural adjustments are needed. The most practically available data are collected and reported, and data are reported as soon as they are collected. However, regarding data collection frequency, some staff understand that data are collected bi-annually (every six months) or monthly, which is not in accordance with what is instructed in the DCM (quarterly). Also, some suggest frequency to be too much while others suggest too little depending on the types of beneficiaries. Some suggest that data collection frequency should vary depending on the types of beneficiaries as each beneficiary type has different harvest and sales cycles in a year. In terms of data submission timing, some of CAST’s M&E staff understand and perform data submission in a manner not in accordance with what is instructed in the DCM, which states that data should be submitted within 72 hours after collection. Some CAST’s M&E staff submit data on the Thursday of the week they collect data. Precision: Data quality standard component is being implemented, but some procedural adjustments are needed. Data collection methods/tools adopted are partially fine-tuned to collect and record accurate information. Some of CAST’s M&E staff argue that the data on “market information which beneficiaries received from CAST” does not present from the current sales records and suggest CAST collect this particular data by adding a section to the sales records form. 1.5.3 Recommendation Validity: Regarding the data collection process, it is recommended that CAST’s M&E staff should review the collection standard and agree on a common understanding so that it is in accordance with what is instructed in the DCM. Timeliness: Regarding the timing of data collection, it is recommended that CAST’s M&E team review the DCM and update it to align with the actual situation on the ground. Regarding data submission timing, it is recommended that CAST should review the 72-hour time period stated in the DCM and agree and practice on a common standard in accordance with the actual situation on the ground. Precision: It is recommended that CAST’s M&E team should review the sales records form and update it (if necessary) in order for CAST to collect all data necessary for project implementation. 1.6 Financial service records The data are collected from beneficiaries who have been able to access grants or loans as a result of receiving training and technical assistance (TA) organized by World Vision International (WVI) as part of CAST. All data are collected via ODK Collect, and hard copies are used as a backup if/when necessary. Later on, collected data is submitted to the ODK server. All data sets are stored on a central MIS system, managed by the project’s MIS Specialist. Standard form: B06: Financial Services Tracker (off-line and hard copy) Frequency of data collection and reporting: Quarterly (every 3 months) Indicators against which data are recorded: SIs 5, 7, 8 and 14 69 Figure 5: Performance of financial service records data across the five data quality standards 1.6.1 Meet data quality requirements Validity: Data quality standard component meets requirements. Sound methods are used to collect financial service records data. The collected data measure what they are supposed to measure and fall within a plausible range. There is assurance that the data collection methods used do not produce systematically biased data. Timeliness: Data quality standard component meets requirements. Data are collected regularly to inform project management decision-making. WVI collects and reports the most practically available data, and data are reported as soon as they are collected. Precision: Data quality standard component meets requirements. Data collection methods/tools are fine-tuned to collect and record accurate information. Integrity: Data quality standard component meets requirements. CAST adopts policies (procedures or safeguards) to minimize data transcription errors. Independence is ensured in key data collection management and assessment procedures. In addition, CAST adopts mechanisms to prevent unauthorized changes to data. 1.6.2 Opportunities for improvement Reliability: Data quality standard component is being implemented, but some procedural adjustments are needed. Financial service records data that generates complete and consistent results each time. The adopted data collection methods are partially documented and standardized to ensure consistency over time, except phone data collection due to COVID-19. Regarding the recording of loans/grants released by WVI via CAST, it appears that the grants are currently released in various installments, and the data of each instalment is currently collected and recorded. However, the DCM states that grant data should be recorded once WVI has disbursed 100% of the grant to beneficiaries. 3.0 2.5 3.0 3.0 3.0 Validity Reliability Precision Timeliness Integrity Financial service records 70 1.6.3 Recommendation Reliability: It is recommended that the current data collection process regarding grant disbursement by WVI via CAST should be documented and standardized in the next version of the DCM. 1.7 Technical assistance records Technical assistance comprises a capacity building of farming technology or techniques, such as proper farm management. The purpose of collecting technical assistance records is to inform the technical assistance provided by aquaculture technical leads and aquaculture technical specialists. Technical assistance is conducted one by one. All data are collected via ODK Collect, and hard copies are used as a backup if/when necessary. Later on, collected data is submitted to the ODK server. All data sets are stored on a central MIS system, managed by the project’s MIS Specialist. Standard form: B09: Technical Assistance Records Form (off-line and hard copy) Frequency of data collection and reporting: Routinely Indicators against which data are recorded: SIs #22 and #23 Figure 6: Performance of technical assistance data across the five data quality standards 1.7.1 Meet data quality requirements Validity: Data quality standard component meets requirements. Sound methods are used to collect technical assistance data. The collected data measure what they are supposed to measure and fall within a plausible range. There is assurance that the data collection methods used do not produce systematically biased data. Reliability: Data quality standard component meets requirements. The collected technical assistance data generates complete and consistent records each time. Also, the adopted data collection methods are documented and standardized to ensure consistency over time, except phone data collection due to COVID-19. 3.0 3.0 3.0 3.0 3.0 Validity Reliability Precision Timeliness Integrity Technical assistance records 71 Timeliness: Data quality standard component meets requirements. Data are collected regularly to inform project management decision-making. CAST collects and reports the most practically available data, and data are reported as soon as they are collected. Precision: Data quality standard component meets requirements. Data collection methods/tools adopted are fine-tuned to collect and record accurate information. Integrity: Data quality standard component meets requirements. CAST adopts policies (procedures or safeguards) to minimize data transcription errors. Independence is ensured in key data collection management and assessment procedures. In addition, CAST adopts mechanisms to prevent unauthorized changes to data. 1.7.2 Opportunities for improvement Overall: According to the DCM, the collection of technical assistance data is primarily the responsibility of a technical team, and the M&E team comes as secondary and tertiary. As such, while certain of CAST’s M&E staff are fully familiar with the data collection process, other staff appear to be unfamiliar with it. 1.7.3 Recommendation Overall: It is recommended that those who are secondary and tertiary in terms of responsibility for collecting technical assistance data, including the M&E staff, should be made aware of data collection processes. For this, CAST should organize regular trainings/workshops for its M&E staff in order to enhance their knowledge and understaniding. There is an opportunity for more synergy if all CAST’s M&E staff understand the entire data collection process, including those for whom it is not their primary responsibility. 72 Annex 2: Other results from MTE survey 2.1 Growout harvesting 2.1.1 Source of aquaculture (Wholesaler/Collector and Processor) Provinces Base: all respondents Wholesaler/ collector n=18 (%) Processor n=14 (%) Phnom Penh 22 21 Kandal 17 7 Kampong Cham 56 21 Kampong Thom 17 21 Siem Reap 28 50 Battambang 6 - Pursat 11 21 None - 7 2.1.2 Change in aquaculture fish purchase last harvest (Wholesaler/Collector and Processor) 73 2.2 Climate change 2.2.1 Mitigation against climate change (Producers and Hatchery/Nursery) 2.3 Improved practices 2.3.1 Fish production (Producers and Hatchery/Nursery) 74 2.3.2 Feed management (Producers and Hatchery/Nursery) 2.3.3 Sign of good water quality (Producer, Hatchery/Nursery, Feed Mill and Feed Distributor) Signs of good water quality Base: all respondents Hatchery/ nursery n=10 (%) Producer n=199 (%) Feed mill n=6 (%) Feed distributor n=8 (%) The fish is active 70 28 33 13 Green color water 40 15 50 - Algal bloom - 2 - - Clear water 60 76 33 38 Don’t know - 11 17 50 Average knowledge 1.7 1.2 1.2 0.5 75 2.3.4 Prevent water turbidity (Producer, Hatchery/Nursery, Feed Mill and Feed Distributor) Prevent water turbidity Base: all respondents Hatchery/ nursery n=10 (%) Producer n=199 (%) Feed mill n=6 (%) Feed distributor n=8 (%) Prevent erosion 30 2 33 13 Add water 50 51 33 13 Add manure 20 6 33 13 Don’t know 20 45 33 88 Average knowledge 1.0 0.6 1.0 0.4 2.3.5 Fingerling stocking (Producer and Hatchery/Nursery) 76 2.3.6 Fingerling stocking and transportation (Hatchery/Nursery) 2.3.7 Sales forecast (Wholesaler/Collector and Processor) 2.3.8 Type of buyer agreement (Wholesaler/Collector and Processor) 77 2.3.9 Feed management (Feed mill and Feed distributor) 2.3.10 Keep operation costs and sales records (All beneficiaries types) 2.4 Training and process (All beneficiaries) 2.4.1 Reason for not applying what was learned 78 Annex 3: Theory of change The following flowchart is CAST’s theory of change, which positions CAST’s activities in relation to the desired output and eventual goal of either increasing agricultural productivity or expanding trade of agricultural products. 79 Annex 4: Survey questionnaire Survey supply chain actors [May 2021] Version FINAL Date of Interview __ / __ / 2021 Time begin Time ended Name of Interviewer Supervisor INFORMED CONSENT Thank you for agreeing to talk with me today. I’d like to introduce myself – I am (NAME) from (ORGANISATION). This survey is being conducted on behalf of the CAST. This survey will assess how hatcheries/nurseries, producers, collectors/wholesalers, processors, feed mills and feed distributors are doing, and to identify any constraints that are limiting your performance. It would be very helpful if you could share information about your own experience. There are no right or wrong answers, so please give us your honest opinion. Any information collected from you will be kept strictly confidential. We will not use any personal information by which you could be identified in the report. Your participation in the interview is entirely voluntary and you can withdraw any time. It is possible that you will be approached for an interview in a couple of years’ time. Do you have further questions about this survey? MAKE CLARIFICATION AS NEEDED CONFIRM UNDERSTANDING Do you understand and give your consent to be interviewed for the study? IF YES CONTINUE SECTION A - PROFILE A1 ENTER CONTACT LIST ID _____ (1 – 449) A2 RECORD RESPONDENT TYPE AS PER TARGET LIST (Single) Hatchery/nursery 1 Fish producer 2 Wholesaler/Collector 3 Processor 4 Feed mill 5 Feed distributor 6 A3 RECORD RESPONDENT LOCATION (Single) Phnom Penh 1 Kandal 2 Kampong Cham 3 Kampong Thom 4 Siem Reap 5 80 Battambang 6 Pursat 7 A4 Are you Mr/Ms (NAME)? CONFIRM NAME FROM CONTACT LIST A5 Is this your main contact number? A6 COLLECT NEW NUMBER A7 How old are you? A8 RECORD SEX SECTION B - FARM ASSETS ASK IF A2=1, 2 (HATCHERY/NURSERY, PRODUCER) B1 Do you use ponds for your production? (Single) Yes, with hapa net 1 CONTINUE Yes, without hapa net 2 No 3 GO TO B4 B2 Have you built new ponds that are operational since joining the CAST? Yes 1 CONTINUE No 2 GO TO B4 Yes 1 CONTINUE No 2 ASK TO SPEAK WITH THE CONTACT PERSON OR MAKE AN APPOINTMENT TO CALL BACK Yes 1 GO TO A7 No 2 CONTINUE Years Male 1 Female 2 81 USE POND SHEET Ask the farmer how many new ponds they have added and record on the sheet. Ask the farmer to provide the length and width of the new ponds in meters. Record the dimensions on the sheet. B3a How many new ponds have you added? B3b How long is the pond in meters? B3c How wide is the pond in meters? B3d How deep is the pond in meters? B3e Did you decide to build new ponds as a result of working with CAST? Yes 1 No 2 B4 Do you use cages to raise fish? Yes 1 CONTINUE No 2 GO TO B7 B5 Have you added new cages in the past 12 months? Yes 1 CONTINUE No 2 GO TO B7 USE CAGE SHEET Ask the farmer how many new cages they have added and record on the sheet. Ask the farmer to provide the length, width, and height of the cage in meters. Record the dimensions on the sheet. B6a How many new cages have you added? 82 B6b How long is the cage in meters? B6c How wide is the cage in meters? B6d How high is the cage in meters? B6e Did you decide to add new cages as a result of working with CAST? Yes 1 No 2 ASK IF A2=1 (HATCHERY/NURSERY) B7 For how many months do you produce fingerlings in a year? B8 How many breeding cycles do you do in a year? SECTION C – GROWOUT HARVESTING ASK IF A2=1,2 (HATCHERY/NURSERY, PRODUCER) C1 During what months did you complete your last production cycle? (Multiple) July 2020 1 August 2020 2 September 2020 3 October 2020 4 November 2020 5 December 2020 6 January 2021 7 February 2021 8 March 2021 9 83 April 2021 10 C2 What fish species did you produce in the last production cycle? (Multiple) Pangasius 1 Clarias (walking catfish) 2 Tilapia 3 Channa striata 4 Annabas 5 Giant snakehead 6 Silver barb 7 Carp species 8 Other (SPECIFY) 9 C3 Did your total quantity of fish production increase, stay about the same or decrease in the last production cycle compared to the previous production cycle? (Single) Increased 1 GO TO C6 Same 2 GO TO C7 Decreased 3 CONTINUE Don’t know 4 GO TO C7 C4 What was the main reason for this decrease? (Single) Cannot compete with imported fish 1 GO TO C7 Lost customers 2 Drought 3 High fish mortality 4 Not able to source enough fingerlings or fry 5 Other (SPECIFY) 6 C5 What was the total % decrease? % Don’t know 99 C6 What was the total % increase? % Don’t know 99 84 C7 Did the average price you received per Kg of fish increase, stay about the same or decrease compared to the previous production cycle? (Single) Increased 1 GO TO C9 Same 2 GO TO C10 Decreased 3 CONTINUE Don’t know 4 GO TO C10 C8 What was the total % decrease? % Don’t know 99 C9 What was the total % increase? % Don’t know 99 ASK IF A2=3,4 (WHOLESALER/COLLECTORS, PROCESSORS) C10 From which of the following provinces did you buy aquaculture fish in 2020 and 2021? (Multiple) Phnom Penh 1 Kandal 2 Kampong Cham 3 Kampong Thom 4 Siem Reap 5 Battambang 6 Pursat 7 None 8 C11 Did you purchase more, about the same or less Cambodian aquaculture fish in the last harvest compared to the previous harvest? (Single) More 1 GO TO C14 Same 2 GO TO C15 Less 3 CONTINUE Don’t know 4 GO TO C15 C12 What is the main reason for this? (Single) 85 Cannot compete with imported fish 1 GO TO C15 Lost customers 2 Drought 3 High fish mortality 4 Not able to source enough fingerlings or fry 5 Other (SPECIFY) 6 C13 How many fewer metric tons of fish did you purchase from Cambodian aquaculture in the last harvest? Metric tons Don’t know 99 C14 How many more metric tons of fish did you purchase from Cambodian aquaculture in the last harvest? Metric tons Don’t know 99 C15 Did your total sales revenue from aquaculture fish increase, stay about the same or decrease in last harvest compared to the previous harvest? (Single) Increase 1 GO TO C17 Same 2 GO TO D1 Decrease 3 CONTINUE Don’t know 4 GO TO D1 C16 What was the total % decrease? % Don’t know 99 C17 What was the total % increase? % Don’t know 99 86 SECTION D – CLIMATE CHANGE ASK IF A2=1,2 (HATCHERY/NURSERY, PRODUCER) D1 In the last 5 years, has your fish production been affected by flooding? That is, causing pond overflow or damage that resulted in fish escaping? (Single) Yes 1 No 2 Don’t know 3 D2 Are your ponds fenced? (Single) Yes 1 No 2 Don’t use pond 3 D3 In the last 5 years, has your fish production been affected by drought? That is, causing low water levels so fish were dying, or you had to start harvesting earlier than expected? (Single) Yes 1 No 2 Don’t know 3 D4 Do you have a well to supply water? Yes 1 No 2 ASK IF A2=2 (PRODUCER) D5 Do you grow fast growing fish species that you can harvest in 3 months or less? (Single) Yes 1 No 2 Don’t know 3 D6 Do you target small harvest sizes (150 grams per fish) to shorten the production cycle? (Single) Yes 1 No 2 87 Don’t know 3 SECTION E – IMPROVED PRACTICES ASK IF A2=1,2 (HATCHERY/NURSERY, PRODUCER) E1 Do you use burnt lime or other chemicals to control predators? (Single) Never 1 Sometimes 2 Always 3 E2 When you started production this last year, did you have a target date to have the fingerlings or fish ready for market? (Single) Never 1 Sometimes 2 Always 3 E3 Do you know how much it costs to produce your fingerlings or fish? Yes 1 No 2 E4 Do you keep records of your fish production per pond or cage? Yes 1 No 2 E5 Do you use pellet fish feed? Yes 1 No 2 E6 Do you measure how much feed to give your fish every day? Yes 1 No 2 E7 Do you keep records of how much fish feed you use? 88 Yes 1 No 2 ASK IF A2=1,2,5,6 (HATCHERY/NURSERY, PRODUCER, FEED MILLS, FEED DISTRIBUTORS) E8 Have you heard about the feed conversion ratio (FCR)? Yes 1 CONTINUE No 2 GO TO E12 E9 How is the FCR calculated? (Single) Feed weight given divided by number of fish in the pond 1 Feed weight given divided by the average fish weight gain 2 Number of fish divided by average fish weight gain 3 Don’t know 4 E10 Is a low FCR or a high FCR normally better for keeping feed costs low? (Single) Low FCR 1 High FCR 2 Don’t know 3 E11 What do you need to calculate the feed conversion ratio (FCR)? (Single) Amount of fish feed used 1 Amount of fish feed used and average fish weight gain 2 Amount of fish feed used, average fish weight gain, and total number of fish 3 Don’t know 4 E12 Which of the following are signs of good water quality? (Multiple) The fish is active 1 Green color water 2 Algal bloom 3 Clear water 4 Don’t know 5 E13 What can be done to avoid too much water turbidity due to suspended clay? (Multiple) Prevent erosion 1 89 Add water 2 Add manure 3 Don’t know 4 E14 Which of the following help to determine fingerling quality? (Multiple) Big size, 5 grams or more 1 Good appearance, no sign of disease or deformity 2 Similar in size 3 Same species 4 Same age 5 None 6 ASK IF A2=2 (PRODUCERS) E15 Do you know how many fingerlings you stock per square meter? Yes 1 No 2 E16 In last year, have you changed the number of fingerlings you stock per square meter? (Single) Yes 1 No 2 Don’t know 3 ASK IF A2=1 (HATCHERY/NURSERY) E17 Do you know how many fingerlings your customer should stock per square meter? Yes 1 No 2 E18 Do you know the average survival rate of your fingerlings? Yes 1 CONTINUE No 2 GO TO E20 E19 What is your normal survival rate (%)? % 90 E20 Which of the following do you consider when stocking fingerlings? (Multiple) Fingerling species 1 Fingerling size 2 Quantity of fingerlings 3 Water temperature 4 Supplier of the fingerlings 5 Water PH level 6 None 7 E21 How do you normally transport your fingerlings to your customers? (Multiple) Hauling tanks with aeration or oxygen 1 Plastic bag with oxygen 2 Plastic bag with air 3 Car trunk or small dump trunk with plastic liner 4 Other (SPECIFY) 5 ASK IF A2=3,4 (WHOLESALER/COLLECTOR, PROCESSOR) E22 Do you know in advance how much fish raw material you need to purchase for this harvesting season? (Single) Never 1 Sometimes 2 Always 3 E23 Do you know in advance the cash return of your activity for this harvesting season? (Single) Never 1 Sometimes 2 Always 3 E24 Do you know the person or company that will purchase your product (fish or processed) before sourcing the product? (Single) Never 1 Sometimes 2 Always 3 91 E25 What type of agreement do you have with buyers (e.g., retailers, distributors)? (Single) None 1 Verbal 2 Written contract 3 ASK IF A2=5,6 (FEED DISTRIBUTOR, FEED MILLS) E26 Do you know how to calculate your customer’s feed needs? Yes 1 No 2 E27 Do you teach your customer to use their older feed first? Yes 1 No 2 E28 Which of the following topics do you provide technical support to your customers? (Multiple) How to store the feed 1 Using Feed Table 2 Feed Inspection 3 Feed Conversation Ratio (FCR) 4 Carrying capacity 5 Fish transport and handling 4 Water management 5 Aeration 6 Farm management 7 Farm enterprise planning 8 Water quality 9 None 10 ASK ALL E29 Do you keep a record of your production or operations costs? Yes 1 No 2 92 E30 Do you keep a record of your sales? Yes 1 No 2 SECTION F – TRAINING AND PROCESS ASK ALL F1 Have you received technical support or participated in the training organized by CAST in the past 12 months? Yes 1 CONTINUE No 2 GO TO F5 SHOW CARD F2 Which of the following topics did the technical support or training cover? Yes No Entrepreneurship and innovation 1 2 Sales and marketing 1 2 Financial management 1 2 Bookkeeping 1 2 Business plan creation 1 2 Business operations management 1 2 Fish or fingerling stocking 1 2 How to store fish feed 1 2 Using feed table 1 2 Feed Inspection 1 2 Feed Conversation Ratio (FCR) 1 2 Carrying capacity 1 2 Fish transport and handling 1 2 Water management 1 2 Aeration 1 2 Farm management 1 2 Farm enterprise planning 1 2 Water quality 1 2 Fingerling quality 1 2 Farm record keeping 1 2 Quality standards in fish processing 1 2 93 Any other topic (SPECIFY) 1 2 F3 To what extent were you able to apply what you learned in your daily work? (Single) None 1 CONTINUE Some 2 Most 3 GO TO F5 All 4 F4 Why have you not been able to apply what you learned? (Single) Not relevant or useful 1 Don’t know how 2 Waiting for the right time to implement it 3 Have not had time to do it 4 Don’t have the right resources 5 Other (SPECIFY) 6 F5 Which of the following services have you received from the CAST? (Multiple) Technical assistance and support 1 Public event (e.g., farmers day) 2 Meeting or workshop 3 Publication (e.g., agricultural extension materials) 4 On-line, internet or social media 5 Loan 6 Other (SPECIFY) 7 F6 As a result of the services, are you able to perform your work more efficiently? (Single) Yes 1 No 2 Don’t know 3 F7 Have you been in touch with other farmers not participating with CAST, to share what you have learned? Yes 1 No 2 94 F8 Do you agree or disagree that your involvement with the CAST will achieve the following? (Single) Agree Disagree Don’t know Help you learn useful skills 1 2 3 Provide you with timely support 1 2 3 Motivate you to recommend the program to other farmers 1 2 3 Improve your future living standards 1 2 3 Improve your business performance 1 2 3 Improve the wellbeing of your family 1 2 3 Improve the quality of aquaculture fish in Cambodia 1 2 3 CONTACT INFORMATION G1 WAS THE INTERVIEW WITH THE PERSON ON THE LIST OR A REPLACEMENT PERSON? From list 1 Replacement 2 G2 RECORD RESPONDENTS FULL NAME AND PHONE NUMBER FROM LIST IF INTERVIEW WAS DONE WITH SOMEONE NOT ON THE LIST, COLLECT FULL NAME AND PHONE NUMBER In case I have missed anything and need to contact you, may I have your name and a phone number that I can reach you on? First name Last name Phone THANK YOU FOR YOUR TIME AND COOPERATION 95 Annex 5: KII moderator guide (March 2021) FINAL Respondent full name Organization Location Mobile no E-mail: Date _____ /_______ / 2021 Time begin Time ended DECLARATION I confirm that I have checked that the interview was carried out per this moderator guide and following the guidelines and instructions provided by Rapid Asia Co., Ltd. Signed by moderator: ___________________________ FOR MODERATOR Objectives of the interview  To gain an in-depth and contextual understanding of how has CAST been implemented, particularly concerning relevance, effectiveness, efficiency, sustainability and impact (REESI).  To identify good practices, challenges, lessons learned from the implementation of CAST and obtained recommendations for the future implementation of CAST. Prepare your logistics  Charge your phone or recording device beforehand.  Silence your phone during the interview.  Bring a paper and pen so that you can note follow up questions without interrupting.  Place the recording device somewhere it can easily pick-up sound, and the mic is unobstructed. Prepare yourself for interview  Speak from your heart and be sincere.  Be clear you are not here to judge but to learn.  Convey that you respect and are interested in respondents’ experience with CAST and that you value what they might share regarding CAST. 96 Probing  Do not accept yes/no answers but probe further.  Probing should be open-ended.  Probes with “Why,” “How,” “Can you describe…,” “Tell me about…” INTRODUCTION Thank you for agreeing to talk with me today. I would like to introduce myself-I am (NAME) from (ORGANISATION). We are currently undertaking a mid-term evaluation (MTE) of Commercialization of Aquaculture for Sustainable Trade (CAST) Cambodia project implemented by American Soybean Association (ASA). The information we receive from you will be used to improve future project implementation. It will be very helpful to hear about your experiences and perspectives of the CAST. Your contribution is very valuable and there are no right or wrong answers. Please provide your honest opinion. The discussion will take place over the phone, and no one else but I will be present during this discussion. I will record our discussion and refer back to this recording when summarizing our call. I will also be taking my own notes. This interview should not take longer than 30 minutes. Your name and contact information will not be included in our report. Your participation is voluntary. You can withdraw at any time, or you can decide not to answer a question. Do you agree to be interviewed? Yes, I understand and hereby give my consent to be interviewed. Start recording. WARM-UP QUESTIONS  What is your role in the organization/company?  What is your perception of aquaculture industry in Cambodia? National policies and strategies (All)  How well are the national policies and strategies for aquaculture and feed production in Cambodia being implemented?  Which ministries and organizations are you working with to achieve this?  To what extent do you think the CAST is relevant to these policies and strategic plans? 97 Aquaculture industry’s needs and challenges (All)  What are key needs of Cambodian aquaculture industry? What are the major challenges faced by Cambodian aquaculture industry to meet those needs?  To what extent does CAST help to address immediate needs and overcoming immediate challenges of the aquaculture industry? Why? Project results (All)  What benefits has CAST generated up to now?  To what extent are benefits generated relevant and align with the goals and objectives of the aquaculture industry in Cambodia? Why?  What suggestions do you have for the CAST in the coming years? Unintended consequences (All)  What do you think are the most positive developments for the aquaculture industry in the last couple of years?  To what extent would you attribute these developments to the CAST?  What about negative developments? Project implementation (CAST teams and beneficiaries)  To what extent has CAST provided technical assistance in timely manner? Why?  To what extent has the technical assistance provided been relevant and useful?  To what extent has the technical assistance generated improvements in sourcing supply, production outputs or cost savings? Project management (CAST team, USDA)  To what extent has the management of the project been appropriate, including communication, reporting, and monitoring? Why?  To what extent has CAST been implemented according to work plan? Why?  To what extent has CAST implemented in accordance with the budget allocated? Why?  To what extent has the quality of project implementation appropriate? Why?  What are other factors and constraints (if any) affecting CAST implementation, including technical, managerial, organizational, institutional issues as well as externality unforeseen at the project design stage? 98  To what extent are the project tools/approaches appropriate? Why?  To what extent is the representation by the beneficiaries, stakeholders and partners in the implementation of CAST appropriate? Why?  To what extent is the CAST’s project strategy on knowledge management and outreach, and communication and partnership with its beneficiaries, stakeholders and partners appropriate? Why?  To what extent has the funding utilization process been transparent? Why? Socioeconomic improvement (All)  What contributions has CAST made to the business performance improvement of supply chain actors (i.e., producers, feed mills, feed distributors, hatcheries, processors, wholesalers and collectors) in Cambodian aquaculture industry?  What socioeconomical changes has Cambodian aquaculture industry witnessed (e.g., increase of wages, employment rate) over the past two years? Project ownership (All)  What is your involvement with CAST?  To what extent do you feel you have ownership of the project? Local capacity, processes and systems development (All)  To what extent does CAST have cooperated and collaborated with other related programs/projects/initiatives implemented in Cambodia?  To what extent has CAST established/enhanced capacity, processes and systems of the Cambodian aquaculture industry?  How will CAST’s results be sustained beyond the project lifetime? Why? -------------------------------------------------------- THANK YOU FOR YOUR CONTRIBUTION 99 Annex 6: DQA moderator guide DQA MODERATOR GUIDE (April 2021) Final Respondent full name Organization Location Date _____ /_______ / 2021 Time begins Time ended Objectives of the interview  To evaluate key M&E functional areas (e.g., data entry protocols, indicator definitions, reporting tools) for CAST.  To provide the CAST with a better sense of where improvements could be made (e.g., filing practices, reporting timelines, data verification). Standard data collection process to be assessed:  New beneficiary enrollment  Training/workshop attendance  Technology adoption  Sale records  Financial service records  Technical assistance records 100 Methodology  The DQA will be conducted following a desk review of CAST’s Data Collection Manual (DCM), Data Analysis Plan (DAP), Performance Monitoring Plan (PMP) and Indicator Performance Tracking Table (IPTT). Following the desk review of these documents, CAST’s M&E staff will be interviewed using the script described in Rapid Asia’s DQA Moderator Guide. o The M&E staff to be interviewed include:  Tola Heng, MEL Manager  Sovann Prey, MIS Specialist  Phalla Phan, Field Monitoring Officer (FMO)  Pinet Koh, Field Monitoring Officer (FMO)  Above-mentioned six standard data collection processes will be assessed against five data quality standards including “Validity,” “Reliability,” “Timeliness,” “Precision,” and “Integrity.”  Following the desk review and staff interviews, a three-point scale assessment (none/limited, partly, mostly/fully) will be adopted through triangulating staff responses with what are described in DCM, DAP, PMP, and IPTT to score the data collection components.  The score will be derived from examining the documentation and processes in place, assessing implementation and standard operating procedures (SOPs) with project staff. Scores can be summarized as follows: None or limited: Data quality standard component has not been implemented, not implemented properly or M&E staff appear unfamiliar with the procedures (score=1). Partly: Data quality standard component is being implemented but needs further improvement, M&E staff do not appear to have full familiarity with the procedures, or there are evident procedural gaps (score=2). Fully or mostly: Data quality standard component meet requirements and M&E staff appear familiar about the procedures (score=3).  The results of the DQA will be presented using spider charts and narrative text.  Rapid Asia’s summary of the DQA will include “conclusion”, “limitation,” and “the recommended plan for actions” is also provided. This information will be included in the final report for CAST’s Midterm Evaluation (MTE). Interview notes and any other materials used during this activity will be shared with ASA/WISHH when the final report is submitted. 101 Data collection process Interview Questions Fully or Mostly Partly None or limited Comment/suggestions New beneficiary enrollment Validity Questions 1. Please describe the process of enrolling a new beneficiary into CAST. 2. What information does the Short Screen Form provide to the project? 3. What information does the Full Enrollment Form provide to the project? 4. Please describe when and how data is collected for the Short Screen Form and Full Enrollment Form? Prompt: Is this data collected electronically or via a hard copy? 5. How do you verify that the enrollment data collected with these forms is correct? Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Does the information collected measure what it is supposed to measure? ☐ ☐ ☐ RA: Do results collected fall within a plausible range? ☐ ☐ ☐ RA: Is there reasonable assurance that the data collection methods used do not produce systematically biased data? ☐ ☐ ☐ RA: Are sound research methods being used to collect the data? ☐ ☐ ☐ Reliability Questions 1. To what extent do the new beneficiary enrollment form(s) provide the project with a reliable way to collect demographic data for CAST’s direct beneficiaries? 2. Are there times when this data is collected in a different way than what has been described during this interview? Please explain. 3. Do you feel that CAST’s Data Collection Manual (DCM) accurately describes how this data is being collected? Please explain. Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Do collected data generate complete and consistent records each time? ☐ ☐ ☐ 102 RA: Are data collection and analysis methods being employed documented and standardized to ensure the consistency over time? ☐ ☐ ☐ Timeliness Questions 1. How often is this data collected? Do you think that this data is being collected at the right time? Please explain. 2. How often is this data submitted? Do you think that this data is being collected frequently enough? Please explain. 3. What do you do to make sure that these forms are completed when needed? Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Are data collected regularly enough to inform project management decision-making? ☐ ☐ ☐ RA: Are the data collected reported the most practically available? ☐ ☐ ☐ RA: Are the data reported as soon as possible after data collection? ☐ ☐ ☐ Precision Questions 1. How often are hard copies of the Short Screen Form and Fully Enrollment Form used instead of electronic versions? 2. When entering data from a hard copy, what is done to ensure that survey responses are entered correctly? 3. What else does the project do to ensure that information is accurately recorded in this form? 4. Do you think that the values captured using this form accurately describe CAST’s direct beneficiaries? If yes, why do you think this is the case? If no, what changes need to be made? Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Is the data collection method/tool being used to collect the data fine-tuned or exact enough to collect and record accurate information? ☐ ☐ ☐ 103 Integrity Questions 1. What procedures are in place to identify and correct data entry errors? 2. What do you and/or other M&E staff do to prevent unauthorized changes this data? Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Does CAST adopt policy (procedures or safeguards) to minimize data transcription errors? ☐ ☐ ☐ RA: Is there independence in key data collection management and assessment procedures? ☐ ☐ ☐ RA: Does CAST adopt mechanism to prevent unauthorized changes to data? ☐ ☐ ☐ Training/workshop attendance Validity Questions 1. Please describe the process used to track training/workshop attendance for CAST. 2. What information does the Training/Workshop Attendance Record Form provide to the project? 3. Please describe when and how data is collected for the Training/Workshop Attendance Record? Prompt: Is this data collected electronically or via hard copy? 4. How do you verify that the attendance data collected with this form is correct? Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Does the information collected measure what it is supposed to measure? ☐ ☐ ☐ RA: Do results collected fall within a plausible range? ☐ ☐ ☐ RA: Is there reasonable assurance that the data collection methods used do not produce systematically biased data? ☐ ☐ ☐ RA: Are sound research methods being used to collect the data? ☐ ☐ ☐ Reliability Questions 1. To what extent does the Training/Workshop Attendance Record Form provide the project with a reliable way of measuring program participation for CAST’s direct beneficiaries? 2. Are there times when this data is collected in a different way than what has been described during this interview? Please explain. 3. Do you feel that CAST’s Data Collection Manual (DCM) accurately describes how this data is being collected? Please explain. 104 Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Do collected data generate complete and consistent results each time? ☐ ☐ ☐ RA: Are data collection and analysis methods being employed documented and standardized to ensure the consistency over time? ☐ ☐ ☐ Timeliness Questions 1. How often is this data collected? Do you think that this data is being collected at the right time? Please explain. 2. How often is this data submitted? Do you think that this data is being collected frequently enough? Please explain. 3. What do you do to make sure that the Training/Workshop Attendance Record Form is completed when needed? Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Are data collected regularly enough to inform project management decision-making? ☐ ☐ ☐ RA: Are the data collected reported the most practically available? ☐ ☐ ☐ RA: Are the data reported as soon as possible after data collection? ☐ ☐ ☐ Precision Questions 1. How often are hard copies of the Training/Workshop Attendance Record Form used instead of electronic versions? 2. When entering data from a hard copy, what is done to ensure that survey responses are entered correctly? 3. What else does the project do to ensure that information is accurately recorded in this form? 4. Do you think that the values captured using this form are an accurate reflection of CAST’s programming? If yes, why do you think this is the case? If no, what changes need to be made? Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Is the data collection method/tool being used to collect the data fine-tuned or exact enough to collect and record accurate information? ☐ ☐ ☐ 105 Integrity Questions 1. What procedures are in place to identify and correct data entry errors? 2. What do you and/or other M&E staff do to prevent unauthorized changes this data? Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Does CAST adopt policy (procedures or safeguards) to minimize data transcription errors? ☐ ☐ ☐ RA: Is there independence in key data collection management and assessment procedures? ☐ ☐ ☐ RA: Does CAST adopt mechanism to prevent unauthorized changes to data? ☐ ☐ ☐ Technology adoption Validity Questions 1. Please describe the process used to track the adoption of techniques, technologies, and farm management practices for CAST. 2. What information does the Techniques/Technologies Adoption Tracker provide to the project? 3. Please describe when and how data is collected for the Techniques/Technologies Adoption Tracker? Prompt: Is this data collected electronically or via hard copy? 4. How do you verify that the adoption data collected with this form is correct? Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Does the information collected measure what it is supposed to measure? ☐ ☐ ☐ RA: Do results collected fall within a plausible range? ☐ ☐ ☐ RA: Is there reasonable assurance that the data collection methods used do not produce systematically biased data? ☐ ☐ ☐ RA: Are sound research methods being used to collect the data? ☐ ☐ ☐ 106 Reliability Questions 1. To what extent does the Techniques/Technologies Adoption Tracker provide the project with a reliable way of measuring the adoption of techniques, technologies, and farm management practices by CAST’s direct beneficiaries? 2. Are there times when this data is collected in a different way than what has been described during this interview? Please explain. 3. Do you feel that CAST’s Data Collection Manual (DCM) accurately describes how this data is being collected? Please explain. Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Do collected data generate complete and consistent results each time? ☐ ☐ ☐ RA: Are data collection and analysis methods being employed documented and standardized to ensure the consistency over time? ☐ ☐ ☐ Timeliness Questions 1. How often is this data collected? Do you think that this data is being collected at the right time? Please explain. 2. How often is this data submitted? Do you think that this data is being collected frequently enough? Please explain. 3. What do you do to make sure that the Techniques/Technologies Adoption Tracker is completed when needed? Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Are data collected regularly enough to inform project management decision-making? ☐ ☐ ☐ RA: Are the data collected reported the most practically available? ☐ ☐ ☐ RA: Are the data reported as soon as possible after data collection? ☐ ☐ ☐ Precision Questions 1. How often are hard copies of the Techniques/Technologies Adoption Tracker used instead of electronic versions? 2. When entering data from a hard copy, what is done to ensure that survey responses are entered correctly? 3. What else does the project do to ensure that information is accurately recorded in this form? 4. Do you think that the values captured using this form are an accurate reflection of CAST’s programming? 107 If yes, why do you think this is the case? If no, what changes need to be made? Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Is the data collection method/tool being used to collect the data fine-tuned or exact enough to collect and record accurate information? ☐ ☐ ☐ Integrity Questions 1. What procedures are in place to identify and correct data entry errors? 2. What do you and/or other M&E staff do to prevent unauthorized changes this data? Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Does CAST adopt policy (procedures or safeguards) to minimize data transcription errors? ☐ ☐ ☐ RA: Is there independence in key data collection management and assessment procedures? ☐ ☐ ☐ RA: Does CAST adopt mechanism to prevent unauthorized changes to data? ☐ ☐ ☐ Sale records Validity Questions 1. Please describe the process used to track sales made by CAST’s direct beneficiaries. 2. What information does the Sales Records Form provide to the project? 3. Please describe when and how data is collected for the Sales Records Form? Prompt: Is this data collected electronically or via hard copy? 4. How do you verify that the sales data collected with this form is correct? Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Does the information collected measure what it is supposed to measure? ☐ ☐ ☐ RA: Do results collected fall within a plausible range? ☐ ☐ ☐ RA: Is there reasonable assurance that the data collection methods used do not produce systematically biased data? ☐ ☐ ☐ RA: Are sound research methods being used to collect the data? ☐ ☐ ☐ 108 Reliability Questions 1. To what extent does the Sales Records Form provide the project with a reliable way of measuring the value of sales made by CAST’s direct beneficiaries? 2. Are there times when this data is collected in a different way than what has been described during this interview? Please explain. 3. Do you feel that CAST’s Data Collection Manual (DCM) accurately describes how this data is being collected? Please explain. Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Do collected data generate complete and consistent results each time? ☐ ☐ ☐ RA: Are data collection and analysis methods being employed documented and standardized to ensure the consistency over time? ☐ ☐ ☐ Timeliness Questions 1. How often is this data collected? Do you think that this data is being collected at the right time? Please explain. 2. How often is this data submitted? Do you think that this data is being collected frequently enough? Please explain. 3. What do you do to make sure that the Sales Records Form is completed when needed? Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Are data collected regularly enough to inform project management decision-making? ☐ ☐ ☐ RA: Are the data collected reported the most practically available? ☐ ☐ ☐ RA: Are the data reported as soon as possible after data collection? ☐ ☐ ☐ Precision Questions 1. How often are hard copies of the Sales Records Form used instead of electronic versions? 2. When entering data from a hard copy, what is done to ensure that survey responses are entered correctly? 3. What else does the project do to ensure that information is accurately recorded in this form? 4. Do you think that the values captured using this form are an accurate reflection of CAST’s programming? If yes, why do you think this is the case? 109 If no, what changes need to be made? Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Is the data collection method/tool being used to collect the data fine-tuned or exact enough to collect and record accurate information? ☐ ☐ ☐ Integrity Questions 1. What procedures are in place to identify and correct data entry errors? 2. What do you and/or other M&E staff do to prevent unauthorized changes this data? Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Does CAST adopt policy (procedures or safeguards) to minimize data transcription errors? ☐ ☐ ☐ RA: Is there independence in key data collection management and assessment procedures? ☐ ☐ ☐ RA: Does CAST adopt mechanism to prevent unauthorized changes to data? ☐ ☐ ☐ Financial service records Validity Questions 1. Please describe the process used to collect financial services data from CAST’s direct beneficiaries. 2. What information does the Financial Services Tracker provide to the project? 3. Please describe when and how data is collected for the Financial Services Tracker? Prompt: Is this data collected electronically or via hard copy? 4. How do you verify that the sales data collected with this form is correct? Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Does the information collected measure what it is supposed to measure? ☐ ☐ ☐ RA: Do results collected fall within a plausible range? ☐ ☐ ☐ RA: Is there reasonable assurance that the data collection methods used do not produce systematically biased data? ☐ ☐ ☐ RA: Are sound research methods being used to collect the data? ☐ ☐ ☐ Reliability 110 Questions 1. To what extent does the Financial Services Tracker provide the project with a reliable way of measuring the value of sales made by CAST’s direct beneficiaries? 2. Are there times when this data is collected in a different way than what has been described during this interview? Please explain. 3. Do you feel that CAST’s Data Collection Manual (DCM) accurately describes how this data is being collected? Please explain. Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Do collected data generate complete and consistent results each time? ☐ ☐ ☐ RA: Are data collection and analysis methods being employed documented and standardized to ensure the consistency over time? ☐ ☐ ☐ Timeliness Questions 1. How often is this data collected? Do you think that this data is being collected at the right time? Please explain. 2. How often is this data submitted? Do you think that this data is being collected frequently enough? Please explain. 3. What do you do to make sure that the Financial Services Tracker is completed when needed? Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Are data collected regularly enough to inform project management decision-making? ☐ ☐ ☐ RA: Are the data collected reported the most practically available? ☐ ☐ ☐ RA: Are the data reported as soon as possible after data collection? ☐ ☐ ☐ Precision Questions 1. How often are hard copies of the Financial Services Tracker used instead of electronic versions? 2. When entering data from a hard copy, what is done to ensure that survey responses are entered correctly? 3. What else does the project do to ensure that information is accurately recorded in this form? 4. Do you think that the values captured using this form are an accurate reflection of CAST’s programming? If yes, why do you think this is the case? If no, what changes need to be made? 111 Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Is the data collection method/tool being used to collect the data fine-tuned or exact enough to collect and record accurate information? ☐ ☐ ☐ Integrity Questions 1. What procedures are in place to identify and correct data entry errors? 2. What do you and/or other M&E staff do to prevent unauthorized changes this data? Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Does CAST adopt policy (procedures or safeguards) to minimize data transcription errors? ☐ ☐ ☐ RA: Is there independence in key data collection management and assessment procedures? ☐ ☐ ☐ RA: Does CAST adopt mechanism to prevent unauthorized changes to data? ☐ ☐ ☐ Technical assistance records Validity Questions 1. Please describe the process used to track the delivery of technical assistance to CAST’s direct beneficiaries. 2. What information does the Technical Assistance Records Form provide to the project? 3. Please describe when and how data is collected for the Technical Assistance Records Form? Prompt: Is this data collected electronically or via hard copy? 4. How do you verify that the sales data collected with this form is correct? Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Does the information collected measure what it is supposed to measure? ☐ ☐ ☐ RA: Do results collected fall within a plausible range? ☐ ☐ ☐ RA: Is there reasonable assurance that the data collection methods used do not produce systematically biased data? ☐ ☐ ☐ RA: Are sound research methods being used to collect the data? ☐ ☐ ☐ 112 Reliability Questions 1. To what extent does the Technical Assistance Records Form provide the project with a reliable way of monitoring the amount and type of technical assistance provided to CAST’s direct beneficiaries? 2. Are there times when this data is collected in a different way than what has been described during this interview? Please explain. 3. Do you feel that CAST’s Data Collection Manual (DCM) accurately describes how this data is being collected? Please explain. Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Do collected data generate complete and consistent results each time? ☐ ☐ ☐ RA: Are data collection and analysis methods being employed documented and standardized to ensure the consistency over time? ☐ ☐ ☐ Timeliness Questions 1. How often is this data collected? Do you think that this data is being collected at the right time? Please explain. 2. How often is this data submitted? Do you think that this data is being collected frequently enough? Please explain. 3. What do you do to make sure that the Technical Assistance Records Form is completed when needed? Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Are data collected regularly enough to inform project management decision-making? ☐ ☐ ☐ RA: Are the data collected reported the most practically available? ☐ ☐ ☐ RA: Are the data reported as soon as possible after data collection? ☐ ☐ ☐ Precision Questions 1. How often are hard copies of the Technical Assistance Records Form used instead of electronic versions? 2. When entering data from a hard copy, what is done to ensure that survey responses are entered correctly? 3. What else does the project do to ensure that information is accurately recorded in this form? 4. Do you think that the values captured using this form are an accurate reflection of CAST’s programming? If yes, why do you think this is the case? 113 If no, what changes need to be made? Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Is the data collection method/tool being used to collect the data fine-tuned or exact enough to collect and record accurate information? ☐ ☐ ☐ Integrity Questions 1. What procedures are in place to identify and correct data entry errors? 2. What do you and/or other M&E staff do to prevent unauthorized changes this data? Probing (examples): Please tell me more about that. What do you mean when you say [blank]? RA: Does CAST adopt policy (procedures or safeguards) to minimize data transcription errors? ☐ ☐ ☐ RA: Is there independence in key data collection management and assessment procedures? ☐ ☐ ☐ RA: Does CAST adopt mechanism to prevent unauthorized changes to data? ☐ ☐ ☐ 114 Annex 7: KII moderator training agenda Attendees: KII moderators, supervisor, CAST staff and Rapid Asia staff Location: Zoom Date: 31 March 2021 Time: 10:00-13:00 (ICT) Time Topic Attendees Facilitator 10:00-10:30 Introduction of CAST Interviewers & Supervisor ASA/CAST 10:30-11:00 Overview - Background - Methodology overview - Target groups - Timeline Interviewers & Supervisor Rapid Asia 11:00-12:00 Moderator guides run through - Go through KII moderator guide  Clarifications/questions - Informed consent form Interviewers & Supervisor Rapid Asia 12:00-12:20 Ethical guideline Interviewers & Supervisor Rapid Asia 12:20-12:30 Q & A Interviewers & Supervisor Rapid Asia 12:30-13:00 Practice - Role Play (respondent, interviewer, observer) - Feedback and lessons learned Interviewers & Supervisor BMRS Asia Finish 115 Annex 8: Survey enumerator training agenda Attendees: Survey enumerators, supervisor, CAST staff and Rapid Asia staff Location: Zoom Date: Friday, 14 May 2021 Time: 10:00-12:30 Time Topic Attendees Facilitator 10:00-10:30 Introduction of CAST Interviewers & supervisor CAST 10:30-11:00 Overview of Cambodian aquaculture industry Interviewers & supervisor Rapid Asia 11:00-11:30 Overview of MTE of CAST - Background - Methodology overview - Target groups - Timeline Interviewers & supervisor Rapid Asia 11:30-12:00 Process briefing for Interviewers - Using CAPI on tablet - Calling process - Using showcard - Ethical guideline Interviewer Rapid Asia 12:00-12:30 Going thourgh survey contact list Interviewers & supervisor Rapid Asia 12:30-13:30 Questionnaire overview - Go through all questions - Navigation and checking completeness Questions and clarifications Interviewers & supervisor Rapid Asia 13:30-14:00 Process briefing for supervisor - Quality Control & Checking Questionnaires - Approving and rejecting surveys - Questions and clarifications Interviewers & supervisor Rapid Asia 14:00-14:15 Q&A Interviewers & supervisor Rapid Asia 14:15-15:00 Practice - Break into groups of 3 people for role play (respondent, interviewer, observer) - Feedback and lessons learned Interviewers & supervisor BMRS Asia Finish 116 Annex 9: Bibliography American Soybean Association & World Initiative for Soy in Human Health. (2018). Commercialization of Aquaculture for Sustainable Trade (CAST) - Cambodia: Strategic Analysis. [Draft]. American Soybean Association & World Initiative for Soy in Human Health. (2021). Commercialization of Aquaculture for Sustainable Trade (CAST) – Cambodia: Data Collection Manual [Unpublished]. American Soybean Association & World Initiative for Soy in Human Health. (2020). Commercialization of Aquaculture for Sustainable Trade (CAST) – Cambodia: Data Analysis Plan. [Unpublished]. American Soybean Association & World Initiative for Soy in Human Health. (2020). Commercialization of Aquaculture for Sustainable Trade (CAST) – Cambodia: Performance Monitoring Plan. [Unpublished]. American Soybean Association & World Initiative for Soy in Human Health. (2020). Commercialization of Aquaculture for Sustainable Trade (CAST) – Cambodia: Performance Tracking Table. [Unpublished]. American Soybean Association & World Initiative for Soy in Human Health. (2020). Commercialization of Aquaculture for Sustainable Trade (CAST) – Cambodia: Semi-annual Report (October 1, 2019–March 31, 2020). [Unpublished]. American Soybean Association & World Initiative for Soy in Human Health. (2020). Commercialization of Aquaculture for Sustainable Trade (CAST) – Cambodia: Semi-annual Report (April 1, 2020– September 31, 2020). [Unpublished]. American Soybean Association & World Initiative for Soy in Human Health. (2020). Commercialization of Aquaculture for Sustainable Trade (CAST) – Cambodia: Semi-annual Report (October 1, 2020–March 31, 2021). [Unpublished]. American Soybean Association & World Initiative for Soy in Human Health. (2020). Commercialization of Aquaculture for Sustainable Trade (CAST) – Cambodia: Comments on the Plan of Operations. [Unpublished]. American Soybean Association & World Initiative for Soy in Human Health. (2020.) Midterm Evaluation (MTE) For Cast: Request For Proposals. [Online]. Available at: http://www.usaedc.org/media/2385/cast-cambodia-mte-rfp_110920.pdf. Joffre, O., Pant, J., Somony, T., Chantrea, B. & Viseth, H. (2019). Transforming aquaculture in Cambodia through introduction of improved tilapia. Penang: WorldFish, pp. 1–20. [Online]. Available at: digitalarchive.worldfishcenter.org/bitstream/handle/20.500.12348/2663/2019-03.pdf?sequence=2.. [Accessed 23 November 2020]. Leakhena, C., Viseth, H., Somony, T., Da, C., Samnang, P. & Bunly, C. (2018). Aiming for Improved Gender Equity in Cambodian Aquaculture. Fish for the People, 16(2). 117 Manet, S. (2019). “With aquaculture on the rise, fish production up 6 pct in 2018.” Khmer Times, February 6, 2019. [Online]. Available at: www.khmertimeskh.com/575673/with-aquaculture-on-the￾rise-fish-production-up-6-pct-in-2018/. [Accessed 23 November 2020]. Olaganathan, R. & Mun, A. T. K. (2017). Impact of aquaculture on the livelihoods and food security of rural communities. International Journal of Fisheries and Aquatic Studies, 5(2). Ouch, L. (2015). Current Status of Sustainable Aquaculture in Cambodia. Bangkok: Southeast Asian Fisheries Development Center, pp. 1–14. [Online]. Available at: repository.seafdec.org.ph/bitstream/handle/10862/2761/LangO2015.pdf?sequence=1&isAllowed=y. [Accessed 23 November 2020]. Richardson, R. B. & Suvedi, M. (2018). Assessing the Potential for Small-Scale Aquaculture in Cambodia. Environments, 5(76). Richardson, R., Sudevi, M., Kaplowitz, M. & Mith, S. (2017). Cambodia HARVEST Aquaculture Program Evaluation in R. Ghimire, M. Kaplowitz, & M. Suvedi (Eds.), Feed the Future - Cambodia Impact Evaluation and Strengthening of Local Capacity to Monitor and Evaluate Food Security Programs (pp. 55–95). [Online]. Available at: pdf.usaid.gov/pdf_docs/PA00N42Q.pdf#page=62. [Accessed 23 November 2020]. World Fish. (2019). Baseline Study Report: Commercialization of Aquaculture for Sustainable Trade (CAST) Cambodia, [Online]. Available at: https://pdf.usaid.gov/pdf_docs/PA00XB16.pdf. [Accessed June 29, 2021]. 118 Annex 10: Terms of reference REQUEST FOR PROPOSAL (RFP) FROM THE WORLD INITIATIVE FOR SOY IN HUMAN HEALTH (WISHH) PROGRAM MIDTERM EVALUATION (MTE) FOR CAST CAMBODIA FFPr18-RFP-01-2020 The American Soybean Association’s (ASA) World Initiative for Soy in Human Health (WISHH) program is posting this Request for Proposals (RFP) for a consultant to conduct the Midterm Evaluation (MTE), under the Food for Progress (FFPr) 2018 Project funded by the United States Department for Agriculture (USDA) and Implemented by the Commercialization of Aquaculture for Sustainable Trade (CAST) project in Cambodia. ISSUANCE DATE November 9th, 2020 PROJECT OVERVIEW The purpose of this RFP is to solicit applications from companies and/or individuals to conduct a Midterm Evaluation (MTE) for ASA/WISHH’s CAST project in Cambodia. CAST is a five-year USDA-funded project that was awarded in September 2018. The MTE will provide CAST with meaningful feedback about what progress has been made towards reaching performance indicator targets. Additionally, this exercise should assess the relevance of the intervention, provide some insight about the effectiveness of the project, document lessons learned, assess sustainability, and discuss and recommend course corrections. The external evaluator conducting this evaluation should highlight the strengths and weaknesses of the project and make justifiable recommendations for next steps. A data quality assessment (DQA) will also be conducted at this time to evaluate key M&E functional areas (e.g., data entry protocols, indicator definitions, reporting tools) and provide the project with a better sense of where improvements could be made (e.g., filing practices, reporting timelines, data verification). The MTE team will be comprised of the lead consultant and his/her assistants, and shall work in cooperation with USDA’s Monitoring and Evaluation Staff (MES) in Washington, D.C., ASA/WISHH’s Director of M&E based in St. Louis, Missouri, CAST’s Chief of Party and Monitoring, Evaluation & Learning (MEL) Manager based in Cambodia, and CAST’s implementing partners in Cambodia (i.e., World Vision, KSU/CE SAIN, Auburn University). The Period of Performance of this contract is expected to fall between January and June 2021. The anticipated award date for this contract is January 4th, 2021. The final report should be submitted to ASA/WISHH on or before June 28th, 2020. ORGANIZATION BACKGROUND The American Soybean Association (ASA) was founded in 1920 by soybean farmers and extension workers to promote soy for high protein applications in developmental settings. Overseas activities initiated in the mid-1950s and to date ASA has worked in over 80 countries. The World Initiative for Soy in Human Health (WISHH) was founded in 2000 to expand the work of the American Soybean Association (ASA) in developing emerging markets to improve health, nutrition and food security—building the groundwork for future markets of soy. WISHH provides services in food technology, business development, nutrition services, and program and proposal development. WISHH expertise extends its network into additional areas, such as aquaculture and animal feed. ASA/WISHH connects trade and development to strengthen agricultural value chains in emerging markets, creating trade and long-term demand for U.S. soy. Trade can improve lives worldwide for both farmers and consumers. U.S. soy trade in emerging markets is pivotal to improve accessibility, affordability, and acceptability of high-quality plant and animal-sourced proteins in developing economies. Rising incomes in emerging economies generate further opportunity for trade. ASA/WISHH builds opportunity for long-term trade by improving agricultural value chains, human and animal nutrition, and farmer net incomes. ASA/WISHH initiatives broadly fit in two arenas: (1) trade-building long-term, early-stage market development, and, (2) trade-building 119 international agriculture and economic development. The St. Louis-based Program operates in sub-Saharan Africa, Asia and Central America. ASA/WISHH has supported market and economic development activities in the human food sector since its inception in 2000 and works to increase the use of quality soy protein ingredients and products in food for human consumption through commercial development and nutrition programming. ASA/WISHH offers over six years of proven feed sector capacity and decades of accumulated knowledge from ASA programs. Principal approaches include market and economic assessments, technical assistance and capacity building, food and feed trials and demonstrations, farmer field days, youth mentorship programs, food and feed value chain development, and market linkages. ASA/WISHH achieves its mission by working in close partnership with: (1) the public sector (e.g., USDA, USAID, U.S. land grant universities, and host country governments), (2) the private sector (e.g., trade associations: Qualified State Soybean Boards and U.S. Soybean Export Council), and (3) private voluntary organizations. The Program draws upon the resources and experience of these partners and the services of a cadre of ASA/WISHH consultants with technical expertise in agricultural, international and commercial development spheres. ASA/WISHH relies on decades of experience in food commercial development and agricultural development programming. The Program takes pride in its growing portfolio of success with U.S. government-funded projects, including USDA (Food for Progress, McGovern-Dole, Global Broad-Based Initiative, Foreign Market Development, Market Access Program, Emerging Markets Program, and Quality Samples Program) and USAID funding in both prime and sub-recipient capacities. ASA/WISHH has also attracted both private sector and other complementary funding sources from various donors to build on and leverage core funding from Qualified State Soybean Boards. For more information about ASA/WISHH, please visit wishh.org. PROJECT DESCRIPTION CAST Cambodia is a five-year program financed by the USDA and implemented by ASA/WISHH signed by both parties on September 27th, 2018. CAST’s operational budget consists of $15.55 million in monetization proceeds plus $1.62 million in CCC funds. The M&E budget for this program is $2.08 million or 12.1% of the total project budget. The project supports the USDA FFPr objectives of increased agricultural production and trade by increasing the use of improved agricultural techniques and technologies among target populations and improving linkages between buyers and sellers. CAST will enroll and provide training and technical assistance to 900 aquaculture producers working at 600 businesses across seven Cambodian provinces (i.e., Siem Reap, Battambang, Pursat, Kampong Thom, Kampong Cham, Kandal, Phnom Penh). The project also plans to work with 14 hatcheries/nurseries, 12 feed distributors, and five feed mills and will provide business development training and financing support to 117 post-production actors. The two main objectives of CAST Cambodia are: Increase agricultural productivity in the freshwater aquaculture industry through improved inputs and practices; and Increase trade in Cambodian aquaculture, by volume and value. These objectives will be supported by the following activities: Activity #1: Capacity Building: Agricultural extension agents/services ASA/WISHH will build the capacity of Cambodian aquaculture education and extension systems through the transfer of U.S. agricultural research and extension methodologies. ASA/WISHH will conduct needs assessments with stakeholder groups to determine technology gaps and develop training agendas for conducting capacity building activities. ASA/WISHH will also support the establishment of an aquaculture professional course and certification for university students and professionals and will institute an internship/mentorship program to pair aquaculture students with industry professionals. Furthermore, ASA/WISHH will support the Center of Excellence for Sustainable Agricultural Intensification and Nutrition (CE SAIN) to establish aquaculture training and feed demonstration facilities. 120 Activity #2: Inputs: Develop agrodealers and/or other input suppliers ASA/WISHH will increase supply and improved access to quality, domestically-produced aquaculture inputs by providing targeted technical assistance to private hatcheries and nurseries. ASA/WISHH will work to develop aquaculture small to medium-sized enterprises (SME), including seed, feed, and veterinary supplies. ASA/WISHH will conduct an assessment of the seed value chain to assess technical needs, supply chain challenges, and barriers to capacity extension. Through this assessment, ASA/WISHH will provide technical assistance to private hatcheries and feed mills, develop a local aquaculture feed database, and support the development of aquaculture feed formulations. Furthermore, ASA/WISHH will work on the development of aquaculture feed standards in collaboration with the Ministry of Agriculture, Forestry and Fisheries, and support participation of freshwater aquaculture actors in local trade promotional events. Activity #3: Training: Improved farm management ASA/WISHH will work with large and medium-sized intensified pond and cage operations in order to increase their productivity and profitability by addressing issues related to technical knowledge and access to quality inputs, including feed and seed and aquaculture infrastructure. ASA/WISHH will provide technical assistance to SME aquaculture producers and link input supplies to potential buyers. Activity #4: Training: Demonstration plots ASA/WISHH will establish model aquaculture and training hubs. These research demonstration sites will serve as hubs for expertise on freshwater aquaculture information, training and applied research in aquaculture. ASA/WISHH will work with key stakeholders to develop technology park facilities into primary locations for showcasing proven aquaculture research and training. Activity #5: Training: Sanitary and phytosanitary standards (SPS) ASA/WISHH will support private sector actors to develop voluntary standards based on the Government of Cambodia’s proposed sanitary-phytosanitary (SPS) framework and will help facilitate the transition to the government’s final standards. ASA/WISHH will engage with the Government of Cambodia to facilitate standards and measures adopted by the private sector. ASA/WISHH will develop a communications campaign among value chain actors and consumers to disseminate the benefits of SPS standards to the public. ASA/WISHH will engage with the Department of Fisheries Post-Harvest Technologies and Quality Control to train private sector actors on voluntary industry SPS standards. Activity #6: Capacity Building ASA/WISHH will strengthen the business management capacity of aquaculture enterprises, input and service providers, and other value chain stakeholders. ASA/WISHH will provide technical assistance and support identified stakeholders to develop and implement business plans. ASA/WISHH will provide financial support through cost-sharing capital investments to improve market position. Furthermore, ASA/WISHH will establish a technical assistance loan facility to provide revolving loans to SMEs seeking technical assistance. Activity #7: Capacity Building: Producer groups/cooperatives ASA/WISHH will organize aquaculture stakeholders, including producers, aggregators, wholesalers, and input suppliers at local and national levels to address issues of common interest, including public policy, development of voluntary standards, information sharing, and general promotion of Cambodian aquaculture to consumers. ASA/WISHH will facilitate the creation of a Cambodian national aquaculture association. ASA/WISHH will conduct outreach to promote the association, solicit feedback on association services, and identify potential members. ASA/WISHH will facilitate the selection of association leadership and support members to develop the association mission, services, and sustainability strategies. Once formed, ASA/WISHH will facilitate recruitment via member outreach and promotion, and support participation from project stakeholders. Furthermore, ASA/WISHH will train association leadership and staff in marketing and advocacy strategies, and policy analysis. 121 Activity #8: Training: Improved marketing and branding ASH/WISHH will raise consumer awareness of high-quality, Cambodian-grown, freshwater aquaculture fish and strengthen buyer-seller linkages along the value chain. ASA/WISHH will organize and launch a consumer awareness and branding campaign targeting consumers, consumer facing companies, and exporters. ASA/WISHH will conduct a market survey on consumers’ current perceptions, preferences, purchasing considerations, and identify media contact points. ASA/WISHH will facilitate the identification and prioritization of consumer perceptions, development of behavior change messaging, identification and selection of media, and design of a marketing and branding strategy. ASA/WISHH will support producers, wholesalers, distributors, and retailers to build a sustainable supply chain for locally-sourced quality Cambodian fish. ASA/WISHH will promote value-chain linkages and branding techniques for advocating locally produced fish. ASA/WISHH will link high-volume domestic buyers and industries with local producers. ASA/WISHH will also identify regional market opportunities for export and assist exporters in finding reliable supply chains of quality fish. ASA/WISHH will facilitate marketing events, including trade shows and discussion panels to support buyer-seller relationships and promote local aquaculture. Activity #9: Market Access: Facilitate a traceability system for domestic aquaculture ASA/WISHH will support selected distributors and export companies to pilot a traceability system to support compliance with production and sanitary and phytosanitary standards. ASA/WISHH will identify a cost-effective technology platform and support industry stakeholders to establish effective, business-friendly protocols to be integrated into a traceability software program. Once protocols are established, ASA/WISHH will train stakeholders on traceability protocols and the use of software and equipment. ASA/WISHH will ensure that marketing materials, consumer awareness/branding campaigns, and export promotion initiatives incorporate information about the benefits of traceability to ensure sanitary and phytosanitary standard compliance. ASA/WISHH will co-invest in the operational costs of the traceability system, and train selected participants on operations management and reporting requirements. Activity #10: Financial Services: Provide SME finance, technical assistance loans and grants ASA/WISHH will commission and publish a study outlining the business environment, growth potential, and market opportunities in the aquaculture sector, as well as recommendations for financial institutions, the private sector, and other stakeholders and aquaculture initiatives. ASA/WISHH will conduct project seminars, farmer￾field days, agricultural exhibitions and other project activities with representatives from key agricultural financing institutions to build relationships with sector stakeholders and leaders. ASA/WISHH will provide co-investment opportunities in promising aquaculture value chain enterprises. ASA/WISHH will identify a local financial institution to manage co-investment capital, including joint development of investment criteria, return-on-investment hurdles, and sub-sector prioritization. ASA/WISHH will also establish a separate no-interest revolving technical assistance loan fund. Technical assistance loans will enable the provision of technical and business development services needed to grow and expand SMEs. ASA/WISHH will support a private agricultural insurance provider to collect quality data related to pond and cage production. Additionally, ASA/WISHH will facilitate the insurance product’s introduction to the aquaculture value chain via farm visits, access to information including production models, and meetings with stakeholders. ASA/WISHH will work with the private sector to test new insurance products via simulations with a representative sample of producers to improve the design and reach a final product that can be taken to market. By the end of the project period, it is expected that ASA/WISHH will have facilitated the first underwritten insurance policies for the aquaculture farmers in Cambodia, which will lay the groundwork for expansion and growth to further reduce the risk in commercial aquaculture. SCOPE OF WORK MTE Objectives ASA/WISHH is seeking the services of a third-party consultant or team of consultants to carry out a Midterm Evaluation (MTE) for CAST Cambodia in accordance with the terms of reference (TOR) herein. 122 The MTE is meant to serve the following purposes: To determine the extent to which the project is on track to achieving stated goals and objectives, To highlight major achievements, strengths, and weaknesses of the project including factors influencing project achievements, and To provide recommendations for mid-course corrections and strategic programmatic adjustments The MTE will also seek to identify key lessons learned and best practices that can be capitalized on in the second￾phase of implementation to accelerate achievement of the project’s stated objectives. The performance evaluation will be conducted using the Relevance, Effectiveness, Efficiency, Management (Planning, Monitoring and Controlling), Coordination and Partnership, Lessons Learned and Best Practices criteria. This will include an assessment of the project management strategies and coordination mechanisms among key partners and their implications for sustaining project gains. While designing the study and creating the methodology for data collection, the Consultant should consider CAST’s Project-Level Results Framework(s) (Attachment B). Research Questions The Consultant should propose a detailed and reproducible strategy for answering the following research questions: Relevance To what extent is the project aligned with national policies and strategies for aquaculture and feed production? How relevant is the project to the stated and immediate needs of stakeholders and beneficiaries? Effectiveness To what extent is the program producing worthwhile results (outputs, outcomes) and/or meeting each of its objectives? What, if any, unintended consequences to the aquaculture sector can be attributed to the project? Efficiency To what extent is the relationship between inputs and outputs timely, cost-effective and to the expected standards? To what extent has the project been efficiently implemented and managed (including financial review and audit information), as appropriate? Impact Has the project contributed to short- and medium-term socioeconomic improvements for aquaculture producers, hatcheries and nurseries, and feed mills? What difference has the project made in the lives of beneficiaries? Sustainability To what extent has the project fostered ownership by local and regional partners (e.g., CE SAIN, CTU)? To what extent has the project established/enhanced capacity, processes and local systems that are likely to be sustained? It is also expected that the Consultant will work with ASA/WISHH staff to further develop these questions. The Consultant will use a combination of primary and secondary data to answer these questions. CAST’s Monitoring, Evaluation, and Learning (MEL) Manager will provide the Consultant access to the project’s performance monitoring data. 123 The following is a preliminary list of standard indicators that the Consultant will need to consider for the performance evaluation: Number of hectares under improved management practices or technologies that promote improved climate risk reduction and/or natural resources management; Number of hectares under improved management practices or technologies; Number of individuals in the agriculture system who have applied improved management practices or technologies; Yield of targeted agricultural commodities; Value of annual sales of farms and firms; Volume of commodities (metric tons) sold by farms and firms. The following is a preliminary list of custom indicators that will need to be considered for the performance evaluation: Number of private enterprises, producer organizations, water user associations, women’s groups, trade and business associations, trade and business associations, and community-based organizations (CBOs) who received short-term agricultural sector productivity, food security or other training as a result of USDA assistance; Area of ponds (cubic meters) under improved management practices or techniques with USDA assistance; Number of market actors selling fish who are compliant with national SPS standards or are using an independent quality seal; Number of farms that increased the number, size, or amount of ponds, cages, or equipment with USDA assistance This information will also help the project to conduct proper and accurate monitoring of the program performance and results. Where applicable, the standard and customs indicators should be disaggregated as specified in CAST’s PMP. Approach and Methodology The MTE should apply a mixed-methods approach with both quantitative and qualitative measures and tools appropriate to the stated research questions. It is expected that both quantitative and qualitative data is provided to support findings. Additionally, even if/when secondary data are available for the Consultant to build on, attempts should be made to obtain primary data with reliable methods to ensure data integrity. ASA/WISHH expects that the Consultant will collect data from the following partners, stakeholders, and beneficiaries: World Vision, Kansas State University (KSU)’s Feed the Future for Sustainable Intensification Innovation Laboratory (SIIL), Center for Excellence on Sustainable Agricultural Intensification and Nutrition (CE SAIN), Auburn University, Can Tho University (CTU) Fisheries Administration (FiA) at the Cambodian Ministry of Agriculture, Forestry and Fisheries (MAFF), and the various Cambodian Government offices working in Aquaculture as well as members of EU- CAPFISH led aquaculture sub-working group Feed mills (e.g., AgriMaster/Medivet), aquaculture producers, hatcheries and nurseries, fish processors and buyers, financial institutions Middlemen/services providers, retailers/supermarkets, transporters, distributors The methods and tools used in the study should include a combination of the following: Document review: The Consultant will find it useful to reference a broad range of background documents related to the aquaculture sector in Cambodia apart from project documents provided by ASA/WISHH. The Consultant will also review the project’s Baseline Study Report, Semiannual Reports (SAR), Plan of Operations, Annual 124 Work Plan, and several M&E tools and materials (e.g., Data Collection Manual, Performance Monitoring Plan, Indicator Performance Tracking Table). On-site data verification: A visit to the project sites will be conducted to; 1) conduct a systems assessment to ascertain the robustness of data collection and reporting processes; and 2) verify the data sources and ascertain the authenticity of data reported to USDA. Where sub-grantees are involved, field visits to their sites of operation will be organized as well. Data quality will be assessed using the five USDA DQA criteria of Validity, Precision, Reliability, Timeliness, and Integrity. CAST will provide the Consultant with USAID’s Data Quality Checklist and, if necessary, provide additional information about how this tool should be administered. The Consultant will utilize and refine CAST’s Baseline Survey and routine monitoring tools (e.g., Full Enrollment Form, Techniques/Technologies Adoption Tracker, Sales Record Form) as needed during this exercise. CAST verifies 5% of records submitted each quarter and expects the Consultant to include a representative sample of beneficiaries in this verification task. The Consultant will be expected to survey 40-50 aquaculture producers, 10+ hatcheries/nurseries, 5-10 feed distributors, 20-25 post-production actors, and staff at one feed mill. Key Informant Interviews: The Consultant will review the interviewer scripts developed for CAST’s Baseline Study and conduct Key Informant Interviews (KII) with a select number of ASA/WISHH and partner staff and stakeholders. These interviews will include a select number of financial institutions, FiA representatives at MAFF, fish processors and buyers, and members of the EU-CAPFISH led aquaculture sub-working group. Data Analysis and Reporting: The Consultant will use an objective criteria for analyzing DQA data. Working sessions will be organized for all technical staff who will take part in the annual reviews to analyze the data obtained from both the document review and the on-site data verification exercises. Findings from the analysis including relevant recommendations will be duly documented. If requested, ASA/WISHH will assist in making appointments with partners, community members, government officials and other stakeholders. Deliverables The Consultant is expected to provide the following deliverables to ASA/WISHH: Implementation Plan The Consultant will submit a draft of their implementation plan on or before the 18th of January. This document should include the evaluation approach, methods for data collection, overview about data processing and analysis, list of team members’ roles and responsibilities, and planned schedule of activities. The final implementation plan should be submitted on or before the 1st of February and include a step-by-step account of how data will be collected and managed during this exercise. This document should also include a detailed schedule of activities and data collection instruments, tools, and guidelines. It is expected that data collection will commence on or around February 22nd and be completed within six weeks or by April 5th . Data Analysis Plan The Consultant will submit a Data Analysis Plan that clearly describes the sampling strategy to be used for the MTE and which includes an overview of how data will be analyzed. This document should be submitted to ASA/WISHH on or before February 15th. The Data Analysis Plan should describe what procedures will be used to analyze quantitative data and how the Consultant will triangulate qualitative data. The Consultant will be expected to conduct a theme analysis on this data using approved qualitative data analysis (QDA) software. Each data collection, analysis, and/or presentation tool, including the specific instruments, needs to be approved by ASA/WISHH prior to the start of data collection. Descriptive Report The Consultant will complete a desktop review of all data and share a Descriptive Report with ASA/WISHH on or before April 19th. Data analysis should be completed by May 17th and a first draft of the MTE Report should be submitted to ASA/WISHH on or before May 31st. ASA/WISHH will review this report and provide feedback to the 125 Consultant within two weeks. The Consultant will be expected to review these comments and submit a final copy of this report by June 28th, 2020. MTE Report and MTE Data The MTE Report, not to exceed 30 pages (not including annexes), should be concise and to the point, utilizing charts, graphs and diagrams where appropriate. The Consultant will be required to describe methods used for data collection in enough detail so that the results reported are reproducible. This document should make clear linkages between the data collected and analyzed to key findings reported in the results section of the report. As described, the Consultant will be expected to solicit and incorporate feedback from ASA/WISHH before submitting their final report. The following table displays requirements for the final report: Language ENGLISH Report Length Maximum of 30 pages, excluding the Table of Contents and Annexes Executive Summary Include a two-page Executive Summary that provides a brief overview of the study’s purpose, project background, methods, findings, and lessons learned. Methods Explain study methodology in detail. Disclose limitations, especially those associated with the methodology (e.g., selection bias, recall bias). NOTE: A summary or methodology can be included in the body of the report, with a full description provided as an annex. General overview and research questions Provide a general overview and analysis of the current status of the aquaculture sector in target areas. Findings and MTE data Present findings, data, and supporting documentation supported by strong quantitative and qualitative evidence and not anecdotes or hearsay. 126 Recommendations Provide recommendations for the project’s MTE staff on effective data collection and monitoring of the program, lessons learned, and best tools and methodologies to be applied. Annexes Include the following as annexes, at minimum: Terms of Reference Full description of the methodology used for the MTE All MTE tools (e.g., questionnaires, checklists, discussion guides, surveys) A list of sources of information (e.g., key informants, documents reviewed, other data sources) Performance indicator table(s) Conflict of interest forms If applicable, include as an annex Statement(s) of Differences regarding any significant unresolved differences of opinion on the part of funders, implementers, and/or members of the research team. The Consultant will be expected to provide ASA/WISHH with access to all primary data collected for the MTE. This should include all cleaned data sets and the raw data collected for this exercise. Please note that the final report will be made publicly available. This document should not include any personally identifiable information (PPI) or propriety information. The final report should be accessible to persons with disabilities. For guidance on creating documents accessible to persons with disabilities, please see the following resources: https://www.section508.gov/create/documents https://www.section508.gov/create/pdfs Evaluation Brief The Consultant will develop a 2-3 page brief that describes the evaluation design, key findings, and other relevant considerations. This document will help to inform any interested stakeholders about the MTE and should be written in language easily understood by the public. It should include tables and graphs where appropriate. Research Team The Consultant must have a team comprised of individuals with strong technical skills, experience in quantitative and qualitative research, study design, data collection and analysis, and technical competence in key project activities. In particular, the lead Consultants(s) will have the following qualifications: At least ten years of demonstrated experience conducting studies and/or evaluations of agricultural development programs (required) and hold a graduate degree in agriculture, monitoring and evaluation, statistics, economics, or other related field (preferred). Experience conducting similar studies in Southeast Asia (required) and within Cambodia’s aquaculture industry (preferred). Consultants who do not have experience evaluating aquaculture programs in Cambodia are strongly encouraged to sub-contract with a technical expert(s) based in country to collect data for the MTE. Experience conducting studies for USDA, United States Agency for International Development (USAID), or other United States Government (USG) projects (required). Preference will be given to those who demonstrate experience with USDA and/or USAID indicators. 127 Experience in quantitative data collection, statistics/econometrics such as quasi-experimental design, propensity score matching, regression, design effects, and questionnaire development. Experience designing tools for focus group discussions and key informant interviews and facilitating these sessions Demonstrable knowledge of participatory methods and commitment to best practices. Experience and capability in producing survey reports. Experience in study design and data collection and analysis of quantitative and qualitative data and supervisory capabilities of each of these areas. Fluent in written and spoken English; proficient in written and spoken Khmer (preferred). The assessment will be independent and conducted by a third party. Specifically the regulations specify that the third party conducting the assessment: Is financially and legally separate from the participant’s organization; Has staff with demonstrated knowledge, analytical capability, language skills and experience in conducting assessments of development programs involving agriculture, education, and nutrition; Uses acceptable analytical frameworks such as comparison with non-project areas, surveys, involvement of stakeholders in the assessment, and statistical analyses Uses local consultants, as appropriate, to conduct portions of the assessment; and, Provides a detailed outline of the evaluation, major tasks, and specific schedules prior to initiating the evaluation. The Consultant is expected to: Develop a comprehensive study and data collection methodology (e.g., sample size and sampling methods). Conduct/coordinate and supervise quantitative and qualitative data collection. Communicate directly and frequently with ASA/WISHH staff (e.g., weekly updates, monthly progress reports) and invite stakeholders to participate in KII. Conduct data analysis and generate summary of findings. Draft the final report. ASA/WISHH staff and implementing partners will be available to answer technical questions about program structure and implementation, and to provide guidance/advice on logistics, meeting arrangements, and other matters. TIMING The Consultant should complete all activities by June 28th, 2021. Intent to Bid (For Q&A Dissemination) November 23rd, 2020 Proposals due to ASA/WISHH December 7th, 2020 Contract awarded December 21st, 202 Contract signed and activities begin January 4th, 2021 First draft of Implementation Plan January 18th, 2021 Final Implementation Plan* February 1st, 2021 Data Analysis Plan February 15th, 2021 128 Start primary data collection February 22nd, 2021 Finish collecting primary data April 5th, 2021 Complete Descriptive Report April 19th, 2021 Finish data analysis May 17th, 2021 First draft of MTE Report to ASA/WISHH May 31st, 2021 Submit draft MTE Report to USDA June 7th, 2021 Final draft of MTE Report to ASA/WISHH June 28th, 2021 Submit final MTE Report to USDA June 30th, 2021 * With schedule of activities and data collection instruments, tools, and guidelines SELECTION CRITERIA AND CRITERIA WEIGHTS Proposals submitted will be evaluated according to the following criteria: Soundness of proposed 40% Skills and past experience 30% Past aquaculture experience in Cambodia 10% Quality control methods 10% Financial competitiveness 10% TOTAL 100% Special consideration will be given to Women or Minority Small Owned Business (WMSOB) entities with SBA certification (proof of certification must be provided at time of submission). INSTRUCTIONS TO BIDDERS Proposals must contain at a minimum the specific criteria requested in the Request for Proposal: The candidate shall submit a full proposal to ASA/WISHH via an electronic submission to James Bernhardt at jbernhardt@soy.org (cc: tfang@soy.org) no later than Monday, December 7th, 2020 at 5pm CST with the following: Curriculum Vitae(s) of the lead consultant and assistants References of similar work conducted by the applicant and recommendations if available. A technical proposal not to exceed 10 pages (no including annexes) that includes a clear description of the MTE’s objectives and demonstrates a good understanding of the scope of work, proposed methodology, and plan for data collection and analysis. A line item budget including staff, travel, equipment, supplies, and professional services, if applicable. A detailed work plan showing the resources needed, the time anticipated, and the outcomes to be reached that includes a timeline of deliverables. Proposals and associated materials must be submitted in English. Checklist of items that must be submitted: Attachment A Proposal Authorization Cover Page Completed and Signed 100-word executive summary of the proposal 129 Full detailed proposal Proposal detailed budget WMSOB Proof of Certification (if applicable) Proposals received after 5:00 PM CST on Monday, December 7th, 2020 will not be considered. ASA/WISHH will consider proposals with a maximum consulting budget and materials no greater than $150,000. This is an all-inclusive contract which includes individual’s or firm’s time, materials and all travel related costs. Travel costs will not be reimbursed separately. No further funds will be allocated for this activity. If your proposal is not authorized by signature on Attachment A Proposal Authorization Cover Page, it may not be considered and may be rejected. Non-Discrimination Statement: In accordance with Federal civil rights law and U.S. Department of Agriculture (USDA) civil rights regulations and policies, the USDA, its Agencies, offices, and employees, and institutions participating in or administering USDA programs are prohibited from discriminating based on race, color, national origin, religion, sex, gender identity (including gender expression), sexual orientation, disability, age, marital status, family/parental status, income derived from a public assistance program, political beliefs, or reprisal or retaliation for prior civil rights activity, in any program or activity conducted or funded by USDA (not all bases apply to all programs). Remedies and complaint filing deadlines vary by program or incident. ASA/WISHH is an equal opportunity provider and employer. The firm or individual selected will be required to sign the ASA/WISHH code of conduct and conflict of interest statement. Award: This RFP does not commit the ASA/WISHH to award a contract or to pay any costs incurred in the preparations or submission of proposals, or costs incurred in making necessary studies for the preparation thereof or to procure or contract for services or supplies. The ASA/WISHH reserves the right to reject any or all proposals received in response to this RFP and to negotiate with any of the vendors or other firms in any manner deemed to be in the best interest of the ASA/WISHH. The ASA/WISHH reserves the right to negotiate and award only a portion of the requirements; to negotiate and award separate or multiple contracts for the elements covered by this RFP in any combination it may deem appropriate, at its sole discretion to add new considerations, information or requirements at any stage of the procurement process, including during negotiations with vendors; and reject proposal of any vendor that has previously failed to perform properly or in a timely manner contracts of a similar nature, or of a vendor that, in the opinion of the ASA/WISHH, is not in a position, or is not sufficiently qualified, to perform the contract. This RFP contains no contractual proposal of any kind, any proposal submitted will be regarded as a proposal by the vendor and not as an acceptance by the vendor of any proposal by the ASA/WISHH. No contractual relationship will exist except pursuant to a written contract document signed by the authorized procurement official of the ASA/WISHH and by the successful vendor(s) chosen by the ASA/WISHH. Offerors submitting proposals must (1) be officially licensed to do such business in the country of their business operation, (2) be able to receive USDA funds, and (3) not have been identified as a terrorist prior to being awarded a contract. In addition, Offeror may be required to provide the following information prior to awarding of the contract: Documentation to verify licensure (i.e. tax id, registration certificate, etc.) Code of Conduct Conflict of Interest Attachments Attachment A: Technical Proposal Submission Sheet Cover Page Attachment B: Project Level Results Framework(s) 130 ATTACHMENT A: Technical Proposal Submission Sheet (Complete this form with all the requested details and submit it as the first page of your technical proposal, with the documents requested above attached. Ensure that your proposal is authorized in the signature block below. A signature and authorization on this form will confirm that the terms and conditions of this RFP prevail over any attachments. If your proposal is not authorized, it may be rejected.) Date of Technical Proposal: RFP Number: RFP Title: We offer to provide the goods/services described in the Scope of Work, in accordance with the terms and conditions stated in the Request for Proposal referenced above. We confirm that we are eligible to participate in public procurement and meet the eligibility criteria specified. The validity period of our proposal is days/weeks/months from the time and date of the submission deadline. Type of Business/Institution (Check all that apply) Offeror certifies that it is: [ ] Non U.S. Owned/Operated [ ] Government Owned/Operated (If Non U.S. Owned/Operated is selected, continue to Anti-Terrorism Certification) OR FOR U.S. ORGANIZATIONS ONLY: [ ] Non profit [ ] For-profit [ ] Government Owned/Operated [ ] Large Business [ ] Small Business[ ] College or University [ ] Women Owned [ ] Small and Disadvantaged Business Anti-Terrorism Certification The Offeror, to the best of its current knowledge, did not provide, within the previous 10 years, and will take all reasonable steps to ensure that it does not and will not knowingly provide, material support or resources to any individual or entity that commits, attempts to commit, advocates, facilitates, or participates in terrorist acts, or has committed, attempted to commit, facilitated, or participated in terrorist acts. The Offeror also verifies that it does not appear on 1) the website of the Excluded Party List: https://www.sam.gov; or 2) the website of the United Nations Security (UNSC) sanctions committee established under UNSC Resolution 1267 (1999) (the “1267 Committee”): http://www.un.org/Docs/sc/committees/1267/consolist.shtml. The undersigned declares s/he is authorized to sign on behalf of the company listed below and to bind the company to all conditions and provisions stated in the original RFP document including attachments from ASA/WISHH. Proposal Authorized By: Signature: Position: Name: Date: Authorized for and on behalf of: (DD/MM/YY) Company: Address: Duns No: Business Registration Number: ATTACHMENT B: Project Level Results Framework(s) 131