FY 2017 Food for Progress Creating Linkages for Expanded Agricultural Networks (CLEAN) Project in Lao PDR. Midterm Evaluation 20 August 2021 DISCLAIMER: This publication was produced at the request of the United States Department of Agriculture. It was prepared by an independent third-party evaluation firm and edited for clarity by Winrock International. The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Midterm Evaluation Program Title: Creating Linkages for Expanded Agricultural Networks (CLEAN) Agreement Number: FCC-439-2017/028-00 Funding Year: Fiscal Year 20 Project Duration: September 6, 2017 to December 31, 2022 Implemented by: Winrock International Evaluation Authored by: Elixirs Sole Co., Ltd. Report writing and analysis by: Ms. Barbara Ann Lewis Mrs. Phetmany Cheuasongkham Mr. Santi Inthavong 1 Table of Contents List of Acronyms and Abbreviations 4 Executive Summary 6 Background 6 Evaluation, Purpose and Objectives 6 Methodology 6 Key Findings 7 Activity 1: Facilitate increased access to regional and high-value domestic markets 7 Activity 3: Strengthen production systems 8 Activity 7: Grants to incentivize innovation and investment 9 Assessment of Project Progress 10 Conclusions and Recommendations 11 1. Introduction and Purpose 16 1.1. Project description 16 1.2. Progress of the program 21 1.3. Overall project management 24 1.4. Purpose of the evaluation 29 2. Evaluation Design and Methodology 29 2.1. Evaluation questions 29 2.2. Evaluation design 30 Review of project documents and data 30 Beneficiary-based survey 30 Sampling size methodology 30 Beneficiaries-Based Survey Questionnaires 31 BBS sample size vs. actual sample size interview 31 2.3. Focus group discussions 33 2.4. Stakeholder interviews 35 2.5. Training of field supervisors and enumerators 35 2.6. Data collection methods 35 2.7. Data management and data analysis of descriptive statistics 35 2.8. Ethical consideration and personal data protection 36 2.9. Evaluation limitations 36 3. Findings 37 3.1 Baseline findings 37 3.2 Key findings from MTE 38 a) Demographic and agriculture profile 38 2 b) Findings by Indicators 41 c) Findings from stakeholder interviews 56 d) Findings from the FGDs 58 Cabbage group discussion: 59 Cassava group discussion: 61 Coffee group discussion: 63 Sacha inchi group discussion: 65 3.3 Assessment of project progress 66 4 Conclusion and Recommendations 67 Annexes 77 Annex 1. Sampling Size Tables by Commodity for MTE BBS 78 Annex 2. Annotated Bibliography 80 Annex 3. GAP Certification Process 82 Annex 4. Guiding Questions for FGDs 85 Annex 5. Guiding Questions for KIIs 105 Annex 6. Approved Terms of Reference 111 Annex 7. Participant and Event Disaggregates 124 Annex 8. The Calculation of Sampling Weights 132 List of Figure Figure 1: CLEAN’s Theory of Change ................................................................................................ 17 Figure 2: CLEAN Results Framework ................................................................................................. 20 List of Chart Chart 1: Main water source by commodity ....................................................................................... 40 Chart 2: Percentage of share of respondents applying improved techniques ............................ 43 Chart 3: Period of first adoption of improved technology by commodity ................................. 45 Chart 4: Percentage of participants reporting usefulness of project trainings .......................... 45 Chart 5: Percentage of famers who said they are applying techniques learned from project 46 3 List of Tables Table 1: Expected Results and Key Activities ................................................................................... 18 Table 2: CLEAN's performance indicators by routine reporting system results ...................... 25 Table 3: Evaluation questions and methodology ............................................................................... 29 Table 4: Sampling size needed vs. completed interviews by commodity and region ............... 32 Table 5: Sample size for FGDs by commodity and region ............................................................. 34 Table 6: Profile of the respondents ..................................................................................................... 38 Table 7: Percentages of ethnic composition by commodity .......................................................... 38 Table 8: Farm characteristics ................................................................................................................ 39 Table 9: Producer size and type of farm leadership......................................................................... 39 Table 10: Determination of sampling weight values ........................................................................ 41 Table 11: Estimates on indicator S1-1 ................................................................................................. 42 Table 12: Yield per hectare ................................................................................................................... 42 Table 13: Estimates on indicator S1-2: Number of individuals who have applied new techniques or technologies as a result of USDA assistance .......................................................... 44 Table 14: Top three new practices applied by farmers as a result of USDA assistance ......... 44 Table 15: Main obstacles experienced during application of new techniques ............................ 47 Table 16: Number of private enterprises, CBOs that have applied improved technologies .. 48 Table 17: Number of Grantees and partners that have applied improved technologies ........ 49 Table 18: Farmer groups that have received in-kind grant ............................................................ 49 Table 19: Farmers with storage availability ........................................................................................ 50 Table 20: Estimates on indicator S1-13 .............................................................................................. 51 Table 21: Estimates on indicator S1-14 .............................................................................................. 52 Table 22: Number of direct beneficiaries as result of USDA assistance. .................................... 54 Table 23: Estimates on indicator S1-18 .............................................................................................. 55 Table 24: Custom Performance Indicators C1 – C8 ....................................................................... 55 Table 25: Summarized the discussion with cabbage farmer groups ............................................. 59 Table 26: Summarized discussion with cassava farmer groups ..................................................... 61 Table 25: Summarized the discussion with coffee farmer groups ................................................ 63 Table 28: Summarized the discussion with sacha inchi farmer groups ........................................ 65 Table 29: Sampling size for cabbage commodity .............................................................................. 78 Table 30: Sampling size for cassava commodity ................................................................................ 78 Table 31: Sampling size for coffee commodity .................................................................................. 78 Table 32: Sampling size for coffee commodity .................................................................................. 79 4 List of Acronyms and Abbreviations Abbreviation Full Term APB Agriculture Promotion Bank AVSF Agronomes et Vétérinaires Sans Frontières BBS Beneficiary-Based Survey BMO Business Membership Organization CI Custom Indicators CIAT The International Center for Tropical Agriculture CLEAN Creating Linkages for Expanded Agricultural Networks CMD Cassava Mosaic Disease CWBD Cassava Witches’ Broom Disease CQI Coffee Quality Institute DAC Development Assistance Committee DAFO District Agriculture and Forestry Office DB Direct Beneficiaries DoA Department of Agriculture FAO Food and Agriculture Organization of the United Nations FDI Foreign Direct Investment FFPr Food for Progress FG Farmer Group FGD Focus Group Discussion GAP Good Agricultural Practices GCDA Green Community Development Association GESI Gender Equality and Social Inclusion GOL Government of Laos HA Hectare HH Household ICS Internal Control Systems IMC Interim Management Committee Kg Kilogram KII Key Informant Interview Lao PDR Lao People’s Democratic Republic LCA Lao Cassava Association LOP Life of Project MoAF Ministry of Agriculture and Forestry MEL Monitoring, Evaluation and Learning MOE Margin of Error MoU Memorandum of Understanding MSL Mai Savanh Lao MT Metric Tonnes MTE Midterm Evaluation MTE-BBS Midterm Evaluation Beneficiary-Based Survey NAFRI National Agriculture and Forestry Research Institute 5 Abbreviation Full Term PAFO Provincial Agriculture and Forestry Office PIRS Performance Indicator Reference Sheet PMP Performance Monitoring Plan PPP Public Private Partnership PPS Probability Proportional to Size QR Code Quick Response Code SCA Specialty Coffee Association SE South East SI Standard Indicator SME Small and Medium-sized Enterprises SO Strategic Objective SPS Sanitary and Phytosanitary measures ToT Training of Trainers USD United States Dollars USDA United States Department of Agriculture USG United States Government 6 Executive Summary Background The United States Department of Agriculture (USDA) under the Food for Progress (FFPr) initiative awarded funds to Winrock International (Winrock) to implement the Creating Linkages for Expanded Agricultural Networks (CLEAN) Project in the Lao People’s Democratic Republic (PDR). This five-year project is designed to increase agricultural productivity and expand the trade of agricultural products in Lao PDR. CLEAN’s two strategic objectives of FFPr are the following: Strategic Objective (SO) 1: Increased agricultural productivity by building the capacity of producers in improved productivity and profitability; training producers in improved production techniques; post-harvesting handling, marketing, sanitary and phytosanitary practices; and providing grants for equipment and agriculture inputs. Strategic Objective (SO) 2: Expanded trade of agricultural products by developing both domestic and export market linkages, facilitating trade relationships, researching export opportunities; promoting a coordinated and strategic approach to build market share by associations and promoting food issues and requirements. The CLEAN project is being implemented in five provinces in Lao PDR – Champasak, Sekong and Salavan Provinces in the southern part of the country and Vientiane Capital and Vientiane Province in the central region of Lao PDR. Prior to this Midterm Evaluation (MTE), two baseline studies were conducted – the first in December 2018, which investigated the possibility of CLEAN focusing on five cash crops (cabbages, sacha inchi, black pepper, chili peppers, limes, oranges, and cardamom). Following further discussions with the Lao Department of Agriculture (DoA), the project partners decided to focus on sacha inchi and cabbages from the initial study and add coffee and cassava as the 3rd and 4th commodities. The second baseline study, which incorporated these new crops, was completed in December 2019. Evaluation, Purpose and Objectives The purpose of the midterm evaluation (MTE) is to collect information on the progress and results of the project through March 2021 and determine if adaptations are required to the project activity design, implementation, and targets. Please see Annex 6 for a full copy of the terms of reference. Methodology The MTE’s methodology used a mixed method approach with quantitative and qualitative data collection to assess CLEAN’s results and the attainment of indicator targets agreed upon in the Performance Monitoring Plan approved by USDA. Elixirs Sole Company Ltd. (Elixirs) conducted three phases of the MTE between February and April 2021. Phase 1 preparation of the study included a desk review of reference documents that was used to design the survey, complete sample size calculations, and develop the topic guides for key informant interviews (KIIs) and focus group discussions (FGDs). Once these tools were developed, with CLEAN’s feedback incorporated, then Elixirs engaged and trained the field enumerators, who collected the field-based data in Phase 2. Phase 3 was focused on final data cleaning, summarizing, and analyzing the data and report writing. The focus groups and KIIs were summarized through the lens of the semi-structured interviews. 7 Key Findings Activity 1: Facilitate increased access to regional and high-value domestic markets SI – 1: Number of hectares under improved techniques or technologies as a results of USDA assistance The target for this indicator is 43,165 hectares (HA) by the end of the project. Under the data collected through the Midterm Evaluation Beneficiary-Based Survey (MTE-BBS), the weighted total is 2,161.6 HA under improved techniques or technologies. More than half of this land area (61%) is currently farmed by cassava producers, followed by coffee and cabbage farmers. SI – 9: Value of new public and private sector investment leveraged as a result of USDA assistance (USD) The target value of this indicator is $2,640,000 by the end of the project. In their last semi￾annual report (October 2020-March 2021), the reported total of project investment is $51,133. SI – 11: Total increase in storage capacity (dry and cold storage) as a result of USDA assistance (in cubic meters) The target for this indicator by the end of the project is 1,250 cubic meters and the current life project output is 19 cubic meters, with reporting from11 coffee and cassava producers, who have a small amount of dry storage, which they constructed before the project’s interventions. None of the cabbage producers have cold storage, as this requires more investment by producers and companies, as well as electricity. However, without cold storage, cabbage will rot quickly. As a result, cabbage producers sell their cabbage crops as quickly as possible from communal collection areas, where they can dry and pack their cabbage into plastic bags, or immediately after cutting the cabbage to traders who directly from the cabbage fields. During project implementation, CLEAN’s partners, BB Agriculture and Trading and Sky Vision, signed contracts with cabbage farmers. These firms have cold storage facilities and through grant partnerships with CLEAN, they will be enlarging them. During the MTE, the Elixirs teams did not measure the size of the producer’s storage. It is noted in the Performance Indicator Reference Sheet (PIRS) that the storage facilities should be measured by CLEAN’s monitoring teams. In addition, the Elixirs team was not able to access the storage spaces as they were often interviewing farmers in their fields and not at the sites of the storage. SI-13: Value of Sales by project beneficiaries in USD The project’s target for value of sales by project beneficiaries is $10,000,000 and the total value noted in the recent semi-annual report1 is $11,998,103 cumulatively, which includes an increase from the cumulative reporting period through FY 2020, of $4,256,0482 . According to the MTE-BBS, the total weighted value of annual sales for the previous year amounted to US$7,530,851, because sales from firms and government enterprises were not included in the MTE-BBS calculations. Of the total sales reported, cassava producers generated almost half of the total weighted value of sales (46%), while the sales reported by sacha inchi farmers represent a minor share of the total sales value. Despite the adjustment made with the corresponding sampling weight, 8 this low proportion could be explained by the low number of farmers selling sacha inchi who could be contacted for interviews (22 were interviewed in the MTE-BBS, out of the total of 77 people interviewed). This impression was established during the MTE-BBS field interviews. SI- 14: The volume of commodities (metric tons) sold by project beneficiaries The target for this indicator is 49,804 metric tons (MT) and the current cumulative value from monitoring is 96,614 MT from the most recent semi-annual report (Oct 20 to Mar 21)3 , with the largest reported volume from sales of cassava (78%). In the MTE-BBS, the total weighted values of commodities sold during the previous year amounted to 41,585 MT because the data reporting from farmers for the previous year did not include value of commodities sold by firms and government enterprises. As an example, for the inclusion of firms and enterprises in the semi-annual report vs. the MTE-BBS provides the most up to date information to the semi-annual report (Oct. 2020 – Mar. 2021). It noted also that the McCullagh Coffee Chain shipped a container of 18 MT of specialty coffee to the US; this information is not included in the MTE-BBS. SI-17: Number of individuals benefiting directly as a result of USDA assistance The target for this indicator is 14,877 direct beneficiaries. This pool of beneficiaries includes partners who have produced commodities, attended trainings, became members of farmer groups as well as government officials and proprietors of the enterprises and grantees who received funds from CLEAN. The current cumulative total is 2,023 individuals. This slow growth in the number of farmers is due to both delays in implementation, as well as to procedures necessary to ensuring the quality of training. One reason for this delay is because each of the training modules requires that PAFO and DAFO trainers, who provide several sessions to the same farmers. This is the case for sessions such as technical training for improving agriculture methods and production, which is done separately from the GAP and management trainings. Activity 3: Strengthen production systems SI -2: Number of individuals who have applied new techniques or technologies as a result of USDA assistance and SI- 3: Number of individuals who have applied improved farm management practices: After the signing of the MoU in July 2019, the project’s activities and training commenced. As a starting point, under this activity, CLEAN worked with master trainers from the central government, who developed training materials and lesson plans for a PAFO and DAFO capacity building training session. Once these materials were developed and the government trainers were in place, PAFO and DAFO staff trained the farmers from CLEAN’s value chain. Through this process, out of 814 respondents, 692 (85%) of them affirmed that they attended a CLEAN project training and applied one of the techniques demonstrated on their farm. Out of CLEAN’s pool of trainees, the application rate of the technology differs from one production system to another as cabbage producers applied new techniques at a higher proportion (67.7%) when compared to coffee growers (47%). Across the four commodities, 258 out of 391 respondents (66%) affirmed that the training sessions they attended were useful and very useful. SI – 7: Number of private enterprises, organizations and community-based organizations that applied improved techniques and technologies as a result of USDA assistance 9 With a target of 178 private enterprises, organizations and community-based organizations using applied techniques and technologies as a result of USDA assistance, the project has now achieved the registration of 81 farmer groups with the government and these are now working towards GAP certification. In addition to this registration, the Pak Kouang Cassava Farmer Group has successfully applied and received an in-kind grant of technical assistance, farming tools and supplies to improve their production of dried cassava. At the time of the MTE, CLEAN staff were planning to improve the capacity of other farmer groups to apply for in-kind grants (through proposal writing, developing business plans, etc.), such as cassava root chippers or irrigation pumps for planting crops in dry season fields. The cassava chippers are particularly useful, according to the most recent semi-annual report (October 2020 to 31 March 2021) in the cassava drying process as farmers receive higher prices for cassava which has been sliced and dried. SI – 16 Number of individuals who have received short-term agricultural sector productivity or food security training as a result of USDA assistance: The target for this indicator is14,877 farmers receiving training by the end of the project (SI￾1). According to the routine project monitoring, a cumulative total of 1,983 individuals attended training sessions. However, according to the sample size of the MTE-BBS of 814 people interviewed, 391 attended short-term training which reflects the 57% of participants interviewed. SI -18 Number of individuals benefiting indirectly as a result of USDA assistance This number reflects the number of family members, not counting the lead farmer, who benefited from increased knowledge and productivity as a result the project. From the MTE￾BBS, a weighted total number of 7,931 people benefited. In FGDs, farmers noted that the indirect benefits of the increased income and productivity included increased investment in farming, education benefits, and/or home repairs. Activity 7: Grants to incentivize innovation and investment SI – 8 Number of public-private partnerships formed as a result of USDA assistance CLEAN’s target for its grant-related indicator is 12 public private partnerships (PPPs) established by the end of the project. The project had previously recorded no PPPs. However, during the most recent semi-annual period, two PPPs are now in the process of being established. This indicates that there is an increasing ability for the project to establish relationships between the public and private sector, though these will require time to establish. SI – 12 Number of policies, regulations, and/or administrative procedures in each of the following stages of development as a result of USDA assistance There is a target of three policies, and these are being developed through the close working relationships between CLEAN and the Department of Agriculture (DoA). These policies are the1) Guidelines for the Management of Agribusinesses, 2) an amended version of the Law on Plant Protection and Quarantine and 3) International Standards for Phytosanitary Measures4 Project implementation procedures have been greatly improved with the initiation of regular Interim Management Committee (IMC) meetings. According to the Director General of DoA, the purpose of these meetings is to review past activities and lessons learned, discuss how to 10 improve implementation plans and plan for the next time period. Since these meetings started, coordination and communication has improved. Assessment of Project Progress CLEAN has encountered many challenges from the beginning of implementation. Before the project started full implementation of activities, it spent time on preparation and implementation, as well as establishing good working relationships with partners. Once the partnerships were in place, the project developed training materials. These training materials were used for training sessions for both the province and district trainers, together with inspectors to ensure the mutual understanding for the foundation of the project. This meant that once the project was authorized to begin, the materials and human resources were already in place. In addition to delays in the signing of the project’s MoU by the Government of Laos, additional challenges have also delayed project progress. These challenges included flooding in target areas in July 2019 and the COVID-19 travel restrictions in March through to May 2020, which limited on-site training activities. In response to these delays, CLEAN focused on developing an implementation model that integrates the quantity and quality of farm production with access to and an understanding of markets. CLEAN has provided a strong model for the growth of four commodities and for partnerships between government, private partners, and farmers. There are definite signs of success including:  Through CLEAN’s training, farmers understand that they can make positive changes to their land so that they do not have to continually open new land but can learn ways to ensure sustained productivity through soil preparation, particularly through the use of organic methods, and soil conservation so that soil fertility is preserved.  Farmers are learning new methods to increase quality as well as quantity of products and are learning about the value of increasing quality to improve sales and potentially to achieve higher prices and markets for their products.  A cascade method of training has been used, with training the trainer sessions included to enable people to pass their knowledge on. In this way, DoA students have gone on to train PAFO and DAFO staff, who then teach the farmers. As a result of this process), 93 government officers have been trained in order to train others in quality standards and food safety (Custom Indicator 4).  Government partners in agriculture have learned to work with farmers to understand GAP and ICS and this leads to an improvement in product quality. This in turn opens up high-end market options.  Farmer groups are being formed and they have started developing methods of working together. The formation of these groups, however, does not result in overnight change as farmers are accustomed to working with middlemen and vendors who come to the villages. However, with the formation of these groups, farmers are starting to understand the benefits, as they can provide mutual support, as well as capacity development (business planning, accounting, support of improved agriculture practices), support each other in the group in crop management, procurement of equipment and seeds, group sales and transportation. Additionally, and most importantly, these groups can provide the farmers with the bargaining power to negotiate prices. 11  Farmers and government partners are establishing relationships with private enterprises and grantees for improved practices, such as the relationship between sacha inchi farmers and Mai Savanh Laos.  However, as noted above, the growth in reaching large numbers of farmers has been slow due to both implementation challenges, a shortened implementation timeline and the need to conduct several training sessions to participating farmers. It’s not likely that the project will be able to reach its target of 14,877 farmers by the end of the project period in 2021. The project’s indicator progress, from CLEAN project reports, is incorporated in Error! Reference source not found.. Conclusions and Recommendations The following conclusions and recommendations are based on DAC criteria and the questions posed in the terms of reference 1. Relevance: How accurate have the critical assumptions underpinning the project proven to be in practice? Is the Theory of Change still appropriate for the project goals and objectives?  The critical assumptions of the project are that government partners and farmers have the interest to learn the skills to increase extent of improvements to small-holder production systems. Based on the feedback provided by the farmers who have joined training sessions, they were interested in learning about organic farming methods and GAP standards and were interested in joining farmer groups to improve the management of their farms and technical capacity in farming.  For government partners, their understanding of market integration, trade, and production methods has increased throughout the project period. In addition, their understanding of the relationships between production, markets and food safety compliance has also evolved from the start of the project.  On the production side, farmers who employ organic methods of fertilizer and appropriate chemical pest control have indicated that they feel these methods are safer for both the consumer and their own families. Are there previously unanticipated opportunities or challenges that may require revision of the project strategy or redirection of project resources?  The project has developed a management system in conjunction with DOA representatives. This system has provided a wider core group of people who understand the project and the integration of production and marketing skills. However, groups involved in the private sector, outside of the GoL, are not actively included at this point in time.  While CLEAN has been working directly with the DoA and government partners at province and district levels, and from there, with the farmers and farmers groups, there might be more opportunities of including INGOs and non-profit associations (NPAs) in networking.  There were several major challenges after the delay in the MoU signing, which included floods in the rainy season of 2019 and the COVID-travel restrictions as the project was starting to ramp up in 2020. As a result, CLEAN and government partners had difficulty accessing villages and individual farmers for training. However, the travel restrictions increased project reliance on the farmer groups to monitor the quality of their products 12 through technical and GAP trainings, as well as exploring the coordination of farmer groups through WhatsApp and other methods of social media for those farmers who can access 3G or 4G services through their cell phones.  The delayed start of the project and unexpected events slowed project progress and the number of farmers trained and included in project activities fell a long way short of the anticipated target. With the current total number of 2,023 beneficiaries (1,918 farmers, 93 government partners, six private enterprise staff and six grantee staff) reached by the project, project implementation, with the target of reaching 14,877 direct beneficiaries, has been slow but may pick up momentum, especially with a longer period for implementation. Are Monitoring & Evaluation and Learning (MEL) tools and tracking systems working well and identifying relevant project impacts? Monitoring is being done through government partners, trainers, and CLEAN staff, using paper monitoring forms. CLEAN has piloted the use of ID cards and QR codes with Mai Savanh Lao’s farmers.5 The project intends to roll out these cards so that monitoring of farmers groups can be done through electronic means in the future. Paper-based systems have the advantage of allowing the double checking of data; however, use of the data requires staff time to enter the data from the paper files. The project does have a Performance Indicator Reference Sheet (PIRS) which defined the indicators, and how the indicators should be measured, and annual targets for each indicator. The use of the data for learning is still being developed through the establishment of regular IMC meetings for reviewing project activities and the discussion of the results. Recommendations:  Improving the perception of the importance and application of the project interventions so that they are more relevant to the lives of farmers should involve both interventions directed towards farmers and governance structure of the villages.  At the village level, including village leadership in decision making and enlisting the services of the village administration would increase the productivity of activities and promote the project’s training activities.  Expanding governance within the village would also lend support to farmers groups which would enable them to increase their bargaining power with enterprises and especially traders, as farmers would not undercut the prices that the farmers groups had established.  Increase cohesiveness in farmers groups by encouraging farmers to be involved as co￾facilitators in training, and for farmers with higher levels of education to take an active role.  The use of a mobile app like KoBo would help to decentralize data collection to the district and village levels and make the information more quickly accessible to the stakeholders so that they can use the data. 2. Effectiveness Training  The cascade method of training (i.e. training of trainers) has provided training at each level of the agriculture sector. It has created 93 trainers who have improved their own skills in 5 The use of IDs and QR codes was noted in the semi-annual report (October 2019 to March 2020, page 24) as a method to streamline monitoring information and collect the personal details of members of farmer’s groups. 13 agricultural technical knowledge, marketing and as trainers. According to MTE-BBS, of the 692 farmers who received training, over half (391) have been able to apply the new skills and knowledge to their production.  The DoA developed picture-based training materials with simple Lao text, which farmers can use as reference materials. This has included handbooks and posters. For cassava producers, the WhatsApp group helps to build linkages between farmers and trainers. Market access  Sacha Inchi: Within the lens of a market systems approach, the example of MSL in sacha inchi cultivation and processing of sacha inchi products serves as a model of how the market and production-level stakeholders can work together.  Cassava: The project has provided cassava chippers to help one farmers group to increase their market access through the sales of sliced and dried cassava, which bring better prices to the farmers as well as being more in demand by companies. CLEAN has also funded the training has helped farmers to improve the quality of the cassava they grow through increased awareness of integrated pest management.  Coffee: The project has been promoting Lao coffee through various events and demonstrating to farmers that they can grow and meet the demand for specialty coffee. The project used an innovative approach for a cupping event during COVID-19 travel restrictions through a virtual cupping event in collaboration with the Coffee Quality Institute (CQI).6  Cabbage: This household staple product has a high level of demand and FGs have started to produce GAP-certified cabbage. CLEAN has been developing a public private partnership (PPP) with BBA Company and Sky Vision for improving the cold storage of cabbage. Factors inhibiting the effectiveness:  Some farmers who attended training sessions have not yet set up FGs; the project coordinator attributes this to PAFO and DAFO not having the capacity to follow-up on the activities of the farmers. As a result, there is no information on the problems that farmers have in setting up FGs.  Currently, the need for multiple training sessions for current farmers slows the growth to reach out to new farmers. In addition, there are only a limited number of trainers who also have other responsibilities in their government positions.  Lack of cold storage for cabbage farmers limits their ability to sell their cabbage at the optimum time for higher prices; whereas coffee, cassava and sacha inchi farmers are able to store their products in dry storage areas and wait for the ideal time.  In 2020, the training sessions were delayed due to COVID-19 related travel restrictions which reduced access to villages. As a result, the training in improved farming practices took place after the restrictions were lifted, which was during the dry season, after the 2020 harvest.  Although this was not assessed in the MTE-BBS, the education level, in particular literacy and numeracy skills, may affect the farmers’ ability to access and understand training materials, keep records and calculate the weight and sales of their products. 14 Recommendations:  Decentralize activities by increasing capacity at village and district levels.  At the village level, train the leaders of the farmer’s groups who have the necessary literacy and numeracy skills to assist farmers with less skills in these areas.  Increase regularity of farmers group meetings with specific agendas. These meetings can also be helpful to review training sessions to refresh skills. This will be especially important as farmers who received training in some techniques after harvesting their might not remember what they had learned for soil preparation and planting.  Develop a knowledge repository that farmers can access online. The knowledge repository would include both the written and audio-visual materials that farmers can download to their phones. The WhatsApp groups can inform farmers when new content is available.  Schedule exposure trips to established farmer groups for fledgling farmer groups so the newer groups can see the benefits, together with a model for setting up a similar group.  For future trainings and activities with farmers, it would be beneficial if PAFO and DAFO made an agriculture calendar/map to understand when farmers are planting and harvesting.  DAFO should plan more monitoring visits after training so that if villagers have problems with using the skills and knowledge, the DAFO staff can provide assistance. In addition to monitoring, DAFO, PAFO and CLEAN staff should develop the skills of farmers to share their knowledge, both formally and informally with other farmers.  There are some non-profit associations which have been working with farmers groups and in developing local markets and collection points. Visiting groups such as Green Community Development Association (GCDA) and Agronomes et Vétérinaires Sans Frontières (AVSF) may provide some lesson learned which might be useful for CLEAN. 3. Efficiency Are intended beneficiaries receiving services as planned? What are their experiences and recommendations? The roll out of training to farmers has been slow, as it requires that government partners improve their capacities as trainers and it takes time for trainers to feel comfortable with training. Training of farmers has been delayed due to other environmental and outside factors such as COVID-19 travel restrictions and floods. However, as mentioned above, training trainers at each level, including peer support at the local level, increases efficiency as the training sessions can be conducted more closely to the village and farmer level. Farmers can develop personal familiarity with the trainers and extension services so that they can consult them when they have questions. To what extent is the project being implemented as designed? The project model seems to be working as described in the theory of change. Agriculture staff and enterprises involved in the project have demonstrated an improved knowledge about the relationship between the market and production. Farmers have not been able to access traditional methods of sales, through middlemen during the COVID-19 travel restriction period, as farmers could not leave the villages and traders from outside the village could not enter the village. International trade has been slowed 15 because of border closures, which affects both access to supplies and equipment but also country transit time. Is the project on track to meet with targets? If targets are not being met, why not? If targets are being exceeded, why? Do targets need to be adjusted? As described above, the project is not on track to meet targets. However, CLEAN may be able to leverage existing activities and results by working to strengthen farmer group membership and formalize more groups, who will then be able to access further training and technical assistance. Recommendations:  Farmers recommend more training during the time period before planting and that DAFO should conduct more follow-up; this will be more important to refresh the farmers’ knowledge before the subsequent growing season of their product.  If farmer’s groups become more effective, it is possible that the DAFO can do some ‘task shifting’ in the form of assigning some of the tasks of monitoring, training particular modules and ICS monitoring to the FGs. 4. Sustainability Does the original strategy for the project’s sustainability and lasting impact remain feasible? The original model of the project still holds and it is important to strengthen the model. Agriculture staff at all levels are interested, involved and enthusiastic as they have seen several models developed with good results. However, the input of private sector actors would help to make the market activities more sustainable. If not, why? Are additional efforts needed, or are other changes to the strategy recommended? Although farmers in FGDs believe that they can train others, they also should have additional training, using an activity-based approach, so they can advise other farmers, as noted in the recommendations above. PAFO and DAFO should also train and monitor farmers and assess their skills as peer advisors. Recommendations:  Include action plans with training, to follow a format of: o 1. What did the training accomplish? o 2. How do farmers plan to apply what they have learned? Farmers could review this as a final session of their training where they can work together as a group to develop a business plan, calculate what funds they need, and exact what steps they need to take.  Include training in peer to peer education so that farmers can tell others about what they have learned and what improvements they are making. 5. Impact How has CLEAN’s interventions affected targeted beneficiary groups? Are the project’s interventions having the expected effect on identified constraints? Farmers, who received training, have been able to improve their understanding of skills required for business development, although the practice of working as an FG still needs strengthening in order to understand the benefits of FGs in establishing and maintaining market relationships. Recommendations: 16  The strengthening of farmer groups has the potential for improving impact as they will take more ownership and also contribute to sustainability.  Continue development of FGs with consideration for how the farmers can help each other within the groups, such as reviewing lessons and helping with recording, business planning and the calculation of volume produced and sales.  Currently the project does not include social assessments or interventions. In future project planning, it would be useful to add social considerations, such as gender assessment (such as Gender Equality and Social Inclusion (GESI) assessment) to include gender as part of strategy and interventions. 6. Coherence Coherence is a more recent criterion added to the other five components of the DAC criteria, which defines how a project fits in with local and government strategies, policies and plans of action. The project deals with the policy objectives of improving markets, leveraging enterprise support, and impacting policy. As mentioned above, three policies are in the process of revision through work with DoA. Recommendations:  As policies are the framework for change at central and local levels, CLEAN should continue working with the DoA to develop agriculture and farmer’s group policies.  CLEAN has taken a lead role in developing a model on how public private partnerships can be put into action. There is currently a PPP policy drafted by the government of Laos in Lao language and is in the process of full approval; this would be a useful policy for providing structure to the continuation and expansion of the project marketing goals. 1.Introduction and Purpose 1.1. Project description The CLEAN project is a five-year project implemented in partnership with the Department of Agriculture and Forestry (DAFO), and the Ministry of Agriculture and Forestry (MoAF), Lao PDR. With an award ceiling of $10.5 million, the CLEAN project is funded by the Food for Progress Program of the United States Department of Agriculture (USDA). Winrock International (Winrock) started implementation of the CLEAN project in September 2017, with an end date of December 2022. The CLEAN project is being implemented in five provinces of Laos: Champasak, Sekong and Salavan Provinces in the southern region and Vientiane Capital and Vientiane Province in the central region of Laos. The CLEAN project aims to conduct principal activities with outputs and outcomes contributing to increasing quality and quantity of agricultural commodities produced in Lao PDR for both export and consumption by consumers within Lao PDR. 1) FFPr SO1: increase the productivity of horticultural products through 1. training in Good Agriculture Practices (GAP) standards 2. strengthening plant protection services 3. improving the functionality of quality compliance and certification systems 2) FFPr SO2: expand domestic and regional trade through 1. integration of Lao horticultural products into regional market networks 2. improved branding of Laotian horticultural products 17 3. increasing producer adoption of GAP standards. CLEAN’s activities focus on levels of production and trade, from farmers/producers, agro￾enterprises, entrepreneurs, and traders in the geographic areas of the Bolaven Plateau and the Vientiane Plain. The project’s foundation depends on an expanding middle-class who are interested in food quality and safety and willing to pay for high-quality products, as well as to increase Lao domestic demand. The market infrastructure for exports will be improved by improving the efficiency of the markets to stimulate the growth of the domestic market. This will create a suitable environment for private investment, supply chain management to reduce transaction costs, reduction of post-harvest losses, and development of market linkages. CLEAN’s theory of change (Figure 1) is: If market systems in high-value horticulture are established and strengthened; and if producers have access to appropriate use of technology and investment, and the private sector is leveraged, then smallholder farmers and businesses in Lao PDR will increase their participation in domestic and regional trade. The approach is built on three distinct, but interrelated pillars: (1) smallholder production; (2) quality compliance, certification, and plant protection; and (3) market linkages and infrastructure. The CLEAN project will focus on high-potential districts in the provinces of the Bolaven Plateau, and the Vientiane Plain, as were prioritized with in-country national and provincial government authorities, donors, and experts. There are three districts in the highlands of the Bolaven Plateau: Paksong, Laongam and Thateng. In Vientiane Capital and Vientiane Province in the central plain area of the country, there are five districts, which are Sangthong, Xaythany, Pakngum, Vangvieng and Kasy. The intended targeted beneficiaries are smallholder farmers, farmer organizations (FO), cooperatives, traders, processors, local, national, and regional buyers, trade associations, equipment suppliers, financial institutions, consumers, and government staff. CLEAN plans to impact 14,877 direct beneficiaries and 59,508 indirect beneficiaries over the course of the five-year project term. Figure 1: CLEAN’s Theory of Change7 7 Source: Lao PDR: CLEAN Monitoring, Evaluation, and Learning Plan, Version January 12, 2018, page 33. 18 Table 1: Expected Results and Key Activities8 Result Expected Result Key Activities SO1: Increased Agricultural Productivity Increased Use of Improved Agricultural Technologies and Techniques (FFPr 1.2) 1. Strengthen Production Systems—through identifying FO for CLEAN project participation; strengthening FO management and technical production capacity; strengthening the capacity of the DoA, PAFO and DAFO; increasing private sector extension providers and embedded services; increasing financial literacy and access to finance. 2. Launch Grant Program to incentivize innovation and investment—through developing LIFT implementation plan; launching small activities fund for farmer group organizations. Increased Capacity of Government Institutions (FFPr 1.4.1) 3. Strengthen Food Safety Systems—through supporting Government of Laos (GOL) institutions on Sanitary and Phytosanitary Standards (SPS) for export; strengthening network of plant health centers and clinics; technical assistance on integrated pest management and pesticide risk reduction. 8 Source: Lao PDR: CLEAN Monitoring, Evaluation, and Learning Plan, Version January 12, 2018, page 5 and 6 19 Result Expected Result Key Activities SO2: Expanded Trade of Agricultural Products (Domestic, Regional, and International) Increased Value￾Added to Post￾Production Agricultural Products (FFPr 2.1) 4. Facilitate the Adoption of Quality Standards and Certifications—through benchmarking and operationalizing Good Agricultural Practices (GAP); introducing a participatory guarantee system certification scheme for domestic market and export; creating a fast track for required documentation for exports. 5. Develop links to Regional Agricultural Innovation— through creating a national community of innovation linked to regional innovation platforms; launching a regional agricultural innovation challenge; testing, scaling, and commercializing new technologies. Increased Access to Markets to Sell Agricultural Products (FFPr 2.2) Improved Transaction Efficiency (FFPr 2.3) 6. Facilitate Increased Access to Regional and High￾Value Domestic Markets—through developing a market-led value chain upgrading plan; developing a regional market network; providing technical support; building on existing trade linkage models; developing buyer/seller capacity for contracting; encouraging linkages to a project communications platform; identifying mechanisms for market information. 7. Increase Domestic and Regional Consumer Awareness of Clean and Safe Agriculture—through regional export marketing and branding of Lao PDR as a source of clean and safe produce; geographic indicator branding; developing a domestic marketing campaign integrating nutrition messages; improving access and visibility of clean/safe horticulture in domestic markets; seeking innovative ways to leverage support for continued marketing. 20 Figure 2: CLEAN’s Approved Results Framework9 9 Source: Lao PDR: CLEAN Monitoring, Evaluation, and Learning Plan, Version January 12, 2018, page 32 21 1.2. Progress of the program At the time of the Midterm Evaluation (MTE), the project had gone through full implementation for 18 months (October 2019 to March 2021). The CLEAN project was awarded to Winrock International in September 2017. However, in Lao PDR, the registration process for new NGOs can be a lengthy process. As a result, the Memorandum of Understanding (MoU) with the GoL was not signed until July 2019, after which, the program immediately started full implementation in December 2019. Activities are already achieving results even though the project did meet with some challenges after 2019, including flooding in 2019 which affected the commodity production in those areas and travel restrictions in March through May 2020, which affected access to project sites due to COVID-19 movement restrictions. During the preparation of the MoU, to overcome the impact of the delays in implementation, the CLEAN team worked with the DoA under the Ministry of Agriculture and Forestry (MoAF) to obtain authorization to conduct some activities utilizing USDA funds. These activities included the review of the initial baseline survey in 201810 to identify the target commodities and organized meetings with entrepreneurs, traders in markets, farmers, and representatives from government, USG and USDA to assess the viability of commodities and those which would be targeted for project implementation. With the DoA, CLEAN staff revised the project workplan and reviewed the list of commodities. The first baseline report included a wide range of commodities but after consultations and looking at village production, the project and stakeholders agreed on the four commodities of coffee, cassava, cabbages and sacha inchi using the criteria of:  At least 50 households (HH) producing that commodity in a specified location  These HH earn at least 20 million kip per year from horticulture  Agriculture provides at least 60% of total HH income  And the HH spends at least three months per year producing agriculture products. a) The benefits of each of the commodities were explored. Coffee has a short value chain, where buyers would go to villages and then sell directly to roasters, or even the roasters themselves would buy from villages so prices don’t increase with each of the middlemen. The 2019 survey, validated by USDA in 2020, indicated that coffee producers had problems with pests and productivity, and farmers tended to grow the Catimor variety of coffee which produces a lower quality coffee bean in greater quantities, but the bush and cherries are also prone to insect infestation. The CLEAN project aims to increase the quality specialty coffees through improving the quality of the coffee cherries, increasing productivity and post-harvest handling along with increasing use of organic fertilizer through better composting and soil preparation methods. The project also plans to increase farm management, management of farmers groups, and ensuring that coffee beans will meet the Specialty Coffee Association (SCA) standards, in order to open international and domestic markets for specialty coffees. Cassava is a crop, noted in the Government of Laos’ 8th Five Year Plan, that has a high potential for growth and profit, with exports to Vietnam and Thailand of 90 million USD in 10 The 1st baseline survey was conducted in September 2018 and finalized in December 2018, the targeted crops were cabbage, black pepper, sacha inchi, citrus, and cardamon. The second baseline evaluation, conducted in October/November 2019 for cassava and coffee, finalized in 2020. 10 22 2018. CLEAN has helped farmers groups to organize so they can learn about improved methods of cassava cultivation Cassava Disease Control: There are also obstacles with plant diseases such as Cassava Witches’ Broom Disease (CWBD) and Cassava Mosaic Disease (CMD) which kill plants and reduces production. Because of the threats of exporting diseased plants along with healthy ones, Thailand increased quality checks of cassava imported from Lao PDR to ensure that cassava follows international quality agreements. CLEAN targeted increasing cassava production management, post-harvest practices and strengthening the linkages between research organizations (NAFRI, CIAT and the Lao Cassava Association (LCA)) and farmers. Currently, the only method of controlling cassava diseases is for farmers to identify and burn diseased plants and replace those plants with fresh cut stems. NAFRI, CIAT, Winrock and the LCA are collaborating on a rapid fresh stems multiplication activity called Future Stems, which currently has a demonstration center in Vientiane Capital and is building a ‘Tunneling Greenhouse’ in the south for providing an increased number of clean stems to meet farmers’ needs.11 In December 2019, CLEAN organized an event for “Enhancing Productivity, quality and market share for cassava production which included participants in every step of the value chain to meet each other and share their experiences in production, post-harvest and market challenges. Some of the issues raised included the reliance on cassava seed from Vietnam and Thailand which increases the risk of spreading disease, financing for farmers, processing technology and sharing ideas on innovation of processing methods. The project helped farmers to set up a WhatsApp group (CLEAN GAP-Cassava) to increase access to information. A planned conference on “Future stems: the Lao Cassava Seed System” had to be cancelled due to COVID-19 related regulations on no large meetings; however, some participants were able to visit NAFRI, CIAT and the Cassava Research Center which increased the potential for a stronger partnership between these three centers and the Lao Cassava Association. Cabbage: Chinese and White Cabbage are grown throughout Lao PDR but the higher valued White Cabbage is grown mainly in the cooler environments of Paksong, Vangvieng and Kasy Districts. At the time of the first baseline study, the export of cabbages didn’t meet GAP standards required by Thailand. At the time of the original baseline study in 2018, cabbage made up 2/3 of the sales of all commodities; however, the price of cabbages can fluctuate because of seasonal supply. The first GAP certified farmers group was the Ban Huayseth Farmers Group which had received certification after three years; this gave CLEAN a model for helping to reduce the certification time so that cabbages could be exported, through clear training materials on GAP, business planning and record keeping as well as a checklist for requirements so that farmers can have clear guidelines on the steps to follow for certification such as soil sampling, access to water without chemical residues, and packing. Sacha Inchi: This nut, its products and oils are in high demand throughout Asia. CLEAN has supported the Mai Savanh Lao company, though an innovation grant,12 which produces powder, oil, roasted nuts, and nut butter, to train farmers to grow the vine using organic methods. The processed products have been able to meet the standards for organic and Halal certification which has increased their ability to export their products to markets in Asia, the 11 Information available at: https://alliancebioversityciat.org/news_and_blogs/cassava-program-in-asia-brings-together-stakeholders-in-lao￾pdr-for-2-days-of-training-and-the-official-launch-of-the-countrys-first-cassava-clean-stem-multiplication-facility/?fbclid=IwAR1AOk￾TeSMMIfgSvhuKO4WdhHkZIsCKUaj0pvaA-mOHRp9ErpzZYFcEq3M 12 In the semi-annual report of Oct 2019 to 31 March 2020, the project was starting to provide investment grants (scaling up interventions by established enterprises) and innovation grants to extend capacity of enterprises to expand and reach out to new markets. 23 US and Europe. The company trained 137 farmers to grow the nuts, do initial processing of drying the seed cases and removing the nuts and storage, whereupon the company buys the nuts at a pre-determined rate which is satisfactory to the farmers. The demand for sacha inchi has increased and with the increasing demand and sales, Mai Savanh Lao hopes to increase its pool of suppliers to 2,000 farmers, who will benefit from MSL’s increased sales. In addition to training, CLEAN provided processing equipment such as an oil press, which produces both the high demand oil as well as the dried powder, and a nut roaster. The company also produces a nutritious energy bar both for sale and as a food supplement for children in remote areas as part of the company’s corporate social responsibility policy.13 b) Good Agriculture Practices (GAP) adoption by farmers groups Instituting GAP is important for integrating the Lao domestic economy into regional and international systems to increase trade by guaranteeing adherence to international standards. This increases the marketability of the smallholder farmers’ products to a wider consumer audience. CLEAN, the DoA, PAFO and DAFO staff, led by members of the Lao Clean Agriculture Standards Center (CASC), have collaborated on developing materials for farmers to understand GAP systems and requirements for safe and sustainable food production for consumers. The documents produced for the pilot included the definition of GAP, the importance of reducing exposure to agricultural chemicals, increasing market access, and the steps necessary for acquiring GAP certification. In total, 94 GoL representatives attended the different TOT courses supported by CLEAN. After the training of trainers (TOT), the partners instituted a cascade training system where a core group of trainers and resource people were available to train PAFO and DAFO staff to train the farmers. The trainers learned facilitation methods, understanding technical knowledge related to the risk of agricultural chemicals, internal control systems (ICS), farm management, post-harvest practices and storage. These sessions also strengthened PAFO and DAFO coordination. These trainings encompassed three of the four project commodities. The sacha inchi training for farmers was done by MSL. The GAP process is presented in Annex 3. GAP Certification Process. c) Farmers groups and in-kind grants As of the writing of this report, 81 farmers groups have been established and registered (Annex 7). This has opened the potential for these groups to increase domestic and international trade through achieving GAP certification. In March 2021, CLEAN staff trained them in planning and making applications for in-kind support which would improve productivity and post-harvest processing. One group has received in-kind support and an additional 19 groups made applications for in-kind grants in April 2021 which included such items of equipment as cassava chipping machines and water pumps. The plan is to further train those in the use of the equipment and then to provide it by July 2021.14 d) Improved trade of agricultural products The investment and innovation grants have the purpose of promoting enterprises and farmers groups to improve value chains through small grants. This removes the initial risk of investment so that the grantee can try new production methods, processing of products, and marketing practices. 13 Information from Interview with Dr. Phillipe Schmidt, CEO of Mai Savanh Lao 14 KII interview with Ms. Soudachan, CLEAN Senior Grants Manager 24 In addition to MSL, noted above, grantees include improvement in coffee production and marketing. The Coffee Quality Institute (CQI) supported an in-country coffee assessment which found that there is high interest in Lao coffee if the produce can meet their specifications. While farmers do know some processing methods, such as washing and processing Arabica coffee, there still are only a few Lao certified evaluators of coffee quality and that even international firms do not apply international standards for the quality and processing of coffee. With CQI, CLEAN set up a virtual cupping event during the period of COVID-19 travel restrictions; the cupping event was arranged by pre-packaged beans, standard water quantities and brewing instructions with the event taking place through a virtual meeting. This allowed the 47 event participants to try Lao coffee and consider buying and marketing it in the future. The production and export of Lao coffee by the Jing Jhai Cooperative has increased the awareness which has resulted in an increase in the quantity and price of the coffee being exported. This translates to better prices for smallholders. CLEAN established a link with Shopping-D to expand market access to items produced by CLEAN farmers through the Shopping-D online sales site. Since this started during the COVID-19 travel restrictions in Vientiane, the delivery service provided consumers with fresh products which included sacha inchi products, coffee, and GAP-certified cabbages. 1.3. Overall project management Over the life of the project, the coordination and communication between partners has improved. In December 2020, the Interim Management Committee (IMC) had its first meeting to discuss technical and administrative successes and challenges. These meetings have help agriculture leaders and staff to understand the importance of marketing and its relationship with crop production, as farmers will plant and produce what they can sell. The project progress, based on the actual counts from semi-annual reports, towards meeting the indicators is detailed in Table 2. 25 Table 2: CLEAN's performance indicators by routine reporting system results Results Indicator Title Standard (SI) Indicator Type Unit of Measure LOP Target Result 4 - 9/2019 Result 10/ 2019 – 3/ 2020 (FY 2020) Result 4/2020 – 9/2020 Result 10/ 20 – 3/ 2021 Cumulative Result Improved knowledge regarding farm management Number of individuals who have applied improved farm management practices (i.e., governance, administration, or financial management) as a result of USDA assistance SI-3 Outcome Number 10,414 0 0 65 139 204 Number of private enterprises, producers’ organizations, water users' associations, women's groups, trade & business associations & community-based organizations (CBO) that applied improved SI-7 Outcome Number 178 0 0 215 81 81 15 The chart in the Semi-Annual report 10/20 – 3/2021 noted that previous results did not include farmers groups. However, the cumulative result now reflects these groups. 26 Results Indicator Title Standard (SI) Indicator Type Unit of Measure LOP Target Result 4 - 9/2019 Result 10/ 2019 – 3/ 2020 (FY 2020) Result 4/2020 – 9/2020 Result 10/ 20 – 3/ 2021 Cumulative Result techniques and technologies as a result of USDA assistance Improved market and trade infrastructure Number of public￾private partnerships formed as a result of USDA assistance SI-8 Outcome Number 12 0 0 0 2 2 Improved linkages between buyers and sellers Number of policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance SI-12 Output Number 3 0 0 0 3 3 Value of public and private sector investments leveraged as a result of USDA assistance SI-9 Outcome US dollars $2,640,000 0 0 0 $51,133 $51,133 Value of sales by project beneficiaries SI-13 Outcome US dollars $10,000,000 $30,000 $84,300 This refers to prior cumulative results 4,256,048 $7,742,055 $11,998,103 27 Results Indicator Title Standard (SI) Indicator Type Unit of Measure LOP Target Result 4 - 9/2019 Result 10/ 2019 – 3/ 2020 (FY 2020) Result 4/2020 – 9/2020 Result 10/ 20 – 3/ 2021 Cumulative Result Increased use of improved post￾production processing and handling practices Volume of commodities (metric tons) sold by project beneficiaries SI-14 Outcome Metric tons 49,804 3.3 3.91 This refers to prior cumulative results 35,278 61,335.00 96,613 Total increase in installed storage capacity (dry or cold storage) as result of USDA assistance SI-11 Output Cubic meters 1,250 0 0 0 1,094 1,094 Increased knowledge by farmers of improved agricultural techniques and technologies Number of hectares under improved techniques or technologies as a result of USDA assistance SI-1 Outcome Number 43,165 0 0 333 1,825 2,158 Number of individuals benefitting directly as a result of USDA assistance SI-17 Output Number 14,877 33 81 1,926 331 2,023 28 Results Indicator Title Standard (SI) Indicator Type Unit of Measure LOP Target Result 4 - 9/2019 Result 10/ 2019 – 3/ 2020 (FY 2020) Result 4/2020 – 9/2020 Result 10/ 20 – 3/ 2021 Cumulative Result Number of individuals benefitting indirectly as a result of USDA assistance SI-18 Output Number 178,524 165 405 10,030 1,365 8,133 Number of individuals who have applied new techniques or technologies as a result of USDA assistance SI-2 Outcome Number 8,926 0 0 236 904 1,140 Number of individuals who have received short-term agricultural sector productivity or food security training as a result of USDA assistance. SI-16 Output Number 14,877 0 81 1,920 165 1,983 Source: Authors summarized from the CLEAN semi-annual reports Apr. 2019 – Mar. 2021 29 1.4. Purpose of the evaluation The purpose of this midterm evaluation (MTE) is: ● Quantitative and qualitative data collection to assess the progress towards the program results and attainment of indicator targets agreed upon in the Program Monitoring Plan approved by USDA ● Identify changes in the project activity design, the process of implementation, targets, and project management The research questions design was based on the information collected and the analysis is based on the DAC criteria of relevance, effectiveness, efficiency, sustainability, impact, and coherence. 2. Evaluation Design and Methodology 2.1. Evaluation questions CLEAN’s MTE focused on the project’s implementation progress towards meeting its objectives and goals. The following questions are from the evaluation terms of reference Table 3: Evaluation questions and methodology DAC criteria Key questions Methodology Relevance  How accurate have the critical assumptions underpinning the project proven to be in practice? Is the Theory of Change still appropriate for the project goals and objectives?  Are there unanticipated opportunities or challenges that may require revisions to the project strategy or redirection of project resources?  Are Monitoring, Evaluation and Learning (MEL) tools and tracking systems working well and identifying relevance project impact?  Desk reviews and staff interviews  Interviews with project beneficiaries Effectiveness  What factors are facilitating the project’s effectiveness? What factors are hindering it?  What should be improved in order to make the project implementation more effective?  Document review  Key informant interviews (KIIs) and beneficiary focus groups Efficiency  Are intended beneficiaries receiving services as planned? What are their experiences and recommendations?  To what extent is the project being implemented as designed?  Is the project on track to meet with targets? If targets are not being met, why not? If targets are being exceeded, why? Do targets need to be adjusted?  Document review, beneficiary focus groups and key informant and staff interviews  Beneficiary-based survey (BBS) 30 DAC criteria Key questions Methodology Sustainability  Does the original strategy for the project’s sustainability and lasting impact remain feasible? If not, why? Are additional efforts needed, or are other changes to the strategy recommended?  Document review and staff interviews  BBS Impact  How has CLEAN’s interventions affected targeted beneficiary groups? Are the project’s interventions having the expected effect on identified constraints?  Desk review,  KIIs, and FGDs  BBS 2.2. Evaluation design Review of project documents and data The evaluators reviewed relevant documents produced/published by the project, FAO, the Government of Laos, USDA, as well as by other projects with similar scopes of work. The desk review also included policies such as the Government of Laos/MoAF Agreement on “Establishment and Management of Agriculture Production.” The MTE used the project Baseline and Monitoring (MEL) reports, the semi-annual activity reports and data that was developed to measure the project’s progress. See Annex 2. Annotated Bibliography. Beneficiary-based survey Sampling size methodology The CLEAN target area covers 5 provinces, 8 districts with a total of 123 villages and 1,895 direct beneficiaries (DBs)/smallholder farmers. To ensure representation of the participating farmers among the four commodities, the sample size was determined for each commodity using the following formula: 𝑛 = N × 1.96² 1.96² + (2e) ଶ × (𝑁 − 1) n = sample size N= total direct beneficiaries 1.96 = 95% confidence level e = margin of error = 5% After determination of the required sample for each of the four commodities, selection of villages and respondents followed the sampling guide for Beneficiary-Based Surveys (BBS) for Select Feed the Future Agricultural Annual Monitoring Indicators16. Among the four survey design options that are indicated in the sampling guide, the MTE-BBS used a survey design option 417 which is a two-stage cluster design of farmer groups (FGs) with “take all” selection of beneficiaries within selected FGs at the second stage of sampling.18 16 Diana Maria Stukel and Gregg Friedman. 2016. Sampling Guide for Beneficiary Based Surveys for Select Feed the Future Agricultural Annual Monitoring Indicators. Washington, DC: Food and Nutrition Technical Assistance Project, FHI 360. 17 According to the Sampling guide for BBS for Select Feed the Future Agricultural Annual Monitoring Indicators (USAID.2016, p.74-76) 18 Gender was no considered in the sampling, as the sampling was done by commodity. In fact, in reference to Table 6 and Table 9, women were represented in both the interviews and well as being join leader in the family commodity participation, or sole leader. 31 At the first stage, a sample of FGs (clusters) is selected from the first stage frame of all FGs using fractional interval systematic sampling. The probability of selection of the cluster was calculated as follows:  Grouping villages by commodity with the number of farmers in each village  Calculation of sampling interval 𝑠𝑎𝑚𝑝𝑙𝑖𝑛𝑔 𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙 = 𝑡𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑐𝑙𝑢𝑠𝑡𝑒𝑟𝑠 𝑜𝑛 𝑓𝑟𝑎𝑚𝑒 (𝑀) 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑐𝑙𝑢𝑠𝑡𝑒𝑟𝑠 𝑡𝑜 𝑠𝑒𝑙𝑒𝑐𝑡 (𝑚)  Calculation of random start with the following formula: rand ()*sampling interval  Selection of cluster based on random start and sampling interval, until the total number of clusters (m) to select is reached. At the second stage, a comprehensive list of all beneficiary farmers within the selection FGs is used to ensure that all beneficiary farmers have a chance to be interviewed. This approach is called “take all” sampling. This method was preferable since it ensured reliable results as well as allowing for the field interviews to be conducted within a reasonable timeframe Finally, the sample size for the total population across the four commodities comes to 799 interviews. The maximum number of field interviews increased to 1,104, in order to take account for non-response rate and the possible number of farmers interviewed in the “take all” approach (to interview all the farmers producing that particular commodity in that particular village), distributed over 55 villages in all 8 districts. See Annex 1. Sampling Size Tables by Commodity for MTE. Beneficiaries-Based Survey Questionnaires The BBS survey collected the quantitative data to understand the current commodity productivity and the number of producers to use their increased skills in capacity building from training and extension activities related to commodity production and post-production to improve the quantity, quality and profitability of each commodity. The BBS collected data on volume and value of the agriculture products. The BBS also collected information on improved techniques (soil cover, conservation, irrigation and use of organic fertilizer) that farmers can apply based on the training sessions they have attended, and the total sales of the various forms of the four commodities. In the original Coffee and Cassava Study, carried out by Rapid Asia, the commodities of coffee and cassava were disaggregated by the volume and price sold for each of the following types: coffee, fresh cherries, dried cherries, parchment, and dried beans; for cassava, the information was disaggregated into fresh and sliced/dried cassava. During the review of the BBS and the agreement on the final questionnaire for the BBS, the volume and sale price for these two commodities were not disaggregated but reported as total values for each commodity, as many of the coffee and cassava farmers did not yet have accounting systems, nor follow-up by DAFO. When field staff collected data, the coffee and cassava farmers confirmed that they only kept records informally and did not keep accurate records for the types of coffee and cassava they harvested and sold BBS sample size vs. actual sample size interview As indicated above, the BBS survey was conducted in 53 out of 55 selected villages with a total of 814 interviews completed, which exceeded the 799 interviews needed. Table 4, below, 32 lists the target villages by each commodity type and the samples – expected and actual – targeted by the MTE-BBS. Field data collection experienced obstacles as many farmers were not available during the time period due to their own field work. For example, the month of March is a key time period for cassava farmers as they are harvesting, slicing, drying, and selling cassava to vendors. While the field enumerators were able to interview the required number of farmers for cassava and coffee, this required that they spend extra time in interviewing farmers in their fields and by phone interviews in the evenings in order to reach the sample size. The target was to conduct 126 interviews with sacha producers and 208 interviews with cabbage producers. With the sacha inchi producers, 61% of the target was reached, compared with 86% for the cabbage producers, while both coffee and cassava exceeded the total planned interviews (see Table 4). It proved difficult to contact farmers in the selected villages; there were no alternate villages due to the limited time that enumerators had in the field. Moreover, there were limitations with conducting phone interviews as many people either did not have phones or the phone numbers had been disconnected. It is for this reason that the interview targets could not be reached for cabbage and sacha inchi. The results need to be interpreted with caution as there may be other factors which would affect the results, as the number of people interviewed is not likely to be representative of the total number of sacha inchi growers. In Table 4 below, the sample size for each village and each commodity is listed; the sampling sizes include a 10% additional number of interviews in order to account for the non-response rate. Table 4: Sampling size needed vs. completed interviews by commodity and region19 Commodity type Sample size needed (n) Planned interviews (n) Completed interview (n) Non￾responsive (n) Completed interview rate (%) Sample size reached Cabbage 19 Source: MTE-BBS in March 2021 Phone interview methodology: Many farmers have cell phones; however, in remote areas, the signal is poor or there might not be electricity to charge phones. As a result, farmers might let their sim cards expire and the number is disconnected. When the Elixirs team could not meet farmers in their fields or in the village, they confirmed the names of these farmers with the village chief, who was present during these interviews. The length of time for the interviews was shorter than in the field as there were fewer distractions (such as people interrupting or taking care of a crying child). The information obtained was the same as the information from face to face interviewing. For the KIIs with government staff, grantees, entrepreneurs and CLEAN staff, the Elixirs team requested that CLEAN contact these people to inform them that the interviews were authorized as part of the MTE. However, when Elixirs team members attempted to contact them, there was either no signal (those staff in the field) or the person said that he was too busy and did not respond to further calls. Elixirs staff would attempt to contact them three times before marking them off as non￾contact. 33 Commodity type Sample size needed (n) Planned interviews (n) Completed interview (n) Non￾responsive (n) Completed interview rate (%) Sample size reached Paksong 56 65 44 21 79% no Laongarm 22 23 15 8 68% no Kasy 64 96 74 22 115% yes Vangvieng 44 38 37 1 83% no Thateng 22 17 9 8 42% no Total 208 239 179 60 86% no Coffee Laongarm 69 115 87 28 126% yes Thateng 129 197 157 40 122% yes Total 198 312 244 68 123% yes Cassava Laongarm 86 141 111 30 129% yes Parkngum 54 76 35 41 65% no Sangthong 10 16 16 0 166% yes Xaythani 22 13 10 3 45% no Thateng 96 173 142 31 149% yes Total 267 419 314 105 118% yes Sacha inchi Paksong 88 87 54 33 62% no Laongarm 8 10 3 7 38% no Thateng 31 37 20 17 65% no Total 126 134 77 57 61% no 2.3. Focus group discussions Qualitative methodologies give an idea of a sample of the experience of members of the four commodities groups on how they viewed the implementation of the project, which provides information to explain the findings in the BBS. In addition, FGDs identified some problematic areas which may be affecting project implementation. The qualitative methodology included both FGDs as well as key informant interviews with government partners at central, province and district levels. Selection and analysis of focus group discussions (FGDs) The evaluation team conducted 16 FGDs when there were enough farmers present in a village at the time in order to bring together a group of between six to 15 people. one or two villages per commodity were chosen for the FGDs with direct beneficiaries who were members of farmers groups; the villages and districts were selected in coordination with CLEAN project staff. The selection of the participants was both purposive as the selection was by commodity and based on villages producing that commodity; the selection was also opportunistic as it relied on farmers producing those commodities being available in those villages at the FGD’s appointed time. These group discussions allow participants to identify how the project affected their lives and production habits. 34 The participants were selected by commodity and based on villages that produce that commodity. The selection was also opportunistic as it relied on farmers producing those commodities being available in those villages at the FGD’s appointed time. Criteria for village selection for an FGD: 1. One village per commodity would be selected per region (north/ center and south); 2. Number of beneficiaries for each commodity and village – to ensure number of people are enough for FGD as each FGD was planned for six-eight people and, 3. Location or accessibility –whether farmers have difficulty accessing the market. The groups in each village were also divided between men and women, as male farmers might not be available as women, who are also working in the family farms (and often substitute for the lead farmers in training when their husbands cannot attend). Separate FGDs for women also allowed them more opportunity to speak up (as very often women are less inclined to speak up when in mixed groups) and to understand if the training and experience of men and women differed. Table 5 summarizes the number of DB for FGDs in MTE. In terms of how the results from the FGDs were analyzed, the group generally reached a consensus for each of the semi-structured questions. The responses were written down during the FGD, and summarized, in Lao language, in a post-FGD debriefing between the focus group facilitator and one or two note-takers. The responses were then translated into English and entered into an Excel file, corresponding with each question. The questions for the FGDs are incorporated in Annex 4. Guiding Questions for FGDs . In the sections below for each of the indicators, there is additional information from the FGDs. The FGDs can provide an explanation for the response in the BBS, such as the enabling practices which has increased the effectiveness and efficiency of the project implementation and the obstacles which have slowed implementation. The group discussions, by their informal nature, which also included some follow-up questions based on responses, provided both the successes, and challenges they considered important. A constraint to the FGD process was that we did not have results from the BBS and could not expand on those findings which were inconsistent, such as responses by the sacha inchi group on the assessment of their ability to apply knowledge gained in training sessions. Table 5: Sample size for FGDs by commodity and region20 Province District Village name Commodity type No. of FGD No. of Participants Total F M F M Vientiane Capital Pakngum Pakkouang Cassava 1 1 8 10 18 Vientiane Province Kasy Viengkham Cabbage 1 1 6 0 6 Vangvieng Phatang Cabbage 1 1 6 6 12 Champasak Paksong Kongtoun Cabbage 1 1 4 6 10 Nongkali Sacha inchi 1 1 3 6 9 Sekong Thateng Kamkok Cassava 1 1 9 7 16 Nongkun Coffee 1 1 6 9 15 Saravan Laongam Laoyar Coffee 1 1 6 6 12 Total 8 8 48 50 98 Note that nine villages and 18 FGDs were originally scheduled but the second FGD for sacha inchi could not be done as none of these farmers were in the village at the time. 20 Source: MTE-BBS in March 2021 35 2.4. Stakeholder interviews Stakeholder interviews were conducted with over thirty relevant key stakeholders in March 2021. The list of people to be interviewed was agreed upon in consultation with the project management team and Winrock to ensure a range of representatives this included employees from: the project management team, the DoA, the MoAF (including PAFO and DAFO), grantees, mobile traders, government partners and others in government departments related to the project. The full list of interviewees can be found in Annex 3. GAP Certification Process and the KIIs questions are in Annex 5. Guiding Questions for KIIs. 2.5. Training of field supervisors and enumerators Prior to the survey, Elixirs field supervisors and the enumerator team received a two-day training session in Vientiane Capital by the team leaders on 27-28 February 2021. This training included an introduction to the CLEAN project, review of the BBS survey tools, qualitative tools, KIIs and data collection methods. The training was designed to ensure that the field supervisors and enumerators had a full understanding of:  An overview of the CLEAN project, its objectives, indicators, and data collection  The questionnaires and their accompanying guidance; the trainers and field supervisors reviewed every question with enumerators and explained in detail the implications of each question and their responses; using visual aids and inviting enumerators to share their experience to the group using practical examples from Elixirs’ experience when needed. This is an effective way to build capacity and help enumerators learn from each other  Qualitative topic guides and follow-up questions, including the role of the field supervisors in leading the FGDs and the role of the note-takers  Practice with the enumerators for using KoBo Collect and the tablet for data collection; the enumerators worked in pairs to test both their understanding of the questionnaires and the use of tablets. This step also allowed the team to validate the questionnaires and responses (especially important when the tools were translated from English to Lao)  Use of the consent form before conducting the interviews and the principles of ‘do no harm’ and ‘personal data protection’ 2.6. Data collection methods Data collection was done by using the KoBoCollect application, by uploading the BBS questionnaires, translated into Lao, into mobile phones. The translation was checked for accuracy before uploading to the KoBoCollect secure server. The data collection process was done in close supervision and the coordination of two team leaders who were available for backstopping in Vientiane Capital to support the 9 enumerators. The team leaders coordinated with the team through WhatsApp and telephone calls during the data collection period, to share results and receive guidance on any issues that farmers have encountered. Data collection took place from 9 March to 26 March 2021, starting with villages in Vientiane Capital and Vientiane province and then continuing with three southern provinces. 2.7. Data management and data analysis of descriptive statistics Monitoring data quality included: 36  Data cleaning in the cell phones to check that data was entered correctly, both before sending data to KoBo Toolbox website with further data cleaning of the Excel file to ensure that the data had no discrepancies;  Using sorting function of Excel to generate the figures and percentages for assessment against outcome indicators; and  Factors which can explain improvements, lack of change in the various outcomes were explored in FGDs and in the KIIs which were entered into an Excel file. 2.8. Ethical consideration and personal data protection Both field supervisors and enumerators were trained on how to introduce themselves and the process to obtain consent from interviewees and permission to take pictures (when appropriate). In addition, the team of Elixirs are fully trained and required to collect data applying ‘Privacy by Design’ principles. The training for the enumerator was focussed on building their understanding of the survey tools and did not include a module on working with vulnerable or marginalized people. However, there was a training module on consent and the concept of ‘do no harm’, which was co-facilitated by Elixirs staff including the field supervisors. Issues surrounding gender and other social constraints was discussed in the FGDs, which were led by the Elixirs field supervisors. These Elixirs staff already have experience and training with working with issues in gender, marginalized peoples, and people with disabilities. 2.9. Evaluation limitations Issues with comparing baseline and MTE studies: Baseline data was collected in two different studies. The first had collected data on cabbages and sacha inchi, which also included a range of other commodities such as citrus (oranges and limes), chili pepper, black pepper, and cardamom. From this information, the commodities of cabbages and sacha inchi were chosen. The second baseline data collection was for coffee and cassava. The analysis methods were different for the two baseline studies methodologies. The first study followed a two-stage cluster sampling design with a random selection of 51 villages and the second stage calculated using the sampling formula based on the sample size of 14,877 farmers (the target number of farmers by the end of the project), from which a random sample of 1,020 people were selected. Of those sampled households, 373 did not produce any of the five selected commodities and 282 households produced more than one of the selected crops. The survey team completed 1,030 crop questionnaires (some farmers produced two commodities) from the 1,020 households. The 2018 baseline study often included all 5 commodities in many of its data sets which could not be disaggregated for analysis, based on the two commodities chosen for the CLEAN project. The second study for coffee and cassava used a two-stage cluster sampling as well with 54 coffee producing villages and 31 cassava producing villages. The targets for CLEAN again were projected at 14,877 of which 10,821 were identified for the two commodities with coffee projected at 9,411 farmers and cassava at 1,410 farmers. A total sample size of 501 farmers with 360 coffee and 140 cassava were selected. Since the sampling was not based on the same sampling methodology, weighting to combine the two samples could not be done. The summary results of this study conducted in 2019 and validated in 2020 were also extrapolated from the final target figures of 10,821 farmers for both coffee and cassava, by the proportion of farmers (farmer base) and multiplying it with the average result for MT produced and value of sales. 37 Constraints in reaching target interviews: The people involved in field data collection and organization of focus group discussions experienced obstacles as many farmers were not available during the time period due to their own field work. See Table 10. While the team was able to reach the targets for cassava and coffee farmer interviews, they had to contact them by phone or meet them in their fields; phone interviews for sacha inchi were limited as some farmers did not have phone numbers or the phone numbers were disconnected. Accuracy of volume of products sold and price for sales: As noted in Section 0, farmers still do not have the skills required for recording the volume of products harvested, nor the value of sales for these products. In particular, they could not provide figures for the different types of coffee (fresh, dried cherries, parchment, or dried beans) nor the volumes and prices of dried vs. fresh cassava sold. Focus group discussion sessions: In spite of the FGDs being organized in close cooperation with the district agriculture office staff, so that participants would know the exact time to attend the sessions, only one FGD could be arranged with sacha inchi producers. Only one village had enough producers in the village at that time, in order to bring a group discussion together. This is possibly due to farmers in that village not having adequate notice of the event taking place. FGDs are often most useful when the quantitative information is available prior to the qualitative portion so that discrepancies and explanation of information from data can be explored. KIIs with government stakeholders, enterprises, grantees and CLEAN were limited because some of these key informants could not be reached, either due to their own work responsibilities, in the field and no phone signal or they were otherwise not available. 3. Findings 3.1 Baseline findings21 Coffee farmers: In the baseline report for coffee and cassava (page 8), only one-quarter of the land for coffee uses some kind of soil cover and organic fertilizer. Farmers interviewed believed that the cooler and moister conditions in the coffee growing areas are adequate and that they don’t need to do anything for soil cover. Just over 20% of coffee farmers used these methods. Irrigation was not used. Cassava farmers: Improved farming methods were generally not used by cassava farmers. They generally relied on rainwater instead of irrigation, and in 2019, they had not used organic fertilizer for their crops. Cabbage farmers: Farmers consider that soil preparation, such as ploughing, are necessary for crops such as cabbage. In Paksong, no mulching was applied whereas in Kasy, where cabbages are planted on rice fields after harvest, farmers mulch with rice straw and green mulching. White cabbage is irrigated using water from an irrigation canal or river. Irrigation techniques such as furrow, sprinkler, or drip line, are used in Paksong, more than Kasy (where 40% of the farmers relied on flood irrigation). 21 Ibid, Baseline reports 2018 and 2019 38 Sacha inchi farmers: 16% of the respondents used irrigation for their vines. Soil preparation did not include ploughing but digging holes for planting new vines. 3.2 Key findings from MTE The findings of the evaluation draw on both the quantitative data from the BBS survey combined with qualitative information from KIIs, FGDs and secondary sources. a) Demographic and agriculture profile Table 6: Profile of the respondents22 Cabbage Cassava Coffee Sacha inchi Total Number of respondents 179 314 244 77 814 male (%) 49.2 64.7 25.4 72.7 53.0 female (%) 50.8 35.4 74.6 27.3 47.0 average age (in years) 44.6 43.6 46.0 42.6 44.2 Average household size 5.5 6.1 6.4 5.9 6.0 members below 15 years old 1.2 1.7 1.8 1.4 1.5 members above 15 years old and above 4.3 4.4 4.6 4.5 4.4 A total of 814 interviews were conducted over the four commodities produced, of which 53% were male respondents. The average age of the respondent was around 45 years old while household-size averaged 6.0. It can be seen from Table 6 that coffee producers have the biggest household size among the four types of farmers. Table 7: Percentages of ethnic composition by commodity23 Ethnicity Cabbage Cassava Coffee Sacha inchi Total Lao Loum 69.3% 30.9% 25.8% 44.2% 39.1% Lavaen 4.5% 17.2% 18.0% 3.9% 13.4% Katue 0.6% 3.5% 29.1% 2.6% 10.4% Arak 0.0% 19.1% 5.7% 5.2% 9.6% Suay 3.9% 11.1% 6.1% 0.0% 7.0% Ta-Oy 1.1% 4.1% 3.7% 16.9% 4.5% Trieng 0.6% 4.1% 6.1% 7.8% 4.3% Khmu 11.7% 0.0% 0.4% 3.9% 3.1% other 8.4% 9.8% 5.0% 15.6% 8.5% Main ethnic groups found in the project area are Lao Loum (39%)24; followed by other sub￾groups like Lavaen (13%), Katue and Arak (both 10%). Table 7 displays the ethnic composition by type of commodity. 22 Source: MTE-BBS in March 2021 23 Source: MTE-BBS in March 2021 24 Lao PDR has a high diversity of ethnic groups although, they are often broken down into Lao Loum, Lao Therng and Lao Sung. Lao Therng and Lao Sung groupings are both highly diverse with different languages, culture, land use patterns, etc. Lao Loum is the majority group of ethnic peoples who speak Lao as a primary language and share similar cultural traits, most often related to Buddhism. 39 Table 8: Farm characteristics25 Cabbage Cassava Coffee Sacha inchi Total Average land size (in ha) 0.56 2.38 2.77 0.46 1.54 HH with land certificate (%) 98.5 94.1 94.9 98.7 96.5 Land ownership (%) Own 72.6 85.7 99.2 96.1 88.4 Rent 19.0 13.4 1.2 0.0 8.4 Provided by relatives 7.8 3.8 0.4 2.6 3.7 Other 1.7 0.0 0.4 2.6 1.2 Average number of total workers 4.4 8.1 8.2 4.2 6.2 share of male workers (%) 46.3 49.9 44.3 54.8 48.8 share of female workers (%) 53.7 50.1 55.7 45.2 51.2 Average land size over the four commodities comes to 1.5 hectares. Most of the farmers surveyed are smallholders cultivating their crops on their own land and with a land certificate received from the government. The average number of workers on the four commodities amounts to 6.2, with a lower number growing cabbage and sacha inchi crops. Female farm workers formed at least half of the workers in the field for most crops, ranging from 50.1% (for cassava) to 55.7% (for coffee), with only sacha inchi having women make up less than half of the farm workers at 45.2%. The Government of Lao PDR retains ownership of all lands. There is now a land titling system in which people can have claimed a level of ownership for growing crops, house-building, etc. 96.5% of people are farming on land in which the owner is the farmer himself, a relative or the farmers is renting from someone else has the land title. Table 9: Producer size and type of farm leadership26 Farm leadership is mainly organized jointly by male and female (47.6%) followed by male (44.4%). The highest female led ownership is mainly observed with the production of cabbage (18.4%). Non-smallholders are more often found in coffee production. See Table 9. cabbage cassava coffee sacha inchi Total # % # % # % # % # % Type of producer smallholder (≤ 5ha) 164 91.6 282 89.8 191 78.3 71 92.2 708 88.0 non-smallholder (> 5 ha) 15 8.4 32 10.2 53 21.7 6 7.8 106 12.0 Type of farm ownership male 63 35.2 139 44.3 126 51.6 29 37.7 357 42.2 female 33 18.4 31 9.9 15 6.2 5 6.5 84 10.2 joint 83 46.4 144 45.9 103 42.2 43 55.8 373 47.6 25 Source: MTE-BBS in March 2021 26 Source: MTE-BBS in March 2021 40 Chart 1: Main water source by commodity27 Chart 1 shows that rainwater is the dominant water source used for cassava, coffee and sacha inchi. Irrigation systems (14%) are used for the production of cabbage with 57% of the farmers still relying on surface water. Of the 25 responders who applied irrigation in cabbage farms, 19 of them are from Vangvieng, four from Paksong, and two from Kasy district. Table 10 Respondents harvest timeline by commodity28 Commodity Jan. Feb. Mar. Apr. May Jun. Jul. Aug. Sept. Oct. Nov. Dec. Cabbage 43 77 51 22 15 19 29 30 20 18 37 37 cassava 199 226 156 57 22 4 0 0 0 6 31 78 coffee 112 87 61 30 6 3 0 0 6 75 138 147 sacha inchi 29 37 33 23 16 13 12 11 12 14 39 28 Legend: Farmers are available Farmers are partially busy Farmers are busy Farmers are extremely busy The green zone indicates that there are some periods of time during the rainy season when the rice has spread out enough and tall enough that weeds are not a problem, so during that several week periods, is a good time for training and however, some people do return to their permanent homes during that time. In the FGDs, the farmers have said about the training occurring at the wrong time. There are many factors that need to be considered when making the training plan. DAFOs also considered road access and not about the time of the harvest and when people are busy It is important thing is that DAFO trainers and CLEAN incorporate the agriculture calendars so that they can maximize time periods when they are free, not just from the commodity cropping but from the other crops they grow. For instance, if farmers are planting maize (not addressed at all in the project data gathering), they often stay out in their fields for days at a time. 27 Source: MTE-BBS in March 2021 28 Source: MTE-BBS in March 2021 2.2 0.3 0.4 2.6 1.4 1.7 0.0 0.0 1.1 14.0 0.0 3.5 19.0 79.3 98.4 83.1 69.9 57.0 2.9 0.0 9.1 17.2 6.2 17.5 1.2 2.6 6.9 0% 25% 50% 75% 100% Cabbage Cassava Coffee Sacha inchi Total Borehole Wells Irrigation Rainwater Surface water Other 41 b) Findings by Indicators Estimates that represent the entire population of beneficiaries have been generated with STATA15 statistical software through svyset and svy:total command. The following settings have been used to the dataset: svyset village [pw = sw], strata(commodity) where: svyset is a pre-fix command that designates variables that contain information about the survey design, such as the sampling units and weights; village is the primary sampling unit; pw is the variable name for the final sampling weight for each respondent; sw is the value of the final sampling weight for each respondent; commodity is the stratification used to break up the population into different groups. The calculation of the final sampling weight for respondents follows the BBS guidelines and steps, where the two-stage probability of selection at the cluster level was used, together with the inclusion of the weights for non-response. Final sampling weights differ from one village to the next, depending on the number of clusters selected by commodity for the MTE and the non-response rate by village. The example of calculation of the final sampling weights values follow the survey design option 4 methodology29, of which detailed steps are displayed on Table 11. The calculation sampling weights for each village by commodity presents in Annex 9. Table 11: Sample of determination of sampling weight values for one village producing cabbage30 Houaywai f1i = Probability of selection of Cluster i at the first stage Where: m Number of villages to be selected = 14 M Total number of villages on frame = 32 = Number of villages to be selected m = 0,44 Total number of villages on frame M f2ij = 𝑝𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 𝑜𝑓 𝑠𝑒𝑙𝑒𝑐𝑡𝑖𝑜𝑛 𝑜𝑓 𝑏𝑒𝑛𝑒𝑓𝑖𝑐𝑖𝑎𝑟𝑦 𝑗 𝑎𝑡 𝑡ℎ𝑒 𝑠𝑒𝑐𝑜𝑛𝑑 𝑠𝑡𝑎𝑔𝑒, 𝑎𝑠𝑠𝑢𝑚𝑖𝑛𝑔 𝑐𝑙𝑢𝑠𝑡𝑒𝑟 𝑖 is 𝑠𝑒𝑙𝑒𝑐𝑡𝑒𝑑 𝑎𝑡 𝑓𝑖𝑟𝑠𝑡 𝑠𝑡𝑎𝑔𝑒 For take all sampling, because the number of beneficiaries selected for sampling in any village is always the same as the number of beneficiaries in that village, f2ij always equals 1 1 wij = Overall sample weight (the inverse of the overall probability of selection) = 1 2,29 f1i x f2ij Adjusting Survey Weights for Non￾Response Where: Number of beneficiaries selected to be interviewed in Cluster i bi = 6 29 Ibid. BBS guideline (USAID.2016, p.78-88) 30 Source: MTE-BBS in March 2021 42 Number of beneficiaries who do not respond (not found, not present or who refuse to respond) NR = 0 Number of beneficiaries interviewed bi - NR = 6 Weight Non￾Response = No. of beneficiaries selected to be interviewed (in a sampled cluster) = 1,00 No. of beneficiaries actually interviewed (in a sampled cluster) Calculating the Final Sampling Weights Weight Final = WProbSelection x W Non-Response 2,285714 Even though the minimum sample size was met on cassava, it can be noticed that the sampling weight for that commodity is slightly higher as compared to cabbage, where the minimum sample was not reached. The svy:total command has been afterwards executed on MTE-BBS relevant variables to provide estimates on the project monitoring indicators that are aligned with the Feed the Future’s standard indicators, namely S1-1, S1-2, S1-13, SI-1431, SI-1732 and S1-18. The weighted values in tables for S1-1, S1-2, S1-13, SI-14, and S1-18 have been generated for the entire population of beneficiaries based on the MTE-BBS sample by selected 55 villages, and by applying sampling weight for each beneficiary who was interviewed in the selected villages and their commodity. The standard error and 95% confidence interval cannot be summed for all four commodities, as the range of volumes and sales the commodities produced is too wide. SI – 1 is the number of hectares under improved techniques or technologies as a result of USDA assistance. This indicator measures the area in hectares where USDA-promoted management practices or improved technologies were applied to areas managed or cultivated by producers participating in a USDA-funded activity. Improved management practices or technologies are those promoted by the recipient as a way to increase the productivity directly or to support stronger and better functioning systems33 . Table 12: Estimates on indicator S1-134 Indicative sampling weight Hectares under improved techniques Std. Err. [95% Conf. Interval] MTE-BBS results Weighted values Cabbage 3.142 67.0 213.3 57.895 88.246 338.393 Cassava 3.285 375.3 1,309.5 339.107 563.082 2055.820 Coffee 2.433 371.8 586.4 192.793 150.293 1022.548 Sacha inchi 3.887 13.9 52.4 19.561 10.884 93.820 Total 828.0 2,161.5 Table 13: Yield per hectare35 31 Winrock and Elixirs are in the process of finalizing the weighting of this result. The cumulative result may be amended once this process is finalized. 32 SI-17 uses the actual numbers of direct beneficiaries involved in the project instead of a weighted value. 33 FFPr INDICATOR 3, 2019 Food Assistance Indicators and Definitions, Foreign Agricultural Service –Food Assistance Division, United States Department of Agriculture. 34 Source: MTE-BBS in March 2021 35 Source: MTE-BBS in March 2021 43 Planted area (in HA) Volume harvested (in MT) Yield per hectare (MT/ HA) Cabbage 100.075 2,078.344 20.768 Cassava 749.893 9,641.626 12.857 Coffee 1,715.034 959.643 0.560 Sacha inchi 35.423 104.013 2.936 Table 12 shows the total area that beneficiaries had cultivated, using improved techniques or technologies; this weighted figure amounts to 2,161.6 hectares. More than half of this land area (61%) is farmed by cassava producers, followed by coffee and cabbage farmers. While there is baseline information for the commodities in the two baseline reports, as noted in constraints, the methods for analysis are different and cannot be compared. An important component for the project is improving the knowledge and practice of farmers in their production and following the guidelines of GAP in order to improve the quality of their produce to increase their marketability. The improved techniques promoted by the project included:  Improved soil cover  Soil conservation which includes development of terracing, green corridors and methods to reduce water run off which can reduce soil fertility. Soil cover and soil conservation methods were covered in the Table 15 in the training module on ‘land selection and preparation’.  Preparation and application of organic fertilizer which includes the composting of hay, leaves, cuttings from vegetables and manure from animal paddocks or chicken coops  Improved irrigation – methods include digging canals, pumping water from a clean water source to the field, and drip irrigation (laying pipes along the rows of plants). SI-2 Number of individuals who have applied new techniques or technologies as a result of USDA assistance This indicator measures the total number of farmers, government partners and enterprise staff in USDA-funded activities who have applied improved management practices and/or technologies promoted by USDA anywhere within the food and agriculture system during the reporting year. The indicator tracks individuals who are changing their behavior while participating in USDA-funded activities36 . Chart 2: Percentage of share of respondents applying improved techniques37 36 FFPr INDICATOR 4, 2019 Food Assistance Indicators and Definitions, Foreign Agricultural Service –Food Assistance Division, United States Department of Agriculture. 37 Source: MTE-BBS in March 2021 67.7 58.0 47.0 62.3 56.5 32.3 42.0 55.8 37.7 43.5 Cabbage Cassava Coffee Sacha inchi Total No Yes 44 Out of 814 farmer respondents, 692 (85%) of them responded that they attended training from the CLEAN project and applied at least one of the techniques demonstrated on their farm. Of those who attended CLEAN trainings, the application rate of technology differs from one production system to another as 67.7% of cabbage producers participated in comparison with 47% of coffee growers (Chart 2). Cabbage requires a short growing cycle, so farmers were able to apply their new knowledge and skills more quickly by, for example, using compost for soil preparation; whereas other commodities require longer growing periods and soil preparation using traditional methods. Table 14: Estimates on indicator S1-2: Number of individuals who have applied new techniques or technologies as a result of USDA assistance38 Indicative sampling weight Individuals applying new techniques Std. Err. [95% Conf. Interval] MTE-BBS results Weighted values Cabbage 3.142 109 334 66.7 190.0 478.2 Cassava 3.285 152 522 117.6 263.5 781.0 Coffee 2.433 92 199 29.5 132.8 266.2 sacha inchi 3.887 38 142 34.6 69.2 215.7 Total 391 1,198 Similar to the settings and final sampling weights used for S1-1 indicator, estimates have been produced on the number of individuals who responded that they applied a new technique or technology taught by the project. Results have been disaggregated by commodity. In relation to the entire population beneficiaries, Table 14indicates that a weighted figure of 1,198 individuals reported to have practiced at least one new technique or technology, mainly with cassava producers (44%) followed by cabbage farmers (28%). Table 15: Top three new practices applied by farmers as a result of USDA assistance39 Rank Cabbage - n=109 % 1 Use of fertilizer and timing for use of fertilizers 67.0 2 Pest management 56.0 3 Use of pesticides and other chemical substances 50.5 Cassava - n=152 1 Land selection and preparation 84.9 2 Seedling selection, preparation and preservation 69.1 3 Proper weeding practice 44.7 Coffee - n=92 1 Use of fertilizer and timing for use of fertilizers 66.3 2 Land selection and preparation 54.4 3 Proper weeding practice 52.2 sacha inchi - n=38 1 Land selection and preparation 92.1 38 Source: MTE-BBS in March 2021 39 Source: MTE-BBS in March 2021 45 2 Seedling selection, preparation, and preservation 86.8 3 Use of fertilizer and timing for use of fertilizers 65.8 Table 15 shows that the technique of land selection and preparation is ranked as one of the top three new practices among cassava, coffee and sacha inchi producers. Cabbage producers mentioned that the use of fertilizer and pest management were the main techniques used on their cultivated land. In cabbage FGDs, farmers indicated that organic methods resulted in slower growth, but the cabbage heads were of better quality in terms of density and appearance. Chart 3: Period of first adoption of improved technology by commodity40 As displayed in Chart 3, most of the farmers first applied their improved technology during Year 3 of the project, especially for the sacha inchi producers (68.4%). It should be noted that at the time of the MTE-BBS (March 2021), there were just six months completed for Year 4 project. The greatest share of farmers starting new technology in those six months in Year 4 was observed with the coffee growers (41.2%). This change is due to the increased number of trainings for coffee growers. The timing of the training coincided with the time during FY’2020, where coffee growers had time to attend to trees after the rice harvest. Chart 4: Percentage of participants reporting usefulness of project trainings41 40 Source: MTE-BBS in March 2021 41 Source: MTE-BBS in March 2021 61.5 63.8 56.5 71.0 63.2 38.5 36.2 43.5 29.0 36.8 0% 25% 50% 75% 100% Cabbage Cassava Coffee Sacha inchi Total Y3 - Oct. 2019-Sept. 2020 Y4 - Oct. 2020-Mar. 2021 19.3 30.9 33.7 71.1 38.7 48.6 48.7 50.0 15.8 40.8 30.3 18.4 15.2 10.5 18.6 Cabbage Cassava Coffee Sacha inchi Total Not useful at all Less useful Partial useful Useful Very useful 46 CLEAN worked with master trainers from the central government who developed training materials and lesson plans for a PAFO and DAFO capacity building training. Once these materials were developed and the government trainers were in place, PAFO and DAFO staff trained the farmers from CLEAN’s value chain. Through this process, out of 814 respondents, 692 (85%) of them affirmed that they attended a training from the CLEAN project and applied one of the techniques demonstrated on their farm. Out of CLEAN’s pool of trainees, the application rate of the technology differs from one production system to another as cabbage producers did participate in a higher proportion (68%) than coffee growers (47%). Across the four commodities, 258 out of 391 respondents (66%) replied that the training they attended was useful or very useful. Cabbage farmers (81.7%) replied that the training was useful or very useful while the data seems to indicate that 71.1% of sacha inchi producers responded that training was only partially useful (see Chart 4 for responses by commodity and totals). However, in FGDs, the sacha inchi farmers expressed high satisfaction with the project and training, as they received more intensive training with MSL as well as more contact with MSL staff. However, the relative number of sacha inchi farmers is lower than the other farmers in the sample. A short summary of their comments from the FGDs in one village is below. For the problems that farmers had in applying new technique see Table 16. Chart 5: Percentage of famers who said they are applying techniques learned from project42 Around one-quarter of the total respondents found the techniques easy to apply (26.2%), while the majority of them think that the new practices are partially difficult (60.7%). One￾third of the cabbage producers mentioned that the techniques are easy to implement (33%), which is not the case for the sacha inchi producers where most of them declared that it was partially difficult (81.6%). The above finding is in contrast with the results of the FGD in Nongkali village, Paksong District. This village previously had experience with growing sacha inchi and selling to Vietnamese and Thai vendors. In the following quote, they talked about their experience of working with MSL in training and raising of sacha inchi nuts. 42 Source: MTE-BBS in March 2021 12.8 4.6 8.7 7.9 9.1 46.8 65.8 60.9 81.6 60.7 33.0 25.7 29.4 10.5 26.2 Cabbage Cassava Coffee Sacha inchi Total Very easy Easy Partially Difficult Difficult Very difficult 47 Farmers’ experience in application of new techniques According to farmers in FGDs, some of the training had just been started at the end of calendar year 2020 and the trainings in the first half of 2021 had occurred near the time of the coffee, cassava and sacha inchi harvests. As a result, they were not able to apply what they learned with their current crop. However, as mentioned above with the challenges due to the delay of the start of the project and obstacles to accessing villages for training due to floods and travel restrictions as a result of the COVID-19 lockdown regulations in March through May 2020, these delays could not have been avoided. The following discussion could provide some insight to the responses in Chart 5 on the ease of application of techniques after training. Table 16: Main obstacles experienced during application of new techniques43 Rank Cabbage - n=13 % 1 I experienced a problem in applying what I learned 38.5 2 The variety of crop did not grow well 30.8 Cassava - n=15 1 I experienced a problem in applying what I learned 46.7 2 I didn't have the equipment or seeds 40.0 3 The variety of crop did not grow well 26.7 Coffee - n=24 1 The variety of crop did not grow well 28.6 2 I didn't have the equipment or seeds 21.4 3 I experienced a problem in applying what I learned 21.4 Sacha inchi - n=3 1 I experienced a problem in applying what I learned 66.7 45 respondents (12%) experienced problems during the application of the new techniques learned from the project. Table 16 shows that the obstacles faced by farmers were similar across the four commodities. The choice of the crop variety and difficulty in applying the new skills they had learned were the two main issues raised by the respondents. Farmers in FGDs indicated that they did not receive the regular follow-up visits by DAFO or farm extension services; as noted in the quote below, farmers might have problems applying techniques which are new to them, which they would have benefited from follow-up sessions from trainers. 43 Source: MTE-BBS in March 2021 Farmer Feedback on sacha inchi, Nongkali Village, Paksong District In comparison with our practices before and after we worked with the company, we have changed our practices a lot. Mai Savanh Lao provided sacha inchi seeds, equipment for making fences for supporting the vines, and equipment for harvesting the nuts. We work together as a group during planting, making the fencing and harvesting the nuts. We have found a lot of benefits in working together as a group. When we have meetings and work together as a group, we motivate each other. We have hope, determination that our nuts will sell, and we have an idea about how much we will sell. It helps each other to talk about issues. 48 According to farmers who attended FGDs, these trainings were most often provided in village cluster center groups and involved several sessions. A challenge to the effectiveness and practice of using new methods by farmers is that the member of the family who attends the training might change. Although the lead farmer name was indicated in the attendance sheet, often the lead farmer’s wife or another family member would attend if the lead farmer could not attend; this might have an impact on the ability to apply training, as the training is organized so that each session builds on the content of the previous training. Some participants in FGDs indicated that they would also like training within the village and in their own fields so that the trainer can provide help to show the ways in which farmers could apply what they are learning in their fields. In FGDs, farmers indicated that even when they have attended training, the activities started after they had planted their crops and harvested them. As a result, they were not able to apply much of the content they have learned in the training with the crops they had just harvested. While they are likely to use these new methods with the next planting season, they might not remember the content. As noted in the following quote, some farmers lost cabbage because they had not fully applied the concepts of integrated pest management (IPM). Source: MTE-BBS in March 2021 SI – 7 – Number of private enterprises, CBOs that have applied improved technologies as result of USDA assistance Table 17: Number of private enterprises, CBOs that have applied improved technologies44 Private Enterprise/ Grantee Technology/ support Used technology for what purpose Source of Info Sky Vision Expansion of Refrigeration Currently have own storage of cabbage from farmers in order to sell to Thailand when there is a good price CLEAN KII Meuang Lao Food Co., LTD Request funds to link farmers to markets Use of GAP and set standards for farmers so company can sell to high end markets KII Lao Cassava Association Greenhouse for producing cassava stems Greenhouses to grow disease free stems; get stems KII 44 Source: MTE-BBS in March 2021 Notes from the field: One cabbage grower in Thateng District, Sekong Province, lost her crop cabbage during the 2020 growing season. She used organic compost for her cabbage field but noted a large number of flies eating the cabbage. As a result, when the cabbages were large enough to sell, the trader rejected the cabbages because they did not look good. She used those cabbage as compost but did lose a large share of her crop and income. Thateng District, Sekong Province 49 Table 18: Number of Grantees and partners that have applied improved technologies45 Grantee/Partners Grant Used technology for what purpose Source of Info Mai Savanh Lao (Grantee) Oil press, roaster, powder machine Expand range of products for domestic and international markets. Grant ran from Nov 2019 to Nov 2020. Trained 137 farmers KII Plantation Hills (Grantee) Wet mill and coffee storage Starting on 25 Sept. 2020 Semi-Annual Report – April – Oct 2020 CQI (Partner) Coffee cupping Upcoming Grant - Coordinated virtual cupping event, assessing coffee export markets and planning for coffee industry Will receive grant for coffee laboratory, promotion of Lao coffee Semi-Annual Report – April – Oct 2020 Table 19: Farmer groups that have received in-kind grant46 Farmers groups Technology Technology used for what purpose Source of info Cassava group in Vientiane Province Cassava chipping machine Reduces labor and produces standard sized chips for drying CLEAN grants manager Currently, there are 81 farmer groups which are registered with local authorities. The CLEAN project, working with local authorities, has been able to facilitate the registration of Farmer groups so that the groups have a leader, their own rules and regulations, have trained on GAP and ICS and filled out a registration form at the district level. When the groups are registered, it gives them more options for domestic and international trade since they can comply with the regulations of neighboring countries. These farmer groups have also been encouraged to apply for in-kind grants to improve their production. Of the total number, 19 have joined training in how to apply for the grants, fill out the grant applications including the budgeting, business plan, estimated costs, etc. Most of the groups were then able to receive the in-kind grants, such as cassava chipping machines or irrigation pumps for dry season crops. SI – 8 Number of public-private partnerships formed as a result of USDA assistance Public-Private Partnerships are cooperative agreements between two or more private enterprises and government entities, with the aim to strengthen the quantity and quality of the commodities. Currently PPPs are governed through regulations set up in the Land 45 Source: MTE-BBS in March 2021 46 Source: MTE-BBS in March 2021 50 Concession Agreement (LCA)47. CLEAN’s project’s target is to connect the domestic private actors and local government to set up a dialogue establishing PPPs for agriculture business. The project has a target of establishing 12 PPPs by the end of the project. Until this past semi￾annual period, there were no PPPs and in the past six months, two are in the process of being formally established. One of them is Lao Cassava Association where the project has worked to coordinate with them and CIAT building “Tunnel multiplier greenhouses” for the production of fresh and disease-free cassava stems, and the BB Agriculture & Trading Sole Co., Ltd for utilizing a packing house for cabbage. This indicates that the project is establishing relationships as a facilitator between the public and private sectors. SI – 11 Total increase in installed storage capacity (dry or cold storage) as result of USDA assistance The target for increasing storage for the four commodities is 1,250 cubic meters. The current cumulative value of storage for all four commodities is 19 cubic meters; however, this storage had been constructed by coffee and cassava farmers prior to the start of the CLEAN project. The MTE field team did not measure the capacity of dry or cold storage as in the PIRS and PMP documents,48 where the data collected is done through “the beneficiary registration system developed and maintained in DevResults by the CLEAN M&E team. In the 2018 baseline report, the estimate for cabbage cold storage was that 1% of farmers had cold storage. The capacity for dry storage of sacha inchi was 6%.49 One respondent in Viengkham District had cold storage for Chinese cabbage. Growers of sacha inchi stored the dried product in a dry room (either a place used for storing paddy rice or in a spare room in their houses). The importance of cold storage for cabbage is that without it, farmers have to sell the cabbage quickly after harvest, which results in oversupply of cabbage and lower prices for the farmer. In the 2018 report, it mentioned that cold storage could allow for a more even supply of cabbage but it would be difficult for small holders to buy and maintain storage, though it might be possible, but difficult, at a farmers group level, and only then if the farmers group had strong management skills and capacity development. Private enterprises would be the best positioned for investing in and maintaining storage. Enterprises such as Sky Vision, Meuang Lao Food and BB Agriculture have some capacity for storing and selling the cabbage when prices are higher. Table 20: Farmers with storage availability50 Cabbage (n=179) Cassava (n=314) Coffee (n=244) Sacha inchi (n=77) Overall average (n=814) n= % n= % n= % n= % n= % yes 0 0.0 1 0.3 8 3.3 2 97.4 11 16.0 no 179 100.0 313 99.7 236 96.7 75 2.6 803 131.9 47 GOL’s Decree on State Land Lease or Concession, No.135/PM (2009), under Article 10: Authorization of State LCA for industrial Plant, Industrial Farm Business – a domestic and international private actor must lease the land at the minimum of 50 Ha up to 10,000 ha per business size with a minimum of 30 years up to 50 years, and under the Article 29: Level of Authorization of LCA for Agriculture business and Plantation is defined that the provincial/City level shall not approval any LCA exceeding 500 ha per agriculture project with a maximum of 40 years but not exceed 60 years. 48 CLEAN internal documents: CLEAN Performance Indicator Reference Sheet (PIRS) August 21(no year), pages 21 – 23 and CLEAN Performance Monitoring Plan (PMP_1.11) Approved (undated) 49 CLEAN baseline report (2018), page 27 50 Source: MTE-BBS in March 2021 51 During the COVID-19 lockdown period, cabbage farmers lost their crops when traders could not enter the village to buy and transport the cabbage. SI – 12 Number of policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance (Stage1, Stage 2) The target number of policies is three, which has been achieved through the close working relationships between CLEAN and the DoA in the past project period. These policies are the  Guidelines for the Management of Agribusiness,  an amended version of the Law on Plant Protection and  Quarantine and International Standards for Phytosanitary Measures Project implementation in procedures has been greatly improved through the initiation of regular Interim Management Committee (IMC) Meetings. According to the Director General of DoA, these IMC meetings have started to review past activities, review lessons learned, discuss how to improve implementation and plan for the next time period. Since these meetings have started, coordination and communication has improved. SI – 13 Value of annual sales for farms receiving USDA assistance This indicator measures the value in USD of the total amount of sales of products and services by USDA-assisted farms and firms within USDA-supported agricultural commodity value chains or markets, during the reporting year. Examples of USDA assistance include facilitating access to improved seeds and other inputs, to extension, business development and financial services, and to micro-enterprise loans; providing technical support in production techniques; strengthening linkages to markets; and other activities that benefit producers or private sector firms in the agriculture and food system. Cumulative sales, noted in the semi-annual report, include all sales by farms and firms participating in USDA-funded activities.51 In the MTE-BBS, firms were not added into our analysis as the MTE did not have all the data for all the firms; some firms bought and sold multiple commodities in addition to cabbage. Table 21: Estimates on indicator S1-1352 Indicative sampling weight Value of annual sales by producers (in USD) Std. Err. [95% Conf. Interval] MTE-BBS results Weighted values Cabbage 3.142 326,595.745 1,085,106.383 382,978.723 258,510.638 1,914,893.617 Cassava 3.285 1,085,106.383 3,446,808.511 618,085.106 2,085,106.383 4,808,510.638 Coffee 2.433 1,851,063.830 2,617,021.277 1,074,468.085 187,234.043 5,042,553.191 Sacha inchi 3.887 87,978.723 381,914.894 372,340.426 0.00053 1,223,404.255 Total 3,350,744.681 7,530,851.064 1 shows that the total weighted value of annual sales during this reporting year amounted to US$7,530,851as sales from firms and government enterprises are not included in the MTE￾BBS calculations. Cassava producers generated almost half of the total revenue (46%) while total income received from sacha inchi farmers represented a minor share of the total sales value. Despite 51 FFPr INDICATOR 18, 2019 Food Assistance Indicators and Definitions, Foreign Agricultural Service –Food Assistance Division, United States Department of Agriculture. 52 Source: MTE-BBS in March 2021 53 The reason for zero is because of lower limit of confidence interval is a negative value due to the very high nonresponse rate of sacha inchi farmers. 52 the adjustment made with the corresponding sampling weight this could be explained by the low absolute number of farmers selling sacha inchi (22) that was collected in the MTE-BBS field interviews. SI – 14 Volume of commodities sold by farms receiving USDA assistance This indicator collects the volume (as calculated in gross metric tons (MT)) of sales of targeted commodities by farms and firms receiving USDA assistance. This includes the volume of all sales of commodities targeted by CLEAN, not just the volume of farm-gate sales54. In the MTE-BBS, the total weighted value of annual sales during the reporting year amounted to 41,585 MT as value of sales from firms and government enterprises not included in the MTE￾BBS calculations. Most of the total volume is produced by the cassava producers (78%). As an example, in the most recent semi-annual report (Oct 2020 – Mar 2021) the McCullagh Coffee Chain shipped a container of 18 MT of specialty coffee to the U.S. Table 22: Estimates on indicator S1-1455 Indicative sampling weight Volume sold in tonnes Std. Err. [95% Conf. Interval]1 MTE-BBS results Weighted values Cabbage 3.142 2,078.344 6,884.842 2,611.108 1,243.886 12,525.800 Cassava 3.285 9,665.626 32,223.850 6,539.756 17,829.940 46,617.750 Coffee 2.433 1,710.504 2,025.277 727.127 380.401 3,670.153 Sacha inchi 3.887 104.013 451.393 436.795 0.00056 1,424.633 Total 13,558.487 41,585.362 Using the sampling weight for each commodity, Table 22 indicates that a total of 41,585 MT of commodities were sold in the past 12 months. Most of the total volume is produced by the cassava producers (77%). Graph 1: Percentage of farmers accessing markets for their commodities57 Over the four commodities, farmers tend to sell their agricultural production to enterprises/firms (36.3%) followed by local traders in the same village (32.4%). Graph 6 shows that destinations vary from one commodity to the next as 83.8% of cassava producers mainly 54 FFPr INDICATOR 19, 2019 Food Assistance Indicators and Definitions, Foreign Agricultural Service –Food Assistance Division, United States Department of Agriculture. 55 Source: MTE-BBS in March 2021 56 The reason for zero is because of lower limit of confidence interval is a negative value due to the very high nonresponse rate of sacha inchi farmers. 57 Source: MTE-BBS in March 2021 68.7 11.2 47.1 2.6 32.4 48.0 17.5 25.4 7.8 24.7 34.1 11.2 14.3 3.9 15.9 4.5 83.8 37.3 19.5 36.3 16.2 4.8 8.2 71.4 25.2 0% 25% 50% 75% 100% Cabbage Cassava Coffee Sacha inchi Total Local trader in village Local trader from other village Trader in district/provincial town Firms Other 53 rely on local mobile traders, while 71.4% of sacha inchi growers selected ‘other’; from information in the FGD, the farmers responded that they sold sacha inchi directly to MSL staff. Many farmers of all four commodities, sell to local traders who then sell the products to a middleman, and there may be a process of sales through several dealers before the product reaches a company/definitive market/wholesaler. At each step of the process, the price increases as each middleman calculates his expenses and profit margin. The exception is Mai Savanh Lao (MSL) who bought sacha inchi directly from the farmers they trained. In the KII, the CEO said that other traders from Vietnam or Thailand used to go to those villages, but with the travel restrictions due to COVID-19, sales to these other vendors have reduced. Some farmers, even those in farmers groups, choose to bring products to the market by themselves and/or sell to traders in the villages. The value to the farmer is that, if he does not have a truck, the local trader provides the transportation to the market; the farmer also does not lose a day of labor bringing the product to a district town or collection point. However, the price that the individual farmer receives is low. In FGDs, farmers said that sometimes it is difficult to wait, as farmers plant and harvest at different times. If the farmer has transportation or if the trader seeks him out in the village, he can sell individually. However, farmers sometimes don’t have negotiating power when they transport their products to sell outside their village; they will sell at a lower price so they do not want to return to their villages with their products. SI – 16: The number of individuals who have received short-term agricultural sector productivity or food security training as a result of USDA assistance The target for this indicator is that 14,877, or all the direct beneficiaries, should receive training by the end of the project. In the April – September 2020 semi-annual report, 1,884 farmers had attended training in this time period, making a cumulative total of 1,983 since the beginning of the project. This number includes farmers, government staff who have been trained as trainers and inspectors, private enterprise staff and grantees. The reason for these limited results is a combination of a late project start and COVID-19 travel restrictions. This question is linked with the adoption of improved technologies, SI – 01 and SI-02 which addresses the methods that farmers use for the number of HA for each of their commodities. In FGDs, farmers reported that they appreciated the training that they had received. They also reported increased use of organic fertilizer methods such as the use of manure and composting, particularly of cabbage, as they feel that their product looks more attractive and appealing to consumers and that the application of organic fertilizer is safer for the health of both themselves, their families and their customers. Training has been done through a cascade training system of training of trainers, whereby the DoA trains PAFO and DAFO staff, who then train the farmers. According to CI-4, 93 “I have land for growing cabbages on a 6,500 square meter plot of land. I can produce around two tonnes. Usually, mobile traders from outside the village come to buy cabbages in the village. The trader who comes to the village sells to middlemen who give a lower price, 250 - 500 LAK per kilogram. The farmers in our village feel this is an unreasonable price as the cost of production is higher than the sale price offered. The farmers in this village are involved with the project and sell to a company as they have achieved this company’s criteria of GAP certification. However, if the cabbages do not pass the GAP guidelines, the company will not buy the cabbages, even at a lower price.” A cabbage farmer in Phatang Village in Vang Vieng District 54 government officers and authorities have been trained in order to train others in quality standards and food safety. Training is often divided into several sessions, which does have the advantage of providing enough information on a topic, so that farmers do not feel overwhelmed by the amount of information. However, the lead farmer in the family (the one who is registered for project inputs) is sometimes not able to attend the training and sends either their spouse or another family member instead. While this can increase the potential to provide the information to other household members, each session builds on the previous one and if that person did not attend previous sessions, it can make understanding difficult. Training tends to take place either in the head village of the village cluster or in the district. Farmers appreciate the theory and knowledge in the sessions, but they also requested that the training include practice in the fields, in addition to the field that is already built into the training curriculum, so they can reinforce how a procedure is actually done. In addition, they felt that follow-up of training should be increased as sometimes they are not confident about using a new method. SI - 17 Total number of individuals benefiting directly as a result of USDA assistance This is an output indicator measuring the number of individuals directly participating in USDA￾funded interventions. The individuals must be engaged with a project activity or come into direct contact with a set of interventions (goods or services) provided by the project. This may include, for example, farmers and others receiving training, inputs, or financial services. This indicator only tracks direct beneficiaries reached with direct USDA assistance (funded in part or in whole by the USDA.58 Table 23: Number of direct beneficiaries as result of USDA assistance59 Cabbage Cassava Coffee Sacha inchi Total Number of villages 32 28 17 46 123 Number of direct beneficiaries 454 857 407 177 1895 A total of 1,895 beneficiaries were used for the sampling frame, as they have benefitted from the CLEAN project, and could give use the information about the project activities at the field level. Most of the DBs using for sampling were cassava producers (45%). The highest number of villages covered by the intervention is in the sacha inchi commodity. With the most recent data from CLEAN monitoring, the cumulative number of farmers, private sector firms and grantees is 2023.60 SI – 18 Total number of indirect beneficiaries This is an output indicator measuring the number of individuals indirectly benefitting from USDA-funded interventions. The individuals will not be directly engaged with a project activity 58 FFPr INDICATOR 17, 2014 Food Assistance Indicators and Definitions, Foreign Agricultural Service –Food Assistance Division, United States Department of Agriculture. 59 Source: CLEAN Beneficiaries dataset, February 2021 60 CLEAN semi-annual report, Oct 2020 to March 2021 55 or come into direct contact with a set of interventions (goods or services) provided by the project. This may include for example family members of farmers trained61 . The MTE-BBS results consider indirect beneficiaries as people working on the farm, including household members, where the lead person has applied a new technique learned during a CLEAN training. Direct beneficiaries, namely the lead person that has received direct assistance through CLEAN, have not been counted under this indicator. Table 24: Estimates on indicator S1-1862 Indicative sampling weight Indirect beneficiaries MTE-BBS Std. err. [95%_Conf. Interval] results Weighted values Cabbage 3.142 395 1,282 290.4 654.6 1,909.3 Cassava 3.285 1,110 4,444 1,702.2 697.7 8,190.7 Coffee 2.433 824 1,714 550.4 468.9 2,959.1 Sacha inchi 3.887 125 491 151.8 168.8 812.3 Total 2,454 7,931 Table 24 indicates that 7,931 people have indirectly benefited from the project interventions, most of them being from the cassava producers (56%) and coffee growers (22%) as they hire more workers on average as compared to cabbage and sacha inchi farmers. The target is 59,508 and the current total in the October 2020 to March 2021 semi-annual report is 8,133. This number refers to the number of people, other than the lead farmer, who are in the family or household and benefit from the increased income or crop as a result of the project. Table 25: Custom Performance Indicators C1 – C863 Code Description Reported cumulative through FY 20 FY 21 Target Value 10/20 – 03/2021 LOP target Cumulative to date CI-1 Yield per hectare (MT/ HA) per target commodity type White Cabbage 20.26 MT/HA N/A 19.38 TBD 19.93 Cassava 19.2 MT/HA N/A 13.75 TBD 16.64 Coffee 3.29 MT/ HA N/A 1.55 TBD 2.46 Sacha inchi 0.28 / HA N/A 0.20 TBD 0.26 C2 Number of agricultural input suppliers/firms that sell or supply improved inputs as a result of USDA assistance 1 N/A 1 N/A64 1 61 FFPr INDICATOR 23, 2019 Food Assistance Indicators and Definitions, Foreign Agricultural Service –Food Assistance Division, United States Department of Agriculture. 62 Source: MTE-BBS in March 2021 63 Source: CLEAN Semi-Annual Report, October 2020 to March 31, 2021, pages 26 – 28 64 In CLEAN’s PIRS, for baseline, it used NA (not available), but for target or LOP, it used TBD (to be determined). CLEAN’s Lop Targets reflect this fact. 56 Code Description Reported cumulative through FY 20 FY 21 Target Value 10/20 – 03/2021 LOP target Cumulative to date C3 Number of agreements signed (contracts, MoU, etc.) between buyers/sellers through trade promotion events (e.g. missions, fairs, etc.) 21 N/A 24 115 45 C4 Number of government officers and auditors trained in internationally recognized quality standards and food safety 86 N/A 25 177 93 C5 Number of grants disbursed to individuals or firms/cooperatives/producer groups/businesses as a result of USDA assistance 1 N/A 4 15 4 C6 Value of grants disbursed to individuals or firms/cooperatives/businesses as a result of USDA assistance $25,566 N/A $50,434 $498,900 $96,000 C7 Number of farmer organizations meeting GAP and SPS requirements 66 N/A 71 100 71 C8 Number of consumers receiving messaging on food safety and “clean” and “safe” Lao products through awareness campaign 52,645 N/A 926,152 450,000 978,797 c) Findings from stakeholder interviews Government partners and people in government Elixirs staff interviewed 15 government partners, both in Vientiane and in field offices in the provinces. The interviews included two interviews with the Director General of DOA and the DOA Project Coordinator for the CLEAN Project. The KIIs identified the results of the training on ICS-related topics and ToT methods involved in the creation of ICS/GAP certification inspectors, in developing guidelines and training materials for farmers and develop a cascade training system starting with central and province guidelines and a ToT curriculum for the delivery of GAP training to farmers in selected CLEAN intervention areas. The interviews focused on key questions such the process of adoption of quality standards, certifications, and operationalizing Good Agricultural Practices (GAP). The key informants provided their positive impressions and challenges on the implementation and provided recommendations for the project. For enterprises, the people interviewed talked about the ways in which they used training and funding through CLEAN to improve their business infrastructures, which then benefited the farmers through increased ability to negotiate with them on orders and prices. Grantees received funds for innovative ideas to improve their services and sales. 57 The perspectives of farmers were collected during the focus groups discussions (FGDs) in the next section. Enterprises/grantees/local exporters Elixirs staff interviewed 5 enterprise key informants and 4 grantee key informants. They all provided positive feedback to the project implementation and shared their perspective for further developing and supporting the links between the farmers and local partners (PAFO and DAFO). The local enterprises wanted to know more about the project and the mechanisms to link them, together with the market and working approaches of the project. The companies also wanted to learn how to support farmers, make contracts to support prices and the system for exporting quality goods. The Mai Savanh Lao Company (MSL) is an example of one of the project’s grantees. MSL used their two grants, valued at $99,500, to train 137 farmers in sacha inchi nut cultivation and organic methods so that the MSL brand can ensure organic and Halal certification of their products. They used CLEAN funding to procure equipment to increase their production of oil, dry powder, and roasted nuts. This has helped to build interest in sales in SE Asia, Europe (France and Italy) and China. CLEAN also facilitated company representatives to attend the exhibitions in China & the U.S. In March 2020, representatives travelled to the US for an exhibition which was cancelled. However, they were able to travel and set up networking opportunities with U.S. based buyers, to increase MSL exports. With the COVID-19 lockdown, their profits were reduced but in 2021, they are already seeing a rebound and will be sending products to the US and Europe, as well as fulfilling an order for oil to Malaysia. MSL and the other grantees have recommended that CLEAN provide additional training support to farmer groups to meet projected market demand; however, this also depends on being able to maintain and establish market links. Mobile traders While villagers in FGDs identified four mobile traders, however, only one mobile trader agreed to be interviewed. The other three traders refused because they move around the provinces all the time and could only be contacted by mobile phone; since they wanted to see an authorization letter and this was no available by phone, those traders refused. The mobile trader who agreed to the interview identified how she and her husband buy a variety of products including cabbage from three villages in Vientiane Province. She resells the products on to other middle people in the district, who then sell to other traders, with the price increasing with each transaction. CLEAN staff Elixirs staff interviewed key people from CLEAN Office in Vientiane. The team planned to interview six key informants but was only able to meet with three people; Elixirs field staff were not able to meet the Clean Project staff in Pakse and the former agribusiness officer had recently resigned. These CLEAN staff in Vientiane were able to provide their perspectives and additional meaning to the data from the BBS. Their assessment of effectiveness is that they are just starting to train people but the challenge is that training often involves several sessions to the same people which challenges their ability to reach numeric targets. While achieving targets has been slow, the quality of the project implementation has improved and is increasingly showing itself as a model for the relationship between markets and quality production. The project model seems to be showing its effectiveness as the project and DoA are working more closely together and agriculture staff at all levels are understanding more about the 58 relationship between market access and production. During the MoU process, there were more opportunities to meet and discuss and starting in December 2020, the Interim Management Committee was set up which included representatives from divisions of the DoA, from PAFO and from DAFO. The IMC meetings are organized every six months and discuss the success factors and challenges in order to lead planning. In addition to technical capacity building of farmers, the project provided training to province and district staff to expand their understanding of GAP so they can train others and increase their roles as GAP inspectors for the GAP certification of farmers groups (see Annex 3. GAP Certification Process). The implementation of ICS and GAP promotion has increased the interest of domestic market in buying these products; these customers include of high-end supermarkets, restaurants, and hotels for GAP-certified, organic, and safe products (coffee, cabbage and sacha inchi). Currently, the domestic firms who trade in cabbage in Vientiane Capital and southern Laos have been buying from farmers and due to refrigeration, are able to store cabbage for sale to Thailand while waiting for a good price before selling. Farmers groups have been getting formal registration which indicates that they are GAP and ICS compliant. There are now 81 self-managing farmers groups, of which 19 were recently able to complete applications for in-kind grants, for which they will receive training and follow￾ups on the use of this equipment (such as cassava chippers and water pumps). CLEAN staff interviewed felt that it might be too early to talk about sustainability as they are just starting with so many activities; however, the expansion of farmers groups and markets indicates that many farmers, in learning and practicing new methods, have become more confident in these techniques and are likely to continue them. Since the DoA understands the market and production relationship, this area of collaboration will continue. CLEAN staff recommends more time for the project in order to achieve its goals for the number of farmers reached as well as establishing the markets for increasing production in order to ensure quality inputs and sustainability. Short- and longer-term monitoring is an important component since, as farmers learn new methods, they also might need problem￾solving assistance when they attempt to try out their new skills. Farmers groups receiving in￾kind support will need follow-up to make sure about the use of the equipment and how this benefits farmers. d) Findings from the FGDs In total, 16 FGDs were done in eight villages, (see Table 5). The FGDs included two groups, one men’s group and one women’s group in each of village. This allowed women to speak more freely and explain what they understand of the project work and how they work with their families for the project. A total of 34 cassava farmers, 28 cabbage families, nine sacha inchi farmers and 37 coffee farmers attended the FGDS giving an amalgamated total of 98 people. The selection of the participants was purposive, as farmers were invited based on the commodity that their family has been producing with the project, and opportunistic as the FGD facilitator worked with the village administration to invite people who were available in the village. After the FGD, the facilitator and note takers summarized their notes and later typed these summaries into an Excel file. Most of the members of the groups reached a consensus although a few other opinions were expressed which are summarized in the table below. The semi-structured questions used in the group are in Annex 4. Guiding Questions for FGDs. In addition to the positive results noted in the FGDs below, there were only two unintended consequences identified: 59 1. Farmers felt they were not supported once they tried new techniques as there was no follow-up by DAFO. Examples, farmers used organic methods but lost significant cabbage crop to insects, a farmer group planted too many cabbages at the same time resulting in loss of cabbages due to no cold storage facility. Large number of cabbages produce resulted in low price for cabbages. 2. Because farmers expanded their cultivation, often they had to hire more labor and so some farmers took out loans. Some of these farmers were concerned about paying back the loans, either to the bank or for village revolving funds. The farmers did pay back the loans. Because the activity was not undertaking direct activities to support access to finance, these results were not captured. Cabbage group discussion: Farmers tended to grow their both white and Chinese cabbage on dry fields after the paddy rice harvest and carry the water for their cabbages unless they were able to dig furrows from a higher water source. Since the land had previously been used for rice, it does not have chemicals in the soil because farmers don’t use chemicals on rice for their own consumption. With the project, they have learned about the benefits of “clean” farming where approved chemicals can be used when necessary (such as for insect infestation) and not sold until after a “wash out period” of two weeks so that no chemical residue is left on the plant. However, this method cannot be organically certified. Farmers grow their crops as chemical-free as possible by using compost or manure for the cabbage and, using the principles of integrated pest management (IPM), by picking off beetles and other insects from the plants rather than use pesticide. However, if the cabbage field is big or the insect infestation is large, they will use a Lao government approved pesticide at an appropriate stage of the insect life cycle. Farmers feel that if they do not use pesticides when there is a large insect or caterpillar infestation, that the pests will consume the entire cabbage crop. The FGs noticed that cabbages using composting for fertilizer grew slower but were smaller in size but better looking than those grown with chemicals. The farmers buy cabbage seeds from local shops. The most common varieties they grow are from Thailand which produces larger heads and looks more attractive; the other variety is from Vietnam. This produces smaller heads, which have been in more demand over the past year, as they cost less. Cabbage is not afflicted with particular diseases but farmers have losses due to insects; farmers have noticed that there are more insects on cabbages where they have used manure for fertilizer. The most common insects are aphids and caterpillars, which they try to pick off the cabbages and drop in hot water to kill them. The cabbage producers sell their crop to traders who come to the villages. The farmers wash, dry and package them, either next to their fields or they sell them individually to traders; they see this as more convenient than waiting for other farmers in their farmers group to collect their cabbage which would enable them to all be sold together. As there is no cold storage, the farmers have to sell their cabbage as soon as possible after they pick them. The members of the farmer groups have received some training from DAFO but no follow￾up on their farms. In one FG, only the group leader went to discuss the situation with DAFO. The following tables are summaries of the 16 FGDs which were conducted in 8 villages. The questions were divided into each of the commodities. Table 26: Summary of the discussion with cabbage farmer groups65 65 Source: MTE-BBS in March 2021 60 Factors FGD summaries Social: The lives, within their families, has improved. The benefits of organic agriculture also extend to their families as they are using less pesticides on the cabbages that they eat. They have increased income, which they have used for investing in seeds for the upcoming planting, home improvements and education for their children. Women’s work has remained the same, as men do heavier work; however, one FG said that they have more work as they have to learn to use the new methods. Children help their families with farm work on weekends and holidays; the families make sure that this does not affect school attendance or health. Families discuss together about cultivation, buying materials, and whether to hire labor. Volume and value of sales When farmers plant a lot of cabbage, they have noticed that the prices are low. Expenditure: Expenditure depends on whether they clear land, the size of the land and labor. The range is 5 – 6 million kip/ 3 rai of land in rice fields, and up to 15 million kip for all capital costs. One FG said they had not received training in business management so don’t know how to track expenses Markets: None of the groups are aware of international markets as they sell to traders Market access since project started: They still use previous sales methods to both local (within village/district) and to traders who come from other provinces. During the COVID-19 travel restrictions, the number of traders went down as they could not come from outside the province and those within the district could not enter the villages. As a result, both prices and income reduced. They could not sell as many cabbages as they had planted and had to destroy the surplus heads. Knowledge of GAP/ICS/farm management 4 FGs said that they learned about GAP and have applied some methods such as organic farming and use of pesticides only as necessary. Otherwise packaging and trade has remained the same as the traders are not concerned about GAP. Credit: All the groups borrowed from banks or village lenders. Three FG members said they borrowed from the Agriculture Promotion Bank (APB) and bought seeds on credit and paid after harvest. The interest rates varied from 1.5% to 11% per year, depending on the group and the collateral. The village bank interest rates are about 3% per year. One group said they were afraid to borrow as they were worried that they could not repay the loan. However, the other groups said they could repay the loans; the village bank loan repayments were the most flexible about returning the full loan, but it did require families to repay the interest every month. Satisfaction/good points/challenges Generally, FGs who received project inputs were satisfied because their cabbages are better and methods safer for their families. Members of three of the FGs said they would continue to use organic methods. The groups which received training and practice clean agriculture methods feel that they can teach others within the village and in surrounding villages. Major challenges: 1) fluctuating prices, from high to low depending on the season and the number of farmers planting; 2) lack of initial training in one village, lack of follow-up with 61 Factors FGD summaries DAFO so problems and solutions could be discussed; 3) groups aren’t sure what solutions to apply to challenges. The recommendation from FGs Farmers group strengthening: FG said they feel they can share lessons with other groups and within villages. This should be another level of ToT – by strengthening the ability for the group leader to teach, and mobilizing others in the group to use peer to peer education, it would reduce the workload on DAFO in follow-up. Peer to peer can also be extended to social media. DAFO should be more responsive to farmers questions and follow-up. There should be a WhatsApp group for cabbage farmers. They can take photos of pests and communicate with DAFO, PAFO or even with NAFRI, central level trainers. ICS should be strengthened within FG where the FG leader could inspect products before shipment. Then DAFO inspectors would only need to do random, unannounced spot checks. However, traders are not likely to prioritize GAP products. This system could be put in place as practice, and as more companies are involved in collecting cabbage directly, then the companies can insist on GAP compliance and ICS monitoring from the FG. Developing cold storage facilities for FGs would help to keep cabbages for a few days until a company can collect a larger number of cabbages from an FG or the group can wait for a better price. This need not only be refrigeration but could also include cold cellars with racks used by the FG Cabbage farmers within FG should discuss and plan on planting schedules to even out supply in order to get better prices by ensuring that there isn’t an oversupply of cabbages to the market. However, this requires other villages to employ similar methods as well as needing traders to prioritize GAP certified products. Cassava group discussion: Table 27: Summary of discussion with cassava farmer groups66 Factors FGD summaries Land preparation and clearance: Use land already cleared for cassava, or land used for other crops (such as coffee orchards). When they prepare land for cultivation, they do not use chemicals but use chemicals only after they have planted the cassava Fertilizer: Three of the four groups didn’t use fertilizer. One group said they mix manure and compost into the soil. All of the groups said that they don’t use chemical fertilizer at all Varieties of cassava: Use two varieties which are available for sale: stems from Thailand grow well and have bigger roots but more prone to plant and insect disease; the variety from Vietnam yields smaller roots but is more resistant to pests and disease. They have not received any stems or varieties from the project. 66 Source: MTE-BBS in March 2021 62 Factors FGD summaries Plant disease and insect diseases: They know about CMD. Most also have problems with insects (black and white aphids, fungus and beetles which attack the roots. 1 group said they do not have problems with CMD. Solutions for plant and insect diseases: One group said they didn't do anything if the problem was minor, only infecting a few plants; the other groups said they cut the infected plants if CMD and burned them. They pick off insects and kill them if only a few but use pesticides if a larger number. Preparation and sale to market 2 groups dry and slice cassava and get a higher price; one village group (2 FGs) received a cassava chipper and tarps to dry the cassava. 2 FGs said they only sell fresh cassava as they have no areas to dry cassava. Sales: They sell to traders who come to the market individually as farmers have cassava at different times and they have limited storage space. One group said they had previously sold cassava to the Indochina Company at a good price, but then sold to a trader who undercut that price. After a while, the Indochina Company stopped coming to the village and this group wants to start reselling to the firm. They can store the cassava roots for a while with low rates of spoilage GAP/ICS/Training: 2 FGs said they joined the training; one group did not have training and one group said only the group leader attended. They felt that the training provided knowledge that improved their cultivation and enabled them to provide better products to the market. Technical training provided useful information on soil preparation because soil would become infertile 3 – 5 years after sole use of cassava Follow-up: For the groups that received just the training, there was no follow￾up. One village which received a chipper did receive follow-ups from DAFO 4 times which included one time when they received the cassava chipper. Impact on families: Increased income which can be used for buying stems for the next planting, and on education for children. There is no change in the women’s work because they divide the work as they had done previously. Children help on weekends as long as there was no effect on health or school attendance. The families decide together on varieties to plant, use of organic methods vs. chemicals. In one woman led HH, she made the decisions, and in another HH, the elder decided. Volume and value of sales: Increased prices for both fresh and dried cassava. One FG was interested in selling to a company. Expenditure: Expenditure had increased because they are using more new methods taught by the project and have hired more labor. 2 groups estimated their expenditure for opening up new land at 1.5 million/rai, the cost of 350 bundles of stems @ 5000 kip/bundle and additional labor at 80,000 kip/day. In one FG, the project hadn’t started so they haven’t learned how to track expenses International trade: When they could sell to Thailand or Vietnam, the prices were higher. However, since the COVID-19 travel and trade restrictions, they can’t trade directly and prices have been lower. They can store dried cassava for a while but they try to sell both fresh and dried as soon as 63 Factors FGD summaries possible to prevent spoilage. Some firms or middlemen have come to villages since the lockdown ended Domestic trade: For 2 groups, no change in sales; 2 FGs have year-round domestic sales and to firms Knowledge Mainly technical knowledge on cultivation, prevention of diseases. 2 groups said no change Credit: 2 FGs members borrowed from APB (large cultivated area), 2 FG borrowed from village fund (smaller areas). They do not have business plans but look at ways to expand cultivation areas and would like to get improved disease-resistant varieties when available. For APB, interest is repaid every three months for village funds, and then the borrowers paid back the lump sum at one time in February 2021. Good points of project:  Improved knowledge and increased income  In August 2020, 1st established group said they got follow-up and advise Weak points/ challenges: No regular monitoring, no provision of equipment or varieties. For groups that received training, this occurred after the harvest so they couldn’t use the knowledge Sustainability: They are keen to share their experiences with others who haven’t joined the project. They have learned to increase productivity and stagger planting so harvesting is spaced out over the year Coffee group discussion: The groups are all planting on original soil because coffee plants are established and producing. Varieties are Catimor and Arabica. Catimor grows quickly and produces a large amount; however, it has a short life and is more prone to pests. Arabica is slower growing but doesn’t have consistent production. Catimor gets lower prices, Arabica higher prices. Arabica cherries ripen at different times so the harvest season is longer (assume this would be more difficult to sell in a group). Table 28: Summary of the discussion with coffee farmer groups67 Factors FGD summaries Pests and diseases: Beetles eat the trunk of the bush and termites destroy the roots. Some kinds of aphids and worms eat the leaves and beetles eat the cherries. Snout beetles and ants destroy the cherries. DAFO advises to cut down burn trees if there is evidence of a large amount of disease. DAFO has recommended planting Catimor. Training and follow￾up: Technical training – cultivation and maintenance of bushes in July 2020. One other training on fertilizer, and organic composting Impact on lives: No change in lives. They use the income to buy food, hire workers, pay off debt. Men do harder work than women though one woman said she worked harder than her husband. Families say that they plant the coffee together. Children help on weekends and holidays. Sales: All 4 FGs sell fresh cherries with less processing and sales of dried beans. Farmers said that they encountered no spoilage as 67 Source: MTE-BBS in March 2021 64 Factors FGD summaries they could sell the fresh cherries quickly. Traders come to the villages and members of the group sell individually when they have harvested beans. Members of these FGs do not have storage so they want to sell their beans soon after harvesting. They have learned about organic methods and GAP certification. One group had not received training on GAP yet; Another group had training but hasn’t had the time to start yet. The Coffee Association had come to one village to discuss buying coffee but never not returned Volume of sales: As coffee trees are old in the orchards of groups, the volume of coffee cherries has decreased; however, with lower volume, the price has increased. The FG in Lao Ngarm district noticed that younger trees are producing less and believe it’s due to warmer temperatures. Expenditure: Ranges from 1.5 to 8 million kip. In one group, the investment ranged from 10 to 50 million kip, and depends on the size of the land, and whether they hire labor for the harvest. International / domestic trading They are not aware of international trade because they sell to traders and don’t know where the coffee is finally marketed. Domestically they sell to traders as before Knowledge from training Learned about pruning, and the use of compost. The other three groups still use traditional methods. All farmers asked for more follow-up by district and by the project in order to improve their techniques. Credit: Usually don’t want to take loans because afraid not able to repay the loans; if they borrow, they ask relatives and friends and repay after harvest. Some people borrow from village fund. They don’t have a business plan because the market price is determined by the trader, so it is difficult to make a plan. One person borrowed 35 million kip and if he has problems repaying it, he renegotiates the loan with the bank; however, he realizes that he loses money on the interest repayments Good points of the project: Can save money, get a good price based on improved technical knowledge. For one FG members who said that they had not yet joined the project activities, they don’t know the benefits for the project yet. Weak points of the project Started after harvest, no follow-up, no coordination, prices of coffee beans changes Challenges: Pests, floods, and droughts. Sustainability: Farmers who got the training can continue to use the methods. They understand more about the market but they have not used the advice of the project to the full extent Lessons learned/ recommendations: More training on coffee tree maintenance, and markets. 65 Sacha inchi group discussion: The sacha inchi farmers received training and materials from MSL, who eventually trained 137 farmers. MSL guarantee pricing and pick up the nuts at the village. Unlike other commodities, they have of following MSL standards of using organic growing so that the nuts and products can get organic and Halal certification. Some farmers had previously planted sacha inchi, but the previous traders did not return and they didn’t know where to market the nuts so stopped growing them, but they are very satisfied now because of the training and that MSL provides a standard price (15,000 kip/ kg). They have now been learning how to grow according to MSL standards and took an exposure trip to another village to learn more about these standards. One of the advantages of the sacha inchi is that is easy to grow and take care of. However, if they don’t have a water supply, the nuts are smaller. Worms can eat the outer seed casings; but they can be picked off easily. Sometimes pigs eat the outer shells when the plants are young. Some farmers have expanded their gardens and have bought their own materials. They don’t know about outside markets now because MSL buy all their product. Table 29: Summary of the discussion with sacha inchi farmer groups68 Factors FGD summaries Impact on family lives: Women say they work harder because they spend more time attending to the vines. Men say that there is no change in women’s work. Everyone in the family helps and children help when not in school. Working as a group and sustainability: One of the groups has 18 member families. They like working together as a group because they motivate each other. Sales: They have a contract with MSL who gives them a standard price of 15,000 kip per kg for the nuts Volume of sales: It is still low as they have just restarted planting sacha inchi Expenditure: They don’t have this recorded as the company provides most of what they need. Some farmers in the group have bought extra fencing to expand their production 68 Source: CLEAN MTE-BBS in March 2021 One coffee grower in Thateng said, “I grew coffee and later I dug up the coffee bushes to grow cassava, because the coffee bushes started to have many problems, with low berry production and insect damage. There was not enough fruit to sell and the price per kg fell to 600 to 1.000 Kip which was not enough to cover my expenditures. I decided to change to grow cassava and I found it has a better price, better market support and the traders live close to my farm. I am satisfied now, as the cassava prices are around 1.300 to 1.700 LAK per Kg.” 66 Factors FGD summaries International/ domestic trading Previously they had sold to traders from Vietnam and Thailand but due to the closure of trade across borders with COVID-19 restrictions, they are selling only to MSL Knowledge from training The sacha inchi farmers in this group had previously sold small amounts of the nuts to Vietnamese or Thai traders. The training has helped them to increase the quality of their product, plus they use manure and compost and organic methods for pest control. Credit: They don’t need to borrow money because the company provides everything. Some farmers also plant coffee and they have borrowed money for coffee cultivation from the ACLEDA bank. Good points of the project: They receive good follow-up from MSL, and they can ask them questions when the staff collect the nuts and pay the farmers. Weak points of the project None Challenges: None Sustainability: Since they have planted sacha inchi in the past, they do have some familiarity with the process. They had stopped planting it because the prices they received from traders was low and inconsistent. They feel that it’s now likely they can continue. Lessons learned/ recommendations: They have found that group work is good for their production. They feel they can come together as a group to discuss and solve problems. Working as a group motivates the members for production. 3.3 Assessment of project progress The CLEAN project has met many challenges from the start of implementation. Before the project started full implementation of project activities, it used the time for preparation and implementation to establish good working relationships with partners. With project partners, the project developed training materials and trained the province and district trainers and inspectors to establish the foundation of the project. Once the project was authorized to start implementation, the materials and human resources were available. In addition to the delayed signing of the MoU which gave Government of Laos approval for project implementation, additional challenges also delayed project progress; these challenges including flooding in target areas in July 2019 and the COVID-19 travel restrictions in March through May 2020 which limited on-site training activities. The project has focused on developing the model of integrating the quantity and quality of farm production with understanding and accessing markets. There are definite signs of success:  The CLEAN project has provided a strong model for the growth of four commodities, and for partnerships between government and private partners and farmers. First, through CLEAN’s training, farmers understand that they can make positive changes to their land so that they do not have to continually open new land but can learn newer ways to ensure 67 productivity through soil preparation, particularly in organic methods, and soil conservation so that soil fertility is preserved.  Farmers are learning new methods to increase productivity and are learning about the value of increasing quality to improve sales and potentially to achieve higher prices for their products. As a starting point, under this activity, CLEAN worked with master trainers from the central government, who developed training materials and lesson plans for a PAFO and DAFO capacity building training. Once these materials were developed and the government trainers were in place, PAFO and DAFO staff trained the farmers from CLEAN’s value chain. Through this process, out of 814 respondents, 692(85%) of them affirmed that they attended training from the CLEAN project and applied one of the techniques demonstrated on their farm. Out of CLEAN’s pool of trainees, the application rate of the technology differs from one production system to another as cabbage producers did apply new techniques in a higher proportion (67.7%) when compared to coffee growers (47%). Across the four commodities, 258 respondents out of the 391 people interviewed who attended the training (66%) affirmed that the training sessions they attended were useful. According to Custom Indicator (CI-4), 93 government officers have been trained in order to train others in quality standards and food safety.  Government partners in agriculture have been learning about the ways to work with farmers to understand GAP and ICS so that the quality of their products is improved which opens up higher end market options for sales.  Farmer groups are being formed and starting to develop methods of working together; this in particular is not an overnight change as farmers have been more accustomed to working with middlemen and vendors who come to villages. Farmers are starting to understand the benefits of farmers groups, as these groups can support each other in capacity development (business planning, accounting, support of improved agriculture practices), procurement, group sales and transportation, bargaining power to negotiate prices and crop management.  Farmers and government partners are establishing relationships with private enterprises and grantees for improved practices, such as the relationship of sacha inchi farmers and Mai Savanh Lao. However, as noted above, the growth in reaching a large number of farmers is slower due to both challenges and shortened time of implementation and the need to conduct several training sessions to current farmers. It’s not likely that the project will be able to reach its target of 14,877 farmers by the end of the project period in 2021. The project progress, based on the actual counts from semi-annual reports, including the most recent semi-annual report of Oct 2020 to Mar 2021, towards meeting the indicators is in Table 2. 4 Conclusion and Recommendations The project has focused on developing the model of integrating the quantity and quality of farm production with understanding and accessing markets. There are definite signs of success:  The CLEAN project has provided a strong model for the growth of four commodities, and partnerships between government and private partners and farmers. First, through 68 CLEAN’s training, farmers understand that they can make positive changes to their land so that they do not have to continually open new land but can learn new ways to ensure productivity through soil preparation, particularly in organic methods, and soil conservation so that soil fertility is preserved.  Farmers are learning new methods to increase productivity and are learning about the value of increasing quality to improve sales and potentially to achieve higher prices for their products.  Government partners in agriculture have been learning about the ways to work with farmers to understand GAP and ICS so that the quality of their products is improved which opens up higher end market options for sales.  Farmer groups are being formed and starting to develop methods of working together; this in particular is not an overnight change as farmers have been accustomed to working with middlemen and vendors who come to villages. Farmers are starting to understand the benefits of farmers groups, as these groups can support each other in capacity development (business planning, accounting, support of improved agriculture practices), procurement, group sales and transportation, bargaining power to negotiate prices and crop management.  Farmers and government partners are establishing relationships with private enterprises and grantees for improved practices, such as the relationship of sacha inchi farmers and Mai Savanh Lao. The project has worked with partners on two private partner partnerships, which is the start of the model for how CLEAN can facilitate actions. The following conclusions and recommendations are based on DAC criteria and the questions posed in the terms of reference. Relevance:  How accurate have the critical assumptions underpinning the project proven to be in practice? Is the Theory of Change still appropriate for the project goals and objectives? The critical assumptions of the project are that government partners and farmers have the interest and skills to increase the integration of improving small-holder production systems, improving market systems and infrastructure. These three components work together to improve sales of agriculture products through improving food safety and compliance with national and international standards. From the information provided by the farmers in the MTE-BBS, they confirm that they have been able to learn new skills and understand more about the relationship of their production with sales; In FGDs, farmers feel that the interventions are important in their own lives, which has increased their interest as sales have improved not only through the quantity of the products they grow but also in the quality of their products. The farmers who have joined training are satisfied with learning about organic farming methods, how to apply GAP standards and to join farmer groups to improve the management of their farms and technical capacity in farming. For government partners, the understanding of integrating market, trade and production have increased throughout the project period. At the beginning of the project, most agriculture staff were accustomed to working with farmers on the production side but had not worked with farmers on the market side so their understanding of the link between marketing and production was limited. The understanding of the relationships between production, markets and food safety compliance has been evolving from the start of the project when most agricultural staff were accustomed to working with farmers on the production side and did not understand how marketing and production are related. 69 On the production side, farmers employing organic methods of fertilizer and appropriate chemical pest control have indicated that they feel these methods are safer for both the consumer and their own families.  Are there unanticipated opportunities or challenges that may require revisions of the project strategy or redirection of project resources? The project has developed a management system with the DoA and representatives of the divisions within the DoA. This has provided a wider core group of people who understand the project and integration of production and marketing. However, groups involved with commerce and trade are still not actively included. While CLEAN has been working directly with DoA and the government partners at province and district levels and from there, directly with the farmers and farmers groups, there might be more opportunities of including INGOs and non-profit associations (NPAs) in networking. These civil society organizations also have projects in agriculture through the lens of nutrition￾sensitive agriculture but also may have lessons learned in agriculture management, development of knowledge hubs (on-line and paper libraries) for sharing resources and monitoring. Travel restrictions during COVID-19 lock down and inability to access some villages due to flooding or poor road infrastructure, increased reliance of farmer groups to monitor the quality of their products through technical and GAP trainings, as well as exploring coordination of farmer groups through WhatsApp and other methods of social media for those farmers who can access 3G or 4G services through their cell phones. The MTE-BBS did not collect data on the farmers’ level of education. Farmers with higher levels of education are probably more likely to access written materials and have less difficulty in applying the knowledge in their farming. The delayed start of the project and unexpected weather events slowed project progress and the number of farmers trained and included in project activities has fallen very short of the anticipated target. With the current total number of 2023 beneficiaries which included 1918 farmers, 93 government partners, 6 private enterprise staff and 6 grantee staff reached by the project, the project implementation to reach the total number of 14,877 direct beneficiaries is slow but may pick up momentum as the CLEAN project gains leverage and expansion of existing outcomes, especially with a longer period for implementation Recommendations for increasing relevance: Governance Improving the perception of the importance and application of the project interventions so that they are more relevant to the lives of farmers should involve both the interventions directed towards farmers as well as the governance structure of the village.  To improve relevance at the village level, governance systems including village leadership would expand the understanding and enlist the services of the village administration in organizing the activities and promoting the project. A representative of the Farmers’ Group could also attend village administration meetings in order to talk about the successes of the project which may encourage other farmers to join farmers groups.  Expanding governance within the village would also lend support for farmers groups to increase their bargaining power with enterprises and especially traders, as farmers, by negotiating individually, would not undercut the prices of other farmers and prices that farmers groups had established. 70  Increase cohesiveness in farmers groups by encouraging farmers to be involved as co￾facilitators in training, especially for involving farmers with higher levels of education who can province peer support to farmers with less education and confidence.  Are Monitoring & Evaluation and Learning (MEL) tools and tracking systems working well and identifying relevant project impacts? Monitoring is being done through government partners, trainers, and CLEAN staff, using paper monitoring forms. CLEAN has piloted the use of ID cards and QR codes with Mai Savanh Lao’s farmers.69 The project intends to roll out these cards so that monitoring can be done through electronic means with farmer groups in the future. Paper-based systems have the advantage of allowing the double checking of data; however, use of the data requires staff time to enter the data from the paper files. The project does have a Performance Indicator Reference Sheet (PIRS) in which the indicators are defined and how the indicators should be measured, along with the annual targets. Using the data for learning is still being developed through the regular establishment of IMC meetings for reviewing project activities and discussion of results. Many organizations have been using applications such as KoBo (an Android OS application built on the ODK platform, which can be used on a smart phone, tablet, or desktop computer) for collecting quantitative data during monitoring and evaluation activities. This allows them to access data quickly by downloading the Excel files and also, as data is received, staff can view charts and tables of responses. Cassava producers are already using WhatsApp for group sharing of successes and questions to enlist the advice of other members of the group which does indicate that farmers can use their cell phones for accessing web-based content. Recommendations for Monitoring: Use of a mobile app like KoBo would help to decentralize data collection at the district and village levels and make the information more quickly accessible to the stakeholders so that they can use the data for learning. KoBo does have a function for being able to view charts and table summaries of the data collected as soon as forms have been uploaded. This would help to answer questions about not only which farmers have attended training, but would include information on how they applied the training so that they can make proposals for improving training methodology. Effectiveness  What factors are facilitating the project’s effectiveness? What factors are hindering it? Relationships with government and private partners have made project implementation more effective, as the project can draw on a variety of resources through strong relationships. As an example, the relationship with CIAT / NAFRI also leverages current research in the control of cassava diseases. Throughout the region, Cassava Mosaic Disease (CMD) and Cassava Witches’ Broom Disease (CWBD) reduces productivity of cassava, kills plants and infects other plants. There are no cures for these diseases, other than identifying infected stems and destroying them. Initiatives such as “Future Stems” in which CIAT will be providing technical support to the Lao Cassava Association will help farmers feel more secure about destroying infected plants as they will be able to receive fresh and non-infected stems from CIAT or the Lao Cassava Sociation. Disease resistant stems are also being researched with possible 69 The use of IDs and QR codes was noted in the semi-annual report (October 2019 to March 2020, page 24) as a method to streamline monitoring information and collect the personal details of members of farmers groups. 71 distribution to farmers, as well as technical advice on maintaining soil fertility in older fields that is also likely to result in increased production.70 Training The cascade method of training has been able to provide training at each level of the agriculture sector, with 93 trainers who have improved their own agricultural technical knowledge and marketing as well as building their skills as trainers. Of the 692 farmers/trainees who received training, over half (391) have been able to apply the new skills and knowledge to their production. The DoA developed picture-based training materials in simple Lao, which farmers can use as reference materials. This has included handbooks and posters. For cassava producers, the WhatsApp group helps to continue a link between farmers and trainers. Factors limiting the project’s effectiveness include:  Scheduling training at an inconvenient time, or a period when farmers cannot use the knowledge or skills. Because of the delayed signing of the MoU and inability to access villages for training due to COVID-19 travel restrictions, the change in scheduling could not be avoided. However, farmers learned about subjects such soil preparation and composting which will benefit their planting, after they have harvested the crops.  Farmers in FGDs and noted in the KIIs by DoA, that follow-up, monitoring, and coaching farmers is not done as often as necessary. From the MTE-BBS, farmers did note difficulty in application of their new knowledge.  Although this was not assessed in the MTE-BBS, education level, literacy and numeracy may affect the farmers ability to access and understand training materials, keep records, and calculate the weight and sales of their products.  After training, not all the farmers have organized farmers groups. The DoA coordinator indicated that this may be due to lack of follow-up and coaching farmers in applying the skills. In addition, Farmers groups have not started to work cohesively as some farmers admitted in the FGDs that they are still selling individually when traders go to the villages, so they set their own prices with the traders individually rather than use the bargaining power of the group. They also have not organized growing periods so that they can share transportation and send their products to enterprises. Market access In the area of markets, the example of MSL in sacha inchi cultivation and production of products does serve as a model of how the market and production can work. MSL has trained the farmers using the guidelines for organic and halal products to increase its marketability. The farmers get a good and established price from the contract set up with the company, and the company is consistent with pick-ups and providing follow-up to the farmers. On the market side, the CLEAN project has supported MSL in establishing international as well as regional and domestic markets through supporting attendance and demonstration of products in trade fairs. By stimulating demand for the project, through increased sales, the farmers have increased interest in improving the quality and quantity of their products as they know there will be a market for sales. For cassava, the training has helped farmers to identify diseases, and understanding the importance of destroying infected stems and planting only non-diseased stems in order to prevent the spread of Cassava Mosaic Disease (CMD) and Cassava Witches Broom Disease (CWBD); in the future, through the collaboration between the Lao Cassava Association, CIAT 70 Interview with Dr. Jonathan Newby, CIAT Director. 72 and CLEAN funding, more fresh stems will be produced through multiplier tunnel greenhouses. Through basics of Integrated Pest Management (IPM) farmers have learned to identify pests and how to destroy them using organic methods and when to use chemicals appropriately. Coffee: The project has been promoting Lao coffee through various events and demonstrating to farmers that they can grow and meet the demand for specialty coffee. The project used an innovative approach for a cupping event during COVID-19 travel restrictions through a virtual cupping event in collaboration with the Coffee Quality Institute (CQI).71 Cabbage has high demand and FGs have started to produce GAP-certified cabbage. CLEAN has been developing a public private partnership (PPP) with BBA Company and Sky Vision for improving cold storage for cabbage. Factors inhibiting the effectiveness  Some farmers who attended training have not yet set up FGs; the project coordinator attributes this to lack of ability for PAFO and DAFO to follow-up on activities of the farmers. However, there is no information on the issues that have resulted in the farmers not setting up FGs.  There has been an inability to reach targets which may be due to the need for multiple training sessions to the current farmers which slows the ability to reach out to new farmers. In addition, there are only a limited number of trainers who also have other responsibilities in their government positions.  Lack of cold storage for cabbage farmers limits their ability to wait and sell their cabbage at higher prices; whereas, coffee, cassava and sacha inchi farmers are able to store their products in dry storage areas. In 2020, the training was delayed due to COVID-19 related travel restrictions with reduced access to villages. As a result, the training in improved farming practices took place after the restrictions were lifted, and they occurred during the dry season, after the 2020 harvest. Farmers were not able to use new methods for cultivation until the next growing cycle. The survey included questions on planting and harvesting, giving an idea of the best times for training. Recommendations: What should be improved in order to make the project implementation more effective? Decentralizing activities by increasing capacity at village and district levels  As mentioned above, increase capacity of village governance by including village administrators and village chiefs, will assist in organizing activities, overseeing implementation and results, and increasing the interest of other farmers in joining the farmer groups and training.  Task-shift some monitoring and reporting activities to the village level through training the leaders of the farmers groups who have the necessary literacy and numeracy skills to assist farmers with less skills in these areas.  Increase regularity of farmers group meetings with specific agendas. These meetings can also be helpful to review training lessons to refresh farmers’ memories. This will be especially important as farmers received training in some techniques after harvesting, so their groups might not remember what they learned for soil preparation and planting.  Develop a knowledge repository that farmers can access online. In addition to setting up WhatsApp groups for all farmers groups, the knowledge repository would include both the 71 Noted in the CLEAN semi-annual report Oct 2020 – Mar 2021. 73 written and audio-visual materials that farmers can download to their phones. The WhatsApp groups can inform farmers when new content is available.  For farmer groups which already have made progress with production and in their group development, develop an agenda for them so that they can receive exposure trips from other fledgling farmers groups so the newer groups can see the model for setting up a group and its benefits.  For setting up training and working with farmers, PAFO and DAFO should make an agriculture calendar/ map to understand when farmers are planting and harvesting and when are the convenient times for training, and in particular, when to organize the training before the next growing cycle. PAFO, DAFO and the project should ensure that the plan is followed so that the villagers know when to expect the training and confirm with the village chief and FG leader on dates and location, so that farmers know to be available.  DAFO should plan more monitoring visits after training so that if villagers have problems with using the skills and knowledge, the DAFO staff can provide assistance. In addition to monitoring, DAFO, PAFO and CLEAN staff should develop the skills of farmers to share their knowledge, both formally and informally with other farmers. Farmers in FGDs indicated that they are already sharing knowledge from training with their family members, neighbors, and relatives and this should be built upon to become a peer to peer process. Markets: There are some non-profit associations which have been working with farmers groups and in developing local markets and collection points. Visiting groups such as GCDA and AVSF may provide some lesson learned which might be useful for CLEAN. Efficiency  Are intended beneficiaries receiving services as planned? What are their experiences and recommendations? The roll out of training to farmers has been slow, as it requires that government partners improve their capacities as trainers and it takes time for trainers to feel comfortable with training. Training of farmers has been delayed due to other environmental and outside factors such as COVID-19 travel restrictions and floods. As mentioned above, villagers are only slowly receiving training and when the training does not fit with the growing cycle, there is likely to be a loss of knowledge and skills over the intervening time period. Some farmers are also limited by lack of equipment for use in improved methods (such as irrigation) and improved seed varieties. However, as mentioned above, training trainers at each level, including peer support and mentorship, increases efficiency as the training can be conducted more closely to the village and farmer level. Farmers can develop personal familiarity with the trainers and extension services so that they can consult them when they have questions.  To what extent is the project being implemented as designed? The project model seems to be working as described in the theory of change. Agriculture staff and enterprises involved in the project are understanding more about the relationship between market and production. The numbers of farmers trained when trainers are not able to access villages has been slow due to COVID-19 restrictions and environmental factors. Farmers have not been able to access traditional methods of sales through middlemen during the COVID-19 travel restriction period, as farmers could not leave the villages and traders from outside the village could not enter the village. International trade has been slowed 74 because of border closures, which affects both access to supplies and equipment but also country transit time.  Is the project on track to meet with targets? If targets are not being met, why not? If targets are being exceeded, why? Do targets need to be adjusted? Recommendations:  As described above, the project is not on track to meet targets. However, CLEAN may be able to leverage existing activities and results by working to strengthen farmer group membership and formalize more groups, who will then be able to access further training and technical assistance. This recommendation is drawn from the following participant experiences and recommendations: Farmers recommend more training during the time period before planting and that DAFO should conduct more follow-up; it will also be important to refresh the farmers’ knowledge before the subsequent growing season of their product.  Improved effectiveness by improving capacity at village and district levels would also reduce expenses such as those for travel, lodging and Daily Service Allowance (DSA). Farmers could work with DAFO to expand monitoring with increased communication through WhatsApp. FG leaders should receive additional training and mentoring for strengthening training at the village level, and similar to the system of village health volunteers meeting with health center staff on a regular basis, the FG leaders could meet with DAFO staff on a regular basis for refresher training which they can bring back to their village and discuss solutions to issues.  Although farmers within farmer groups might have low literacy and numeracy levels, trials of this method with the heads of FGs can provide information on the possibility of using such a model. FG members can also work together to mentor farmers in the group on the use of new methods and management such as record keeping.  If farmer groups become more effective, it is possible that the DAFO can do some ‘task shifting’ of assigning some of the tasks of monitoring, some training, ICS monitoring to the FGs. Farmers teaching farmers through the peer to peer approach would also help to expand the network of farmers who could join farmer groups and CLEAN activities. This multiplier effect, as a result of cascade training, could help to scale up. The role of the PAFO would become one of performing spot-checks to monitor and follow-up on the FGs ability to do this. The effect of this task-shifting would help to free up more time for DAFO and PAFO to train new farmers. Sustainability  Does the original strategy for the project’s sustainability and lasting impact remain feasible? The original model of the project still holds and it is important to strengthen the model. Agriculture staff at all levels are interested, involved and enthusiastic as they have seen several models develop and have good results. However, the input of private sector partners would help to make the market activities more sustainable. Farmers, who have received training believe they can maintain the changes that they have learned through training and that they can train others. This is an important foundation for sustainability. If the market side can continue to be maintained, both market and production will support each other. An important relationship is that with Sky Vision which buys cabbage at a fixed price from farmers and then has the cold storage to keep the cabbage until the price for selling it to Thai merchants is good. 75  If not, why? Are additional efforts needed, or are other changes to the strategy recommended? Although farmers in FGDs believe that they can train others, they also should have additional training, using an activity-based approach, so they can advise other farmers. PAFO and DAFO should also train and monitor farmers and assess their skills as being peer advisors. Recommendations:  Include action plans with training, to follow a format of 1. What did the training accomplish? 2. How do farmers plan to apply what they have learned. Farmers could review this as a final session of their training where they can work together as a group to develop a business plan, calculate what funds they need, what exact steps they need to take.  Include training in peer to peer education so that farmers can tell others about what they have learned and what improvements they are making. Impact  How has CLEAN’s interventions affected targeted beneficiary groups? Are the project’s interventions having the expected effect on identified constraints? Farmers, who received training, have been able to improve their understanding of the skills required for business development, although the practice of working as an FG still needs strengthening in order to understand the benefits of FGs in establishing and maintaining market relationships. On the social level, all participants in FGDs had indicated improvements at family, education, and community levels. Farmer families have been able to use their profit to better their living situations, provide education supplies and ensure that children can remain in school, all through the additional profits generated. Currently, in FGDs, women indicated that they did not carry a heavier work burden; however, this should be monitored in the further to ensure that gender roles are such that men and women divide their work equitably. Recommendations:  The strengthening of farmer groups has the potential for improving the impact as they will take more ownership and also contribute to sustainability.  Continue development of FGs with consideration of how the farmers can help each other within the groups, such as reviewing lessons and helping with recording, business planning and calculation of volume produced and sales.  In future project planning, it would be useful to add social considerations to ensure that there are no other unexpected results which can affect family lives. Coherence Coherence is a more recent element added to the other five components of the DAC criteria, which defines how a project fits in with local and government strategies, policies and plans of action. The project does with a number of policy objectives of improving markets, leverage enterprise support and impacting policy. As mentioned above, three policies are in the process of revision through work with the DoA. Recommendations:  As policies are the framework for change at central and local levels, CLEAN should continue working with the DoA to develop agriculture and farmer group policies.  CLEAN has taken a lead role in developing a model on how public private partnerships can be put into action. There is currently a PPP policy drafted by the Government of 76 Laos in Lao language and is in the process of full approval; this would be a useful policy for providing structure to the continuation and expansion of the project marketing goals. 77 Annexes Annex 1. Sampling Size Tables by Commodity for MTE Annex 2. Annotated Bibliography Annex 3. GAP Certification Process Annex 5. Guiding Questions for FGD Annex 6. Guiding Questions for KIIs Annex 7. ToR Annex 8. Participant and Event Disaggregates Annex 9. Calculation of the Sampling Weight 78 Annex 1. Sampling Size Tables by Commodity for MTE BBS Table 30: Sampling size for cabbage commodity Commodity Cabbage total beneficiaries 454 sample size needed 208 total villages 32 # clusters to be selected 14 Actual sample 239 District total beneficiaries share # sample size needed actual # interviews Paksong 122 27% 122 65 Laongarm 48 11% 48 23 Kasy 140 31% 140 96 Vangvieng 97 21% 97 38 Hateng 47 10% 47 17 Table 31: Sampling size for cassava commodity Commodity Cassava total beneficiaries 858 sample size needed 267 total villages 29 # clusters to be selected 12 Actual sample 419 District total beneficiaries share # interviews needed actual # interviews Lao Ngarm 277 32% 86 141 Parkngum 172 20% 54 76 Sangthong 31 4% 10 16 Xaythani 71 8% 22 13 Thateng 307 36% 96 173 Table 32: Sampling size for coffee commodity Commodity Coffee total beneficiaries 407 sample size needed 198 total villages 17 # clusters to be selected 10 Actual sample 312 District total beneficiaries share # interviews needed actual # interviews Lao Ngarm 142 35% 69 115 Thateng 265 65% 129 197 79 Table 33: Sampling size for coffee commodity Commodity Sacha inchi total beneficiaries 177 sample size needed 126 total villages 46 # clusters to be selected 21 Actual sample 134 District total beneficiaries share # interviews needed actual # interviews Paksong 123 30% 60 87 Laongarm 11 3% 5 10 Thateng 43 11% 21 37 80 Annex 2. Annotated Bibliography These are documents, both provided by Winrock and from other relevant sources, used in the desk review. Documents provided by CLEAN:  EDC (2018) CLEAN Baseline Final Submitted to USDA. The initial commodities selected included sacha inchi, cabbages (both white and Chinese cabbage), citrus (oranges and limes), peppers and cardamom.  Rapid Asia (2019): CLEAN Baseline evaluation. This second baseline evaluation reviewed baseline data on the commodities of coffee and cassava, both high impact crops due to volume and value using the same BBS, sampling, KIIs and qualitative tools as the baseline done in 2018. It is noted that of the five commodities included in the 2018 baseline report, Sacha inchi and cabbages were continued.  CLEAN Semi-Annual Report October 2020 – March 2021  CLEAN Semi-Annual Report, April – September 2020. This included the time period of COVID-19 lockdown and effect on the project implementation.  CLEAN Semi-Annual Report October 2019 – March 2020  CLEAN Semi-Annual Report April – September 2019  CLEAN Project Lao PDR Brief - This is a simple, two-page summary which shares the purpose, objectives of the project, overview of activities, results and Sustainability  Beneficiaries framework for five commodities, include TOTs for Government. This file, provided by CLEAN MEL director, provides the information on the current beneficiaries of the project with their contact details, commodities and village/ district / province locations  CLEAN M&E framework with the indicators listed and targets  CLEAN (2020) Draft: Impact Assessment Report, unpublished and draft document not for distribution Other relevant documents  Agreement 2984 MAF Agreement on Establishment and Management of Agriculture Production Groups, 2014. This agreement sets out the standards for the organization and registration of farmer groups to ensure that they work together well. This was further updated in 2020.  Agreement of the Minister on Good Agriculture Practices (GAP) for Produce Quality Management Standards (Agreement No. 0539/ MAF, dated in Vientiane 09 Feb 2011. This agreement states standards for maintaining quality of agricultural products, from the planting materials, fertilizers (chemical vs. organic), provision of water, harvesting, storage and transportation. FAO72 and WFP (April and June 2020) COVID-19 Rapid Assessment of Food Security and Agriculture in Lao PDR). These reports measure the impact of COVID-19 on food security and agriculture in Lao PDR during the lockdown period in April 2020. With WFP field staff, FAO staff interviewed 1,007 telephone interviews for government agriculture workers in all 17 provinces and Vientiane Capital. Findings on agriculture, markets, household access to 72 Reports are available through the LaoFab repository at http://www.laofab.org/document/view/4371 and http://www.laofab.org/document/view/4373 81 food, and nutrition informed the response of these and other organizations on how to respond to COVID-19. These reports provided information on actual impact of COVID-19: 1) Economic Impact on Employment, Economy, Poverty  Over 350,000 workers have lost their jobs, overall economy will stop growing. The collapse of the tourism sector represents a loss of over 10% of GDP and 50,000 unemployed. Closure of factories are making 100,000 unemployed. Over 200,000 Lao workers returning from Thailand add to unemployment and poverty. Most of these people return to the countryside, how can they survive? 2) Impacts on food security: COVID-19 amplifies an already existing rice and water crisis  In Luang Prabang Province alone, 80% of 756 villages have run out of rice by April 2020. They will need to bridge another seven months before the next harvest. These rice shortages are a result of two years of drought and occur in all northern provinces. Reduced employment options make it difficult for rural households to earn money to buy rice. Water shortages are as urgent as rice shortages and urgently need a response as well. Increased rural poverty induced by COVID-19 risks increasing malnutrition levels as poor households may not be able to buy nutritious foods. 3) Impact on agriculture: reduced demand for farmer’s products  Loss of buying power reduced sales of meat with 30% and vegetables with 40% in Luang Prabang, Savannakhet and Vientiane Capital City. This drops in demand hits especially small-holders who had prepared for annual top-sales during Lao New-Year season. Prices of meat and rice have gone up, prices of vegetables have come down. 4) Secondary impact on agriculture: changing ambitions and concerns on regional export markets and access to inputs  Provinces see they need to accelerate change from production for tourism to regional exports, but they are not ready for it. They see a strong potential for beef cattle raising and slow-perishing vegetables in northern uplands, but they are concerned about getting fair contracts for exporting cash crops in 2020. 5) Reduced international trade due to closure of borders, which includes both sales of products but also inability of farmers to access equipment and materials with the greatest impact with communities and districts on the Thai and Vietnam borders. 82 Annex 3. GAP Certification Process The formation of farmers groups is a new concept for many of the farmers as it involves management systems, of which book keeping and business planning may require a higher level of education than that of many farmers. Farmers require technical assistance from PAFO and DAFO for meeting the basic criteria for farmers group registration. These criteria for farmers group registration are listed in Article 4 under the GOL’s Agreement No. 2984/MoAF (2014) which lists the principles, regulations, and measures for the establishment and expansion of the agricultural producers to strengthen the ability of small farmers to prepare to start small as a village-based group before growing to become a larger production cooperative. The aim for farmers groups is to generate income, improve livelihoods and to support each other for procurement, sales and solving problems. The process of farmers group registration contributes to improving economic and social development of communities. The DoA73 is encouraging of group formations and the principles of GAP promotion, to develop farmers groups to follow the standards of GAP. PAFO and DAFO are applying the basic principles of group formation and group management based on the following principles: 1. Membership in the farmers groups is voluntary 2. Farmers attend GAP training 3. Farmers groups demonstrate the practice of conducting GAP 4. Democratic management within the group 5. Farmers in the group receive the benefits of the group activities 6. The activities of the group are documented and are transparent and verifiable 7. Has been approved to formulate a group according to the agreement of MoAF No. 2984/MoAF (2014). Training focuses on practical measures to create the opportunities for production and agricultural services to the group members and their families in the following areas: 1. Technical and academic promotion and information of GAP. 2. Promote access to sources of capital and marketing 3. Reduce or exempt taxes on the import of agricultural factors. and exporting products according to the law on taxation, the law on duty assistance and other government policies announced in each term 4. Create enabling conditions of farmers group structures to reduce production costs GAP Certification comes under the GoL’s standards on Internal Control Systems and Good Agriculture Practice (ICS/GAP). The GAP steps needed for farmer group registration application and verification process for certification process, are: 1. Application process will be no longer than a total of 17 days for the data collection and verification. The certification process will take no longer than 20 days after the inspection process, in which the GAP committee will verify and organize a meeting for consideration of the application and approval of certification. Once this has happened, it will inform the farmers and handover the GAP certificate. The specifics of the GAP committee consideration and approval process will include: 2. The GAP inspection will be conducted by an inspector from the DoA (with a plan for training and handover of this responsibility to PAFO and DAFO levels). The GAP inspector will inform the GAP applicants of the inspection plan. The GAP applicant pays all the expenses before inspection and the inspectors will draft a report of the inspection to the GAP committee. The total inspection process will be no longer than a total of 45 days. 73 Information for this section comes from personal interviews with DG and DoA coordinator 83 3. The GAP committee will verify the inspection report and consider the application for the certification and inform the GAP applicant of its decision within 20 days. If the application passes, the GAP committee will provide the GAP certificate. If the committee does not pass the farmers group for the GAP certification, then the GAP Committee will inform the farmers group about the steps they need to take to improve their application with follow-up by the inspectors which should take more than 20 days. 4. The GAP certificate will need to be renewed every two years and the farmers group will need to inform the DoA at least 120 days before the renewal date. The DoA will then provide the inspection and new certificate within 60 days. Throughout the project, farmers have been learning the benefits of farmers group membership, and about the standards and benefits of GAP certification. In FGDs, some farmer groups still sell their products individually and through the established channels of selling through local vendors. In the FGD, the farmers indicated that harvest may occur at different times, with some farmers ready to sell their products before others. These farmers have yet to coordinate with each other to benefits from the group process, nor the benefits from international markets. Farmer groups that already obtained GAP certification, such as the Ban Huayseth farmers group, have realized the benefits of the farmer group management structure, ICS and GAP certification in being able to access domestic and international markets. While the initial GAP certification of this FG took three years, interventions by CLEAN and government partners have enabled the speeding up of the process which has benefited the 81 farmers groups. In 2021, these farmer groups were invited to attend workshops to apply for in-kind grants, of which 19 will be soon successfully approved to receive support such as cassava chipper machines and water pumps.74 74 Interview with CLEAN project staff 84 Source: MoAF-DOA, 2021 85 Annex 4. Guiding Questions for FGDs My name is [Moderator Name] and these are my colleagues who are helping to take notes? Is it OK for us to record this session? We are talking to Coffee producers about: Timeline activity – draw timeline for previous growing season before the start of the project, through the project until now. Mark on the timeline when they prepare the coffee planting for new trees, co￾planting of other vegetables, fruits (Potatoes. Kale. Beans. Soybeans. Groundnuts. Or bananas/ other) when coffee is harvested. Mark on the time line when there are problems with coffee trees (pests, floods, trees died to disease, etc.). Processing of coffee interview date village name district name Question guide Please record the ages of the men/women, ethnicity of those who attended, number of years producing coffee , average size of land used for coffee, whether a translator was needed] Section 1: Land preparation 1. What kind of land used (such as clearing forest land or land previously used for other kinds of agriculture)? Burning land, use of herbicides? 2. What kind of agricultural chemicals are used for Coffee? What has project promoted in the use of organic fertilizer useful instead of chemical fertilizer? 3. If land previously used for crops using pesticide or chemical fertilizer, what considerations for certifying the coffee as ‘organic’ Section 2: Type of coffee 4. What varieties of coffee do you grow? (major types are Arabica, Robusta/ Arabusta and Catimor). Before the project and now. If you have changed what type of coffee to grow, are you satisfied with this type of coffee? Advantages of growing this kind of coffee Disadvantages of growing this kind of coffee Section 3: Diseases affecting coffee planting or growth 5. What kinds of plant diseases affect coffee trees before the project, compared with after the project started? 86 Question guide If you received improved varieties from the project, what are the advantages and challenges of planting this kind of coffee in terms of resistance to plant disease? What solutions have you found for plant diseases? Section 4: Pests control What kinds of insects or other pests affect coffee? What methods have you used against insects and other pests in the past? What methods are you using now, which the project has provided? Section 5: Processing Sell fresh cherries without any processing Dry the coffee cherries - dry in the sun Dry the coffee cherries and dry in a barn/ coffee shed with good ventilation or other drying method Wet processing Wet processing and parchment extraction Processing of green beans With each of the methods, what is the rate of spoilage of the coffee cherries/ beans? Section 6: Transportation Sell to a small trader who comes to village As a group, send coffee to market (either individual farmers or members of Farmers’ Group) What do you know about Good Agriculture Practices? How do you apply those principles? If applied, what were the benefits of using these standards? If applied, what were the difficulties in applying If have not applied these principles, what were the obstacles? If there were obstacles, what have you tried to solve the problems? 87 Question guide Section 7: External Assistance What assistance have you received from Agriculture Extension Services? What has been the changes (with Agriculture Extension or other project inputs) from before the project and after the project? When did you last have visit from extension services? What were the topics? How have extensive services helped you to find solutions to challenges? Do you know the Coffee Association? Has the Association provided any training? Section 8: Impact How has the project affected the lives within your families? Impact based on CLEAN indicators A. How has volume and the value of sales of different forms of coffee changed since the time that project started? (Increase, decrease or stayed the same) What are the important factors for making this change? B. How has the cost of production (expenditures, cost of equipment, labor cost for production, etc) changed since the project started (Increase, decrease or stayed the same) What are the important factors for this change? (Example might include the impact of COVID-19 on making imports of equipment more difficult or expensive C. How has your access to international markets changed especially for the specialty coffees that CLEAN has been supporting? What are important factors which have enabled access to these markets or been obstacles for access to these markets? D. How has access to local markets changed since the project started? Are these market outlets always available (opened and remained open; opened and closed) How has your knowledge on coffee production changed in the areas of improved production technologies? GAP? Farm management? How has your knowledge and practiced changed in post￾harvest processing? 88 Question guide How have you used the profit? Examples: to invest in your agriculture business, health care, children’s education, improved nutrition? What has been the effect of the project on women’s work load? Have your children been involved in cultivation of coffee? If so, how has that affected their school attendance or health? How have your wives or other relatives contributed to the decision making on what? 1) crops to grow; 2) how to use education / improved practices from the project; 3) preparation of land (such as clearing of land, tilling, use of herbicide, etc.), 4) use of chemical vs. organic fertilizer and / or pesticides Section 9: Access to credit How have you accessed credit for coffee cultivation? What kind of business plan do you have to either maintain or expand coffee cultivation? Where can you access funds for coffee cultivation? Bank, village bank, microfinance provider, etc. What is the interest rate for the different options? If you are to pay back the loans, when do you pay back the loans If you have not been able to fully pay back the loans, what is your current debt burden? If you have not been able to fully pay back the loans, what agreement do you have with the lender? Examples: only able to pay back the interest, able to pay the interest and some of the capital, not able to pay back the interest or pay down the amount of capital. Section 10: Overall assessment Overall assessment of the positive points of the project (What has gone right?) 89 Question guide Overall assessment of the negative points of the project (What has been more difficult, or not useful?) For challenges, what solutions have you tried? What solutions have succeeded? How have project staff or government partners contributed to finding solutions What parts of what you have learned from the projects, do you think you will be able to continue (sustainability) Key lessons learned – what they would want to teach others about what they had learned from the project Optional question as some villages might not have a farmers group Farmers group – If there is a Farmers Group for coffee, what training have you received? Examples: group organization, preparation for registration of the group, business planning, tracking costs and income, working together for procurement or transportation? What have been the advantages/ disadvantages of working in farmers group? 90 Annex 4.2: Guiding questions for FGD with Cassava producers Introduction: My name is [Moderator Name] and these are my colleagues who are helping to take notes? Is it OK for us to record this session? We are talking to Cassava producers about: Timeline activity – draw timeline for previous growing season before the start of the project, through the project until now. Mark on the timeline when they prepare the cassava planting, planting of cassava, etc. Mark on the time line when there are problems with cassava plants (pests, floods, plants died to disease, etc.). Post-harvest Processing of cassava, preparation for market, etc. Date of interview Village District Question guide Question guide Please record the ages of the men/women, ethnicity of those who attended, number of years producing coffee, average size of land used for Cassava, whether a translator was needed] General Information about the village (village chief or another member of the village administration might be coffee producer so would know this information) 1. Population of the village 2. How many households in the village; how many families? 3. How many households are involved in coffee raising (both within the project and outside the project?) 4. Ease of transportation / road infrastructure to district town or nearest market collection point Section 1: Land preparation 1. What kind of land used (such as clearing forest land or land previously used for other kinds of agriculture)? Burning land, use of herbicides? 2. What kind of agricultural chemicals are used for cassava? Since this is not consumed, is organic fertilizer useful instead of chemical fertilizer? 3. If land previously used for crops using pesticide or chemical fertilizer, what considerations for those crops to use organic fertilizer, based on the training from the project? Section 2: Type of cassava 4. What varieties of cassava do you grow? Before the project and now. If you have changed what type of cassava to grow, are you satisfied with this type of cassava? Advantages of growing this kind of cassava Disadvantages of growing this kind of cassava Section 3: Diseases affecting cassava planting or growth 5. What kinds of plant diseases affect cassava? 91 Question guide 6. What plant diseases affect the cassava that you have grown? 7. If you received improved varieties from the project, what are the advantages and challenges of planting this kind of cassava in term of resistance to plant disease? 8. What solutions have you found for plant diseases? Section 4: Pests control 9. What kinds of insects or other pests affect cassava? 10. What methods have you used against insects and other pests in the past? 11. What methods are you using now, which the project has provided? Section 5: Processing 12. How do you prepare cassava to sell on the market? What are the problems with each method of preparation? Sell fresh cassava without any processing Slice cassava and dry in the sun Slice cassava and dry in a barn with good ventilation With each of the methods, how much of the cassava spoils? Section 6: Transportation 13. How do you each arrange transportation to the market? Are the people attending this FGD members of a Farmers’ Group? Individually Sell to a small trader who comes to village As a group, send Cassava to market (either individual farmers or members of Farmers’ Group) What are the advantages or challenges with each method? 92 Question guide 14. What do you know about Good Agriculture Practices or SPS for cassava? How do you apply those principles? If applied, what were the benefits of using these standards? If applied, what were the difficulties in applying If have not applied these principles, what were the obstacles? If there were obstacles, what have you tried to solve the problems? Section 7: External Assistance 15. What assistance have you received from Agriculture Extension Services? 16. What has been the change from before the project and after the project? 17. When did you last have visit from extension services? What were the topics? How have extensive services helped you to find solutions to challenges? Section 8: Impact 18. How has the project affected the lives within your families? 19. How have you used the profit? Examples: to invest in your agriculture business, health care, children’s education, improved nutrition? 20. Impact based on CLEAN indicators A. How has volume and the value of sales of different forms of coffee changed since the time that project started? (Increase, decrease or stayed the same) What are the important factors for making this change? B. How has the cost of production (expenditures, cost of equipment, labor cost for production, etc) changed since the project started (Increase, decrease or stayed the same) What are the important factors for this change? (Example might include the impact of COVID-19 on making imports of equipment more difficult or expensive) 93 Question guide C. How has your access to international markets changed especially for the specialty coffees that CLEAN has been supporting? What are important factors which have enabled access to these markets or been obstacles for access to these markets? E. How has access to local markets changed since the project started? Are these market outlets always available (opened and remained open; opened and closed) F How has your knowledge on coffee production changed in the areas of improved production technologies? GAP? Farm management? How has your knowledge and practiced changed in post￾harvest processing? 21. What has been the effect of the project on women’s work load? 22. Have your children been involved in cultivation of Cassava? If so, how has that affected their school attendance or health? 23. How have your wives or other relatives contributed to the decision making on what? 1) crops to grow; 2) how to use education / improved practices from the project; 3) preparation of land (such as clearing of land, tilling, use of herbicide, etc.), 4) use of chemical vs. organic fertilizer and / or pesticides Section 9: Access to credit 24. How have you accessed credit for cassava cultivation? 25. What kind of business plan do you have to either maintain or expand cassava cultivation? 26. Where can you access funds for cassava cultivation? Bank, village bank, microfinance provider, etc. What is the interest rate for the different options? 27. If you are to pay back the loans, when do you pay back the loans 28. If you have not been able to fully pay back the loans, what is your current debt burden? If you have not been able to fully pay back the loans, what agreement do you have with the lender? 94 Question guide Examples: only able to pay back the interest, able to pay the interest and some of the capital, not able to pay back the interest or pay down the amount of capital. Section 10: Overall assessment Overall assessment of the positive points of the project (What has gone right?) Overall assessment of the negative points of the project (What has been more difficult, or not useful?) For challenges, what solutions have you tried? What solutions have succeeded? How have project staff or government partners contributed to finding solutions From what you have learned from the project, what do you think you can continue in the future (sustainability) Key lessons learned – what they would want to teach others about what they had learned from the project Thanks for your participation 95 Introduction: My name is [Moderator Name] and these are my colleagues who are helping to take notes? Is it OK for us to record this session? We are talking to Coffee producers about: Timeline activity – draw timeline for previous growing season before the start of the project, through the project until now. Mark on the timeline when they prepare the land for planting sacha inchi, whether there are pests, when harvested and dried, processing of nuts, etc Date of interview Village District Question guide Participants Please record the ages of the men/women, ethnicity of those who attended, number of years producing sacha inchi , average size of land used for planting, whether a translator was needed] Section 1: Land preparation 1. What kind of land used (such as clearing forest land or land previously used for other kinds of agriculture)? Burning land, use of herbicides? 2. What kind of agricultural inputs are used for sacha inchi? What has project promoted according to GAP standard? 3. If land previously used for crops using pesticide or chemical fertilizer, what considerations for certifying the product as ‘organic’ Section 2: Type of coffee 4. What varieties of sacha inchi do you grow? Before the project and now. If you have changed what type of sacha inchi to grow, are you satisfied with this type of sacha inchi? Advantages of growing this kind of sacha inchi Disadvantages of growing this kind of sacha inchi Section 3: Diseases affecting coffee planting or growth 96 Participants 5. What kinds of plant diseases affect the sacha inchi nuts before the project, compared with after the project started? 6. If you received improved varieties from the project, what are the advantages and challenges of planting this kind of sacha inchi in terms of resistance to plant disease? 7. What solutions have you found for plant diseases? Section 4: Pests control 8. What kinds of insects or other pests affect sacha inchi plants? 9. What methods have you used against insects and other pests in the past? 10. What methods are you using now, which the project has provided? Section 5: Processing 11. How do you prepare sacha inchi to sell on the market? What are the problems with each method of preparation? Sell the entire nut and nut casing without drying Dry seed casing in the sun and take out the nut and sell the nut and the seed casing separately Extract the nut from the inner shell 12. With each of the methods, what is the rate of spoilage of the sacha inchi nut? Section 6: Transportation 13. How do you each arrange transportation to the market? Are the people attending this FGD members of a Farmers’ Group? Individually Sell to a small trader who comes to village As a group, send Sacha inchi to market (either individual farmers or members of Farmers’ Group) 97 Participants What are the advantages or challenges with each method? 14. What do you know about Good Agriculture Practices for Sacha inchi? How do you apply those principles? If applied, what were the benefits of using these standards? If applied, what were the difficulties in applying If have not applied these principles, what were the obstacles? If there were obstacles, what have you tried to solve the problems? Section 7: External Assistance 15. What assistance have you received from Agriculture Extension Services? 16. What has been the change from before the project and after the project? 17. When did you last have visit from extension services? What were the topics? How have extensive services helped you to find solutions to challenges? 18. Is there any association for Sacha Inchi growers? Section 8: Impact 19. How has the project affected the lives within your families? 20. Impact based on CLEAN indicators A. How has volume and the value of sales of different forms of coffee changed since the time that project started? (Increase, decrease or stayed the same) What are the important factors for making this change? B. How has the cost of production (expenditures, cost of equipment, labor cost for production, etc) changed since the project started (Increase, decrease or stayed the same) What are the important factors for this change? (Example might include the impact of COVID-19 on making imports of equipment more difficult or expensive) C. How has your access to international markets changed especially for the speciality coffees that CLEAN has been supporting? What are important factors which have enabled access to these markets or been obstacles for access to these markets? D. How has access to local markets changed since the project started? Are these market outlets always available (opened and remained open; opened and closed) 98 Participants E. How has your knowledge on coffee production changed in the areas of improved production technologies? GAP? Farm management? How has your knowledge and practiced changed in post￾harvest processing? 21. How have you used the profit? Examples: to invest in your agriculture business, health care, children’s education, improved nutrition? 22. What has been the effect of the project on women’s work load? 23. Have your children been involved in cultivation of coffee? If so, how has that affected their school attendance or health? 24. How have your wives or other relatives contributed to the decision making on what 1) crops to grow; 2) how to use education / improved practices from the project; 3) preparation of land (such as clearing of land, tilling, use of herbicide, etc.), 4) use of chemical vs. organic fertilizer and / or pesticides Section 9: Access to credit 25. How have you accessed credit for sacha inchi cultivation? 26. What kind of business plan do you have to either maintain or expand sacha inchi cultivation? 27. Where can you access funds for cultivation? Bank, village bank, microfinance provider, etc. What is the interest rate for the different options? 28. If you are to pay back the loans, when do you pay back the loans 29. If you have not been able to fully pay back the loans, what is your current debt burden? If you have not been able to fully pay back the loans, what agreement do you have with the lender? Examples: only able to pay back the interest, able to pay the interest and some of the capital, not able to pay back the interest or pay down the amount of capital. Section 10: Overall assessment Overall assessment of the positive points of the project (What has gone right?) 99 Participants Overall assessment of the negative points of the project (What has been more difficult, or not useful?) For challenges, what solutions have you tried? What solutions have succeeded? How have project staff or government partners contributed to finding solutions Key lessons learned – what they would want to teach others about what they had learned from the project What parts of the growing and processing of sacha inchi and what you have learned about farm management are the most likely that you will continue to use. How do you think that you have make these areas of work sustainable? Optional question as some villages might not have a farmers group Farmers group – If there is a Farmers Group for coffee, what training have you received? Examples: group organization, preparation for registration of the group, business planning, tracking costs and income, working together for procurement or transportation? What have been the advantages/ disadvantages of working in farmers group? Thanks for your participation 100 Guiding questions for FGD with Cabbage Producers Introduction: My name is [Moderator Name] and these are my colleagues who are helping to take notes? Is it OK for us to record this session? We are talking to Cassava producers about: Timeline activity – draw timeline for previous growing season before the start of the project, through the project until now. Mark on the timeline when they prepare the fields for growing cabbage, how they irrigate the cabbage, when they have problems with pests (what kind of pests and what kind of insecticide they use) Date of interview Village District Question guide Question guide Please record the ages of the men/women, ethnicity of those who attended, number of years producing Cabbage , average size of land used for cabbage, type of cabbage (white, Chinese or both), whether a translator was needed] General Information about the village (village chief or another member of the village administration might be coffee producer so would know this information) 1. Population of the village 2. How many households in the village; how many families? 3. How many households are involved in coffee raising (both within the project and outside the project?) 4. Ease of transportation / road infrastructure to district town or nearest market collection point Section 1: Land preparation What kind of land used (such as clearing forest land or land previously used for other kinds of agriculture)? Burning land, use of herbicides? What kind of agricultural chemicals are used for cabbage? Since this is food for consumption do you use organic fertilizer useful instead of chemical fertilizer? If land previously used for crops using pesticide or chemical fertilizer, what considerations for those crops which would be certified as ‘organic’. Does that make a difference in the volume and sales of the cabbage? Section 2: Type of cabbage 101 Question guide What varieties of cabbage do you grow? Before the project and now. Did the project provide cabbage seeds, such as improved varieties? If you have changed what type of Cabbage to grow, are you satisfied with this type of cabbage? Advantages of growing this kind of Cabbage (white or Chinese cabbage) Disadvantages of growing this kind of Cabbage (white or Chinese cabbage) Section 3: Diseases affecting Cabbage planting or growth What kinds of plant diseases affect the cabbages? What other solutions have you found for plant diseases? If you received improved varieties from the project, what are the advantages and challenges of this kind of cabbage in resisting plant diseases? Section 4: Pests control What kinds of insects or other pests affect cabbages? What methods have you used against insects and other pests in the past? What methods are you using now, which the project has provided? Section 5: Processing How do you prepare cabbage to sell on the market? What are the problems with each method of preparation? Sell fresh cabbage without any processing or packaging (such as packaging in plastic) Sell fresh cabbage with special kind of packaging (such as in a box or other container) With each of the methods, how much of the cabbage spoils? Section 6: Transportation 102 Question guide How do you each arrange transportation to the market? Are the people attending this FGD members of a Farmers’ Group? Individually Sell to a small trader who comes to village As a group, send Cassava to market (either individual farmers or members of Farmers’ Group) What are the advantages or challenges with each method? What do you know about Good Agriculture Practices for cabbage? How do you apply those principles? If applied, what were the benefits of using these standards? If applied, what were the difficulties in applying If have not applied these principles, what were the obstacles? If there were obstacles, what have you tried to solve the problems? Section 7: ການຊ່ ວຍເຫືລອຈາກພາຍນອກ / External Assistance What assistance have you received from Agriculture Extension Services? What has been the change in assistance from the project or Agriculture Extension from before the project and after the project? When did you last have visit from extension services? What were the topics? How have extensive services helped you to find solutions to challenges? Section 8: Impact How has the project affected the lives within your families? How have you used the profit? Examples: to invest in your agriculture business, health care, children’s education, improved nutrition? 103 Question guide Impact based on CLEAN indicators Impact based on CLEAN indicators A. How has volume and the value of sales of different forms of coffee changed since the time that project started? (Increase, decrease or stayed the same) What are the important factors for making this change? B. How has the cost of production (expenditures, cost of equipment, labor cost for production, etc) changed since the project started (Increase, decrease or stayed the same) What are the important factors for this change? (Example might include the impact of COVID on making imports of equipment more difficult or expensive) C. How has your access to international markets changed especially for the specialty coffees that CLEAN has been supporting? What are important factors which have enabled access to these markets or been obstacles for access to these markets? D. How has access to local markets changed since the project started? Are these market outlets always available (opened and remained open; opened and closed) E. How has your knowledge on coffee production changed in the areas of improved production technologies? GAP? Farm management? How has your knowledge and practiced changed in post￾harvest processing? What has been the effect of the project on women’s work load? Have your children been involved in cultivation of Cabbages? If so, how has that affected their school attendance or health? How have your wives or other relatives contributed to the decision making on what? 3) crops to grow; 4) how to use education / improved practices from the project; 3) preparation of land (such as clearing of land, tilling, use of herbicide, etc.), 4) use of chemical vs. organic fertilizer and / or pesticides Section 9: Access to credit How have you accessed credit for Cabbages cultivation? What kind of business plan do you have to either maintain or expand cabbage cultivation? Where can you access funds for cabbages cultivation? Bank, village bank, microfinance provider, etc. What is the interest rate for the different options? If you are to pay back the loans, when do you pay back the loans If you have not been able to fully pay back the loans, what is your current debt burden? If you have not been able to fully pay back the loans, what agreement do you have with the lender? 104 Question guide Examples: only able to pay back the interest, able to pay the interest and some of the capital, not able to pay back the interest or pay down the amount of capital. Section 10: Overall assessment Overall assessment of the positive points of the project (What has gone right?) Overall assessment of the negative points of the project (What has been more difficult, or not useful?) For challenges, what solutions have you tried? What solutions have succeeded? How have project staff or government partners contributed to finding solutions With the knowledge you have gained from the project, or use of improved practices, what do you think are the areas which you will continue to use in the future? How can this knowledge or practice be sustainable? Key lessons learned – what they would want to teach others about what they had learned from the project Thanks for your participation 105 Annex 5. Guiding Questions for KIIs Guide Questions for Mobile traders/Middle man/Local shop interview Identify mobile traders by discussing with members of the FGDs for their input on the selection of representative mobile traders so that the Elixirs staff will have the contact details of these traders. The KIIs with the mobile traders / middlemen will look at how they link village producers with the larger merchants and wholesalers in both domestic and export market linkages. In combination with information with the FGDs, the KIIs with these local traders will explore the trade relationships, demands of the producers, the investment for the transportation, inputs for the commodities producers and relationship between both parties such as contractual arrangements Target respondent are 2 (Male and Female) of Small traders / Mobile Traders / Middlemen / Local Shop per district. Questionnaires Respondent •Do you know about CLEAN project interventions? •What is your current occupation? •When did you start working on the commodity trading? •How long have you been working/trading on this commodity? (select one of 4 commodities) •Can you tell me about the key tasks of your activities for trading the commodity? (select one of 4 commodities) •How many villages in the district that you have been trading with the producer? •How many producers you have been trad with them in the past 12 months? •How many producers out of you district area? • Small trader (Male & Female) •How did you build the trust with the commodity producer by making the agreement or contract with them? •If yes, is it an official agreement or contract recognized by authority in which level? •How about market of the commodity that you have been working it is any change? Why? •How much investment cost for using on trading this commodity per year? 106 •How many tone/Kilograms can you be able to trad/buy per year? • How many of the worker do you have? •What kind of vehicle do you used for trading the commodity? • How many of the vehicle do you have? •What kind of support do your get from the local government such as PAFO or DAFO? • Do you get any support as the linkages between producers and large trader or exporters? • Does CLEAN support any kind of your business? • How has CLEAN’s interventions affected targeted beneficiary groups? •Are the project’s interventions having the expected and effect to your business? •Are there any challenges that you may require CLEAN to change or support? •Are there any opportunities for you to plan and collaborate you work with CLEAN project? 107 Guide Questions for Local Exporter interview The Field supervisor will conduct the interview with at least 4 large traders to measure the relationship between the mobile traders, middlemen and local shop and know the opportunities for the commodities market share and any compliance document/process for food safety and GAP requirements. Target respondent are 4 (large traders) which Manager or staff responsible for sale and marketing. Questionnaires Respondent Do you know about CLEAN project interventions?  Manager What type of commodities do you trad or export? Total amount of your investment included properties (land and vehicle) for commodities trading? Total volume (Mt) buy from producer during the whole year? (last year) Total volume (Mt) sold or exported during the whole year? (last year) Total amount spent for trading in the previous (whole year) (Kip/USD/Bath)? Total value of sales during the whole year (last year) (Kip/USD/Bath)? Where are the target for the export market? How did you buy the commodities? Who are the main sellers? How many staff and worker do you have? Do you know about the CLEAN support for staff capacity building on TOT of ICS/GAP certification inspectors conducted? Did you apply for criteria on ICS/GAP/organics commodities? 108 What compliance process did you focused? What kind of support do you get from the government? Do you get any support as the linkages between producers and small trader in the province and district levels? Does CLEAN support any aspect of your business? Are there any challenges that you may require CLEAN to change or support? Are there any opportunities for you to plan and collaborate you work with CLEAN project? 109 Guide Questions for Government partners, People in Government and CLEAN staff interview Identify the result of the training on Internal Control System ICS-related topics and TOT methods resulted in the creation of ICS/GAP certification inspectors. The collaboration between DOA and key staff from PAFO and DAFO to develop guidelines and a TOT curriculum for the delivery of GAP trainings to farmers in selected CLEAN intervention areas. CLEAN staff Elixirs staff will meet with key people in CLEAN, both in Vientiane and in the province during field work, to learn about their work and recommendations. Key results Questionnaires Respondent Relevance/General? Is the project doing the right thing? Coherence How well does the project interventions ‘fit’ with national policies Have you had an opportunity to travel for a visit the project implementation areas to view project? How have government staff or your team member been involved with the project? Do you know about the project goal and main objective? And what do you think about that? What do you think about the MEL tools of the project? What do you think about the tracking systems on ICS and GAP, it is working well and how relevant are these systems for to the producers and district authorities? How relevant has capacity building and TOT been to promotion of ICS/GAP certification inspectors/ the Government staff? How aligned with national agriculture and commerce policies? Such as registration of farmers groups and the Socio￾economic plan IX? • DOA Effectiveness – Is the intervention achieving its objectives? So would have to look at the particular targets – also should identify facilitating and obstacles to achieving objectives (contribute to process) How many trainers have been trained? To what extent can they train producers and other government staff? Please give an example of the topics that trainers (who have been trained in TOT) have delivered on ICS/GAP certification to inspectors. Do you have any feedback for improve to make the project implementation more effective? Do you think the project on track to meet with targets? If targets are not being met, why not? If targets are being exceeded, why? • DOA Efficiency How well are the project resources being used? How many of the staff member had train to be ICS/GAP certification inspectors? Are intended beneficiaries receiving services as planned? What are the topics that you or your team delivered? What are the topics that beneficiaries like most? What are the topics that there are able to apply and gain experiences? What support that the project being implemented as designed? • PAFO & DAFO • CLEAN staff 110 Key results Questionnaires Respondent Sustainability – will the benefits last? Are additional efforts needed, or are other changes to the strategy recommended? From the current observation of project activities, which activities are most likely to continue after the end of the project? What support would be needed? What is CLEAN doing to talk with project partners on making an exit strategy? • DOA • PAFO & DAFA • CLEAN staff Impact – what difference does the project make? How has CLEAN’s interventions affected targeted beneficiary groups? Are the project’s interventions having the expected effect on identified constraints? Are there any opportunities or challenges that may require revisions of the project strategy or redirection of project implementation plan? • DOA • PAFO & DAFO • CLEAN staff 111 Annex 6. Approved Terms of Reference 1. Introduction The Creating Linkages for Expanded Agricultural Networks (CLEAN) Project is a five (5) year program implemented by Winrock International and funded by the Food for Progress Program of the United States Department of Agriculture (USDA). The CLEAN Project seeks to 1) increase the productivity of horticultural products through training in Good Agricultural Practices (GAP) standards, strengthening plant protection services, and improving the functionality of quality compliance and certification systems, and 2) expand domestic and regional trade through integration of Laotian horticultural products into regional market networks, improved branding of Laotian horticultural products, and increasing producer adoption of GAP standards. Winrock works with market-ready farmers, agro-enterprises, entrepreneurs, and traders in two distinct regions: the Bolaven Plateau and the Vientiane Plain. CLEAN’s project activities focus primarily on capitalizing on Lao PDR’s comparative advantage in the Greater Mekong Sub Region (GMS) and global markets, where a large middle-class population is willing to pay a premium price for high-quality horticulture products. Secondarily, activities aim to increase domestic demand. Improving the marketing infrastructure for exports will help create market efficiencies, enabling the domestic market to grow. CLEAN facilitates private investment in supply chain infrastructure that will reduce transaction costs and post-harvest losses, facilitate the development of market intermediation networks in key Lao PDR markets, and build on other successful market linkage strategies. Winrock International (WI) is seeking an experienced firm with an office or presence to Lao PDR to conduct a midterm evaluation to assess the project’s progress, particularly how well its plans and activities are working, and provide feedback on any issues identified. 2. Project Background Lao PDR is the second-fastest-growing economy among ASEAN members. Its gross domestic product (GDP) grew 4.7% in 2019 and per capita GDP has increased, from $1,581 in 2012 to $2,535 in 2019, while poverty levels have declined, from 39.1% in 1997 to 23% in 2012.In 2019, agriculture accounted for only 15% of GDP, as a result of urbanization and the increasing importance of non-agriculture sectors, such as mining and electricity exports. Nevertheless, 64% of the population still relies on agriculture to earn a living (down from 73% a decade ago). Compared with other countries in the region, Lao PDR’s pace of agriculture commercialization has been slow, as reflected in its agriculture exports. In 2015, exports to ASEAN were valued at $70 million, but this represents only 1% of the total value of ASEAN exports. The dominant ASEAN exporters in terms of share of total value of exports are Vietnam (35%), Thailand (31%), and Myanmar (11%). Although Lao PDR’s agriculture exports have recently shown a promising growth rate of 33% annually (2011/2015), these exports are primarily raw crops, with no value added. Maize, rice, and coffee (as raw beans) account for some 46% of total export value; vegetables (mainly cassava and cabbage) account for 16%, and fruits only 3%. 112 The agricultural market offers opportunities for Lao PDR farmers, both domestically and in the ASEAN region. Rapid changes, including higher incomes, urbanization, and a growing domestic and regional tourist industry, are creating demand for increased variety, higher quality, and safety of agriculture products. To enter these high- value supply chains, Lao PDR farmers must produce and market high￾quality fruits and vegetables, competing with regional market leaders, such as Thailand and Vietnam, that already are integrated in regional and global markets. CLEAN will increase the competitiveness of the Lao PDR in regional horticulture markets by building on growing consumer interest in clean and safe food and leveraging Lao PDR’s reputation as a source of clean agriculture. Lao PDR will be positioned to compete in this growing regional and domestic niche market through an integrated approach that increases the capacity of farmers and supply chain actors to meet quality and quantity requirements; strengthens public and private systems for certification; supports market-driven service delivery to farmers; and institutes a branding campaign that will put Lao PDR on the map as the go-to source for high- quality fruits and vegetables. CLEAN collaborates with Department of Agriculture (DOA) to build the capacity of private sector partners to meet the growing demand for high quality agricultural products in regional and international markets. Using a market driven approach, CLEAN focuses on increasing production and reducing post-harvest losses, improving quality compliance and certification systems, developing linkages and boosting demand for higher-quality horticulture products in domestic, regional and global markets. CLEAN works with market-ready farmers, agro-enterprises, entrepreneurs and traders on four targeted crops: coffee (specialty grade), cassava, cabbage, and sacha inchi (a ‘superfood’ seed that possesses high-levels of Omega 3) in two distinct regions: the Bolaven Plateau and the Vientiane Plain. CLEAN implements its activities across all five project provinces: Champasak, Sekong, Salavan, Vientiane Capital and Vientiane provinces. CLEAN is targeting 14,877 direct beneficiaries among the following actors in the horticulture sector: smallholder farmers, farmer organizations, traders, agribusinesses, agricultural input companies, service providers, exporters, importers, and processors. 3. Purpose The purpose of this Request for Proposal (RFP) is to solicit applications from an individual, company, or organization to conduct a Midterm Evaluation for Winrock’s CLEAN project in Lao PDR. A key objective of the evaluation is to collect and provide both quantitative and qualitative data to assess progress towards program results, including performance indicator targets established in the cooperative agreement and defined under the Program Monitoring Plan approved by USDA. The findings of the evaluation will be used to identify and inform any necessary adaptations to the planned activity design, implementation, and management, as well as targets. The Midterm Evaluation should be conducted between December and January 2020. The assessment team will be comprised of the lead consultant and their assistants and shall work in cooperation with CLEAN’s Director of M&E and the project’s Chief of Party, as well as home office-based resources. 4. Research Questions 113 CLEAN’s midterm evaluation will focus on the project’s implementation progress towards meeting its objectives and goals. The below questions are illustrative ones; the final key evaluation questions will be decided in collaboration with the selected firm or consultant. QA Key Questions Proposed Method Relevance  How accurate have the critical assumptions underpinning the project proven to be in practice? Is the Theory of Change still appropriate for the project goals and objectives?  Are there unanticipated opportunities or challenges that may require revisions of the project strategy or redirection of project resources?  Are MEL tools and tracking systems working well and identifying relevance project impact?  Desk reviews and staff interviews  Interviews with project beneficiaries Effectiveness  What factors are facilitating the project’s effectiveness? What factors are hindering it?  What should be improved in order to make the project implementation more effective?  Document Review  Key informant interviews and beneficiary focus groups Efficiency  Are intended beneficiaries receiving services as planned? What are their experiences and recommendations?  To what extent is the project being implemented as designed?  Is the project on track to meet with targets? If targets are not being met, why not? If targets are being exceeded, why? Do targets need to be adjusted?  Document review, beneficiary focus groups and key informant and staff interviews  Beneficiary-based survey (BBS) Sustainability  Does the original strategy for the project’s sustainability and lasting impact remain feasible? If not, why? Are additional efforts needed, or are other changes to the strategy recommended?  Document review and staff interviews  BBS Impact  How has CLEAN’s interventions affected targeted beneficiary groups? Are the project’s interventions having the expected effect on identified constraints?  Desk review, KIIs, and FGDs  BBS 5. Research Target Respondents, Audiences and Results 114 This evaluation is meant to serve the following purposes: 1. Examine the program implementation efforts and accomplishments to-date, collect data and assess the program performance in terms of progress toward achieving the target indicators set. 2. Make recommendations for mid-course adaptations and changes in activity design and implementation that are needed to improve performance. In the design of the assessment methodology, the contractor must take into consideration the program results framework (enclosed to this RFP), the Program Performance Monitoring Plan (PMP) as well as the evaluation design, methodology and CLEAN’s Baseline Assessment. When designing their evaluation approach, prospective contractors should take into the account the following factors: Target Respondents:  Government officials: representative of Department of Agriculture (DOA), Provincial Agriculture and Forestry Office (PAFO), District Agriculture and Forestry Office (DAFO)  Representatives of small, medium, large-scale farmers, cooperatives, collectors and traders  Implementing partner Village Focus International  Grant recipients including Mai Savanh Lao, Plantation Hills Artisans, and Lao Cassava Association. Research results: This evaluation is important as it will allow the project team to measure performance, understand the reasons behind project successes and challenges, and adapt interventions as needed in order to effectively achieve the project goals. The results of this qualitative and quantitative analysis will serve three primary purposes:  To measure progress on quantitative indicators;  To qualitatively measure the progress of project implementation – what are successes, challenges and concerns; and  To inform project and key partners on concerns and recommendations, so that better implementation plans can be improved. Target Audience: The three primary audience groups of this survey include:  Winrock Home Officer Staff, CLEAN Project staff, and implementing partners: Qualitative information as to why or why no and which aspects of interventions are working well or may need adjustment will guide the project team’s strategy, implementation approach and activities during the implementation of the second half of the project. The project team will discuss 115 recommendations and decide on actions or changes in response to the midterm evaluation.  USDA officials: USDA will be provided with midterm findings that demonstrate the degree to which the project has, thus far, achieved project goals and targets. USDA will be furnished with an analysis of the data and recommended adaptations to project activities, which will contribute to the Department’s own institutional research and learning.  Government stakeholders: CLEAN works very closing with the Government of Laos as a key partner. Vientiane-based staff of the Department of Agriculture and targeted frontline Provincial Agriculture and Forestry Office (PAFO) and District Agriculture and Forestry Office (DAFO) staff from the targeted project sites.  Private Sector stakeholders, project beneficiaries, and grantees: o Private sector trade associations or cooperatives o Farmers and Farmer Groups o Collectors o Traders and retailers The evaluation will be shared with stakeholders through existing regular meetings with different key stakeholder groups and will be published in line with requirements for all U.S. funded international food assistance programs. 6. Collection areas and methodology Collection area: Data collection will be conducted in five provinces, where the project is implementing. However, depending on methodological and technical considerations, budget and timelines, the research firm/consultant and Winrock International can jointly select target districts and villages. The consultant must propose a sampling strategy that will result is a sample representative of the entire project area and is designed to allow for comparison between baseline and midterm evaluation results. Methodology: Based on the research questions mentioned above, the project anticipates that the proposals will feature quantitative and qualitative methods suitable for collecting data representative of project participants and probing into the experiences of those involved in the CLEAN project’s implemented activities as well as gain recommendations from them. Research firm/consultant is encouraged to propose cost effective research methods to respond to research questions and approaches that will allow triangulation of data are of particular interest. Data collection methodology, including sampling strategy, and an accompanying rationale should be proposed as part of the technical proposal. The following methods and tools should be used in the evaluation design: 116 Document review. The assessment team will find it useful to consult a broad range of background documents related to the agricultural sector in the project’s ZOI, apart from program documents provided by Winrock. Key Informant Interviews (KIIs). KIIs are suggested to be conducted with key stakeholders, including Winrock staff and project beneficiaries and stakeholders. Focus Group Discussion. It is anticipated that the evaluation team will conduct multiple rounds of focus group discussions with Winrock staff, GoL representatives, farmer beneficiaries, farmer’s groups and cooperatives. Special attempts should be made to ensure even participation and careful wording of the key questions. Survey. The evaluation team will carry out a survey of current program beneficiaries including smallholder farmers, agricultural producers, small and medium enterprises and GoL representatives, to collect the required data. The sampling strategy should be consistent with the one used in the baseline assessment and should ensure 95% confidence level. The total number of current program beneficiaries is 2,006, and the evaluator will be expected to conduct a survey representative at the level of each commodity. Additional details about the composition of the survey participants will be provided to the successful bidder. The selected research firm/consultant is responsible for developing the data collection and qualitative analysis methodology and sampling strategy, but all final decisions regarding methodology will be made in collaboration with CLEAN. 7. Roles and Responsibilities The subcontractor will implement some or all the following activities, depending on the evaluation design:  Collect data with inclusion and gender perspective including collection of socio￾demographic data and desegregation by sex, age, disability and creating enabling environment for female and male, youth and adult participation;  Review project documents (other projects’ data, baseline data, monitoring data, etc.);  In collaboration with CLEAN, develop a detailed research design and timeline outlining proposed methodology (including data sampling strategy and data collection tools), research plan and timeline, and final report structure;  Translate into Lao and pilot all survey questionnaires and tools;  Hire the field team (supervisors and data collectors), preferably recruiting experienced staff with similar research exercises in country;  Prepare a field manual for training, then train data collectors; 117  Arrange all fieldwork logistics;  Oversee data collection and any required data entry or transcription, using appropriate quality control measures and supervision;  Present initial findings and recommendations (drawn from their own conclusion free from organizational or political pressure) to CLEAN MEL team and senior management team, and subsequently to USDA for feedback;  Prepare a draft report using the outline developed in collaboration with CLEAN;  Prepare a revised report that incorporates the feedback provided by CLEAN,  Submit a final report in English to Winrock.  Submit cleaned datasets to CLEAN in line with USDA DLL requirements,  Submit to CLEAN all the documents related to the study (filled questionnaires, electronic versions of the collected data, transcripts, training manual, fieldwork logs, etc.)  Hold bi-weekly status calls with CLEAN – MEL team and senior management team.  Prepare a research brief on any identified ethical issues and how they were addressed.  Prepare a 2-3-page stand-alone brief describing the evaluation design, key findings and other relevant considerations that will serve to inform any interested stakeholders of the midterm evaluation, and should be written in language easy to understand by non-evaluators and with appropriate graphics and tables.  Presentation of key findings delivered via webinar to CLEAN team in Laos and Winrock Home Office team. CLEAN will:  Provide access to the research materials cited above (PMP, baseline surveys, project monitoring database, etc) and will ensure that the contractor receives timely feedback to the research firm/consultant on research design, all data collection tools, translation, sampling strategy and other methodological components.  Provide a complete list of: o Standard and customized Indicators (SI & CI indicators) o Direct and indirect beneficiaries o Firms/associations receive support and in-kind grants from project o Implementing partners and government partners 118 o Informing government partners and other partners about the Midterm Evaluation 8. General instructions to offerors Offerors wishing to respond to this RFP must submit proposals in English in accordance with the following instructions. Offerors must review all instructions and specifications contained in the RFP. Failure to do so will be at the offeror’s risk. Issuance of this RFP in no way obligates Winrock to award a subcontract. Offerors will not be reimbursed for any costs associated with preparation of submission of their proposal. Winrock shall in no case be responsible or liable for these costs. Submission to Winrock of a proposal in response to this RFP constitute an offer and indicates the offeror’s agreement to the terms and conditions of this RFP and any attachments hereto. Winrock reserves the right not to evaluate a non-responsive or incomplete proposal. 9. Submission Details Proposal Submission Deadlines Proposals must be received no later than before the date and time indicated in the cover page of this RFP. Late submissions will not be accepted. Winrock International may request additional documentation after the bid deadline. All submissions are to be made electronically to Ms. Somphavanh Nakhavong, Monitoring & Evaluation and Learning And CC: Mr. Alex Dahan, Chief of Party Questions Submission Deadline Inquiries/questions must be received no later than the date and time indicated in the cover page of this RFP and must be submitted via email. Winrock will review and respond to all potential offers as soon as possible. Proposal Documentation Offerors must submit 2 sets of proposals, including technical proposal and cost proposal in separate files, with all pieces of the proposal labeled clearly. Technical Proposal Technical proposal must not exceed 15 pages and include the following items:  A 1-page cover page shall be included with the proposal on the offeror’s company letterhead with a duly authorized signature. The offeror shall certify in the cover letter a validity period of 90 days for the price provided (1 page maximum)  Management Approach (4 page maximum) will include: 119 o The offeror must present in narrative format a description of: a. Company/individual profile (supplemental material can be placed in the annex); b. The organizational/individual capacity to conduct the scope of work (include 1-3 examples of English reports from similar studies), c. CVs of all proposed team members (placed in annex); d. The organization’s/individual’s previous experience with education/school- focused studies in the Mali context; and e. Letters of support for the offerors’ work or letters of reference (placed in the annex).  Technical proposal will include (10 page maximum) o Technical Approach: The applicant should describe in the technical proposal their approach and methodology for the study in detail. This may include, as appropriate, sampling methods and should outline ethical considerations including issues of consent/assent and plans for protection of human subjects. The narrative should describe the consultant’s plan for the design and implementation of data collection tools; training for data collectors and supervisors; data collection and quality control; data entry; transcription; data analysis, data storage and security, and report writing. A detailed work plan including activities, timeframes, staffing, and resources should be included in the Technical portion of the proposal o Past Performance: Provide a list of at least three (3) recent awards of similar scope and duration. The information shall be supplied as a table and shall include the legal name and address of the organization for which services were performed, a description of work performed, the duration of the work and the value of the contract, description of any problems encountered and how it was resolved, and a current contact phone number of a responsible and knowledgeable representative of the organization. Cost Proposal  Financial Approach: The offeror must present a detailed financial proposal that covers the following items and includes a narrative on the assumptions behind the estimates. o Salaries. Includes personnel for technical assistance, data collection, data o Data entry, and analysis, e.g. staff, enumerators, supervisors, drivers, etc. o Per diem and Travel. Includes daily costs for lodging and meals and incidental expenses during training and during field work, mode of transportation, vehicle rental, gas. o Printing. Includes survey questionnaires (if applicable), other study tools, reports. o Communications. Includes telephone, email, computer, mobile phone, etc. 120 o Supplies. Includes paper, pens, bags, other materials for field work. o Training costs. o Other relevant costs o Cost quoted must include unit price and total price in Lao Kip. Additionally, the offeror shall submit on separate sheet the fixed price for the anticipated deliverables under this RFP. Evaluation Criteria Each proposal will be evaluated and scored against the evaluation criteria which are stated in the table below. Cost/price proposals are not assigned points, but for overall evaluation purposes of this RFP, technical evaluation factors other than cost/price, when combined, are considered significantly more important than cost/price factors. The technical criteria are set forth below in descending order of importance. Evaluation Criteria Maximum Points Technical Approach The extent of the offeror’s responsiveness to the requirements of the Scope of Work in the RFP and understanding of the Scope of Work The appropriateness and validity of the methodology proposed by the offeror 40 Team Qualifications and Management Approach The quality, clarity and responsiveness of the offeror’s overall management approach The proposed research team’s qualifications and background 30 Past Performance The quality of the offeror’s previous work on similar activities The extent to which the offeror has performed similar activities in the past 30 Total 100 121 Award Winrock will review all proposals, and make an award based on the evaluation criteria stated above and select the offeror whose proposal represents the best value to the CLEAN project. Winrock may also exclude an offer from consideration if it determines that an offeror is "not responsible", i.e., that it does not have the management and financial capabilities required to perform the work required. Evaluation points will not be awarded for cost. Cost will primarily be evaluated for realism and reasonableness. Winrock may award to a higher priced offeror if a determination is made that the higher technical evaluation of that offeror merits the additional cost/price. Winrock may award to an offeror without discussions. Therefore, the initial offer must contain the offeror’s best price and technical terms. Anticipated deliverables and associated payments Upon award of a subcontract, the deliverables and deadlines detailed in below table will be submitted to Winrock. Payment will be made within 30 calendar days upon acceptance and approval of a deliverable and Invoice by Winrock. Description Deliverable Due Date Amount Award and signature of contract Fully executed subcontract 4 January 2021 Inception Report and Evaluation Design Plan (Include methodology, data quality assurance plan, tools, enumerators training, field work plan, logistic and others) Approved Inception Report 18 January 2021 35% 1st Draft report The report should include the following sections:  Cover Page (with photo, if possible)  List of Acronyms  Table of Contents, which identifies page numbers for the major content areas of the report.  Executive Summary (3 to 5 pages): should Winrock Approved 1st draft report 8 March 2021 45% 122 be a clear and concise stand - alone document that gives readers the essential content of the midterm evaluation report.  Body of Report (not to exceed 30 pages): o Introduction/Background (2 pages): Briefly describe the CLEAN project and state the objectives of the baseline assessment. o Methodology (4 pages): Describe the data collection methods used including schedule of activities, description of any statistical analysis undertaken, and constraints and limitations of the evaluation process and rigor. o Findings and analysis (14- 18 pages) pertaining to qualitative data addressing the evaluation questions and measures/values for selected indicators and disaggregates, as prescribed. o Conclusions (2-4 pages) summarizing key conclusion from the evaluation data. o Recommendations (6-10 pages): Provide actionable recommendations on project design, implementation, approaches to mitigate potential risks to implementation (including COVID￾19), validation of indicator targets, and monitoring and data collection methodologies based on lessons learned in the process of collecting the evaluation data.  Annexes: Attach this Terms of Reference; additional methodology documentation, including data collection tools and field visit schedule; list of stakeholders interviewed with number and type of interactions; table of indicator values; conflict of interest disclosure form signed by evaluation team members; and any other special 123 Please be noted that: 1. The fixed price for the deliverables is inclusive of all taxes. 2. The anticipated deliverables and associated payments may be adjusted based on agreement between research firm/consultant and Winrock International (after successful firm/consultant is selected) Confidentiality statement This document, and any attachments thereto, regardless of form or medium, is intended only for use by the addressee(s) and may contain legally privileged and/or confidential, copyrighted, trademarked, patented or otherwise restricted information viewable by the intended recipient only. If you are not the intended recipient of this document (or the person responsible for delivering this document to the intended recipient), you are hereby notified that any dissemination, distribution, printing or copying of this document, and any attachment thereto, is strictly prohibited and violation of this condition may infringe upon copyright, trademark, patent, or other laws protecting proprietary and, or intellectual property. In no event shall this document be delivered to anyone other than the intended recipient or original sender and violation may be considered a breach of law fully punishable by various domestic and international courts. If you have received this document in error, please respond to the originator of this message or email him/her at the address below and permanently delete and/or shred the original and any copies and any electronic form this document, and any attachments thereto and do not disseminate further. Where no notice is given, all information contained herein is Copyright 2019 Winrock International. documentation as needed.  Photographs: 20-25 high quality photographs (minimum 300 dpi resolution) of data collection activities, i.e., interviews; focus group discussions, current agriculture and trade practices, etc. Photographs should be provided upon submission of the written evaluation report on a thumb drive or equivalent external storage device. Final report, clean data sets and Evaluation Findings Brief USDA Approved final report, receipt of data set and evaluation findings brief 2 April 2021 20% 124 Annex 7. Participant and Event Disaggregates Record of farmers registered in FG versus actual participation in project activities # Province District Farmer group Name (Eng) FG code # of farmers joining FG # of farmers actually joining project activities (project's beneficiaries) % of actual beneficiaries vs interested farmers Commo dity type by FG registrati on Tota l regis tere d Mal e Femal e Total # Male Female Commodi ty type % 1 Champasak Paksong Cooperative of Agriculture and Agro-Tourism (CAAT) PO#001 Coffee 11 11 0 6 6 0 Cabbage 55% 2 Champasak Paksong Coffee Farmer Bolaven Cooperative (CFBC) PO#002 Coffee 24 24 0 2 2 0 Cabbage 8% 3 Champasak Paksong Paksong Coffee Cooperative (PCC) PO#003 Coffee 11 11 0 1 1 0 Cabbage 9% 4 Champasak Paksong Agriculture Community Buying and Promotion Cooperative (ACBPC) PO#004 Coffee 59 57 2 15 15 0 Cabbage 25% 5 Champasak Paksong Jai Coffee Farmers’ Cooperative (JCFC) PO#005 Coffee 154 152 2 44 44 0 Cabbage 29% 6 Champasak Paksong Agro Cooperative Agriculture Product Thongxayphatthana (ACAPTP) PO#006 Coffee 25 25 0 13 13 0 Cabbage 52% 7 Champasak Paksong Cooperative of Sustainable Eco Coffee (CSEC) PO#007 Coffee 13 12 1 1 1 0 Cabbage 8% 8 Champasak Paksong Cooperative of Agricultural and Southern Rejoin (COSR) PO#008 Coffee 29 18 11 0% 125 # Province District Farmer group Name (Eng) FG code # of farmers joining FG # of farmers actually joining project activities (project's beneficiaries) % of actual beneficiaries vs interested farmers 9 Champasak Paksong Jingjhai Quality Coffee Producers Cooperative (JQCPC) PO#009 Coffee 27 21 6 3 3 0 Cabbage 11% 9 Champasak Paksong Jingjhai Quality Coffee Producers Cooperative (JQCPC) PO#009 2 2 0 Sacha Inchi 7% 10 Champasak Paksong Quality Coffee Producer Group Sepian Village PO#010 Coffee 22 22 0% 11 Champasak Paksong Sacha Inchi Production Group Nongkali Village PO#011 Sacha Inchi 27 23 4 23 19 4 Sacha Inchi 85% 12 Champasak Paksong Sacha Inchi Production Group Thongkalong Village PO#012 Sacha Inchi 24 24 2 1 1 Sacha Inchi 8% 13 Champasak Paksong Quality Cabbage Production Group (GAP) Huayset Village PO#013 Cabbage 12 11 1 14 12 2 Cabbage 117% 14 Sekong Thateng GAP Quality Cabbage Production Group Huaysaiy Village PO#014 Cabbage 8 7 1 10 10 0 Cabbage 125% 15 Sekong Thateng GAP Quality Cabbage Production Group Kongtayoon Village PO#015 Cabbage 11 9 2 17 11 6 Cabbage 155% 16 Sekong Thateng GAP Quality Cabbage Production Group Lakkhao Village PO#016 Cabbage 11 6 5 21 10 11 Cabbage 191% 17 Sekong Thateng GAP Cassava Production Group Chounla Village PO#017 Cassava 23 22 1 34 29 5 Cassava 148% 18 Sekong Thateng GAP Cassava Production Group Kokphung nuea Village PO#018 Cassava 52 41 11 54 42 12 Cassava 104% 126 # Province District Farmer group Name (Eng) FG code # of farmers joining FG # of farmers actually joining project activities (project's beneficiaries) % of actual beneficiaries vs interested farmers 19 Sekong Thateng GAP Cassava Production Group Kandone Village PO#019 Cassava 35 31 4 34 26 8 Cassava 97% 20 Sekong Thateng GAP Cassava Production Group Lidsamphan Village PO#020 Cassava 11 9 2 10 8 2 Cassava 91% 21 Sekong Thateng GAP Cassava Production Group Huaysaiy Village PO#021 Cassava 16 10 6 16 11 5 Cassava 100% 22 Sekong Thateng GAP Cassava Production Group Kafe Village PO#022 Cassava 21 19 2 23 20 3 Cassava 110% 23 Sekong Thateng GAP Cassava Production Group Kapeu Village PO#023 Cassava 45 35 10 42 32 10 Cassava 93% 23 Sekong Thateng GAP Cassava Production Group Kapeu Village PO#023 4 3 1 Sacha Inchi 9% 24 Sekong Thateng GAP Cassava Production Group Chakam Yai Village PO#024 Cassava 29 23 6 33 24 9 Cassava 114% 25 Sekong Thateng GAP Cassava Production Group Kamkok Village PO#025 Cassava 42 36 6 58 45 13 Cassava 138% 26 Sekong Thateng GAP Quality Coffee Production Group Chounla Village PO#026 Coffee 18 18 11 10 1 Coffee 61% 27 Sekong Thateng GAP Quality Coffee Production Group Kokphung nuea Village PO#027 Coffee 38 6 31 25 6 Coffee 82% 28 Sekong Thateng GAP Quality Coffee Production Group Hua Xe Don Village PO#028 Coffee 12 10 2 11 9 2 Coffee 92% 127 # Province District Farmer group Name (Eng) FG code # of farmers joining FG # of farmers actually joining project activities (project's beneficiaries) % of actual beneficiaries vs interested farmers 29 Sekong Thateng GAP Quality Coffee Production Group Chakam Yai Village PO#029 Coffee 11 9 2 10 7 3 Coffee 91% 30 Sekong Thateng GAP Quality Coffee Production Group Huaysaiy Village PO#030 Coffee 20 19 1 20 19 1 Coffee 100% 31 Sekong Thateng GAP Quality Coffee Production Group Kongtayoon Village PO#031 Coffee 48 47 1 23 23 0 Coffee 48% 32 Sekong Thateng OA Quality Coffee Production Group Nongkan Village PO#032 Coffee 63 53 10 65 56 9 Coffee 103% 33 Sekong Thateng OA Quality Coffee Production Group Kafe Village PO#033 Coffee 13 11 2 9 9 0 Coffee 69% 34 Sekong Thateng GAP Quality Coffee Production Group Huaydam Mai Village PO#034 Coffee 49 46 3 50 47 3 Coffee 102% 35 Sekong Thateng OA Quality Coffee Production Group Kamkok Village PO#035 Coffee 21 18 3 16 14 2 Coffee 76% 36 Sekong Thateng GAP Quality Coffee Production Group Lakkhao Village PO#036 Coffee 15 10 5 11 8 3 Coffee 73% 37 Sekong Thateng GAP Quality Coffee Production Group Thateng Tai Village PO#037 Coffee 11 7 4 10 6 4 Coffee 91% 38 Salavan Laongam Coffee Producer Group Nongchoua Village PO#038 Coffee 67 59 8 40 34 6 Coffee 60% 39 Salavan Laongam Coffee Producer Group Lao Ya Village PO#039 Coffee 39 27 12 46 34 12 Coffee 118% 40 Salavan Laongam Coffee Producer Group Phakkud Yai Village PO#040 Coffee 17 17 0 30 29 1 Coffee 176% 128 # Province District Farmer group Name (Eng) FG code # of farmers joining FG # of farmers actually joining project activities (project's beneficiaries) % of actual beneficiaries vs interested farmers 41 Salavan Laongam Coffee Producer Group Dong Village PO#041 Coffee 16 15 1 16 15 1 Coffee 100% 42 Salavan Laongam Coffee Producer Group Vang Yao Village PO#042 Coffee 10 6 4 11 6 5 Coffee 110% 43 Salavan Laongam Cassava Producer Group Lavad Village PO#043 Cassava 49 46 3 49 29 20 Cassava 100% 44 Salavan Laongam Cassava Producer Group Temephopum Village PO#044 Cassava 57 53 4 57 52 5 Cassava 100% 45 Salavan Laongam Cassava Producer Group Temesangthong Village PO#045 Cassava 52 43 9 51 36 15 Cassava 98% 46 Salavan Laongam Cassava Producer Group Dongbang Village PO#046 Cassava 22 22 0 18 18 0 Cassava 82% 47 Salavan Laongam Cassava Producer Group Ngiew Village PO#047 Cassava 23 21 2 16 12 4 Cassava 70% 48 Salavan Laongam Cassava Producer Group Phornuan Village PO#048 Cassava 53 51 2 47 43 4 Cassava 89% 49 Salavan Laongam Cassava Producer Group Bak Yai Village PO#049 Cassava 41 36 5 40 19 21 Cassava 98% 50 Salavan Laongam Cabbage Producer Group Huayseng Village PO#050 Cabbage 15 10 5 15 10 5 Cabbage 100% 51 Salavan Laongam Cabbage Producer Group Phorkhem Village PO#051 Cabbage 15 15 0 16 16 0 Cabbage 107% 52 Salavan Laongam Cabbage Producer Group Dasia Noy Village PO#052 Cabbage 7 7 0 7 7 0 Cabbage 100% 53 Salavan Laongam Cabbage Producer Group Nongtom Village PO#053 Cabbage 10 7 3 10 8 2 Cabbage 100% 54 Vientiane Capital Sangthong Cassava Production Group Yai Nachaleun Village PO#054 Cassava 18 18 0 23 10 13 Cassava 128% 55 Vientiane Capital Sangthong Cassava Production Group Nongboua village PO#055 Cassava 19 16 3 16 10 6 Cassava 84% 56 Vientiane Capital Parkngum Cassava Production Group Pakkuang Village PO#056 Cassava 48 46 2 88 66 22 Cassava 183% 129 # Province District Farmer group Name (Eng) FG code # of farmers joining FG # of farmers actually joining project activities (project's beneficiaries) % of actual beneficiaries vs interested farmers 57 Vientiane Capital Parkngum Cassava Production Group Fangdeng Village PO#057 Cassava 53 37 16 47 27 20 Cassava 89% 58 Vientiane Capital Parkngum Cassava Production Group Xienglae Na Village PO#058 Cassava 42 31 11 25 14 11 Cassava 60% 59 Vientiane Capital Parkngum Cassava Production Group Nongphouvieng Village PO#059 Cassava 26 17 9 16 11 5 Cassava 62% 60 Vientiane Capital Parkngum Cassava Production Group Xienglae Tha Village PO#060 Cassava 54 48 6 48 36 12 Cassava 89% 61 Vientiane Capital Xaythany Cassava Production Group Phonxay Village PO#061 Cassava 40 39 1 29 26 3 Cassava 73% 62 Vientiane Capital Xaythany Cassava Production GroupThongmang Village PO#062 Cassava 13 12 1 21 19 2 Cassava 162% 63 Vientiane Capital Xaythany Cassava Production Group Phonhaikham Village PO#063 Cassava 13 12 1 12 11 1 Cassava 92% 64 Vientiane Capital Xaythany Cassava Production Group Hadkieng Village PO#064 Cassava 8 6 2 9 5 4 Cassava 113% 65 Vientiane Capital Xaythany Cassava Production Group Thadindeng Tai Village PO#065 Cassava 19 19 0 18 18 0 Cassava 95% 66 Vientiane province Kasy Cabbage Production Group Viengkham Village PO#066 Cabbage 26 5 21 28 6 22 Cassava 108% 67 Vientiane province Kasy Cabbage Production Group Viengkeo Village PO#067 Cabbage 16 11 5 14 9 5 Cabbage 88% 68 Vientiane province Kasy Cabbage Production Group Bouam Phuk Village PO#068 Cabbage 12 6 6 21 6 15 Cabbage 175% 69 Vientiane province Kasy Cabbage Production Group Phoukham Village PO#069 Cabbage 22 12 25 9 16 Cabbage 114% 70 Vientiane province Kasy Cabbage Production Group Houay Hok Village PO#070 Cabbage 15 6 9 14 6 8 Cabbage 93% 71 Vientiane province Kasy Cabbage Production Group Jieng Village PO#071 Cabbage 8 7 1 9 8 1 Cabbage 113% 130 # Province District Farmer group Name (Eng) FG code # of farmers joining FG # of farmers actually joining project activities (project's beneficiaries) % of actual beneficiaries vs interested farmers 72 Vientiane province Kasy Cabbage Production Group Phaxang Village PO#072 Cabbage 25 19 6 33 21 12 Cabbage 132% 73 Vientiane province Vangvieng Cabbage Production Group Phatang Village PO#073 Cabbage 49 31 18 39 25 14 Cabbage 80% 74 Vientiane province Vangvieng Cabbage Production Group Phahom Village PO#074 Cabbage 21 14 7 34 18 16 Cabbage 162% 75 Vientiane province Vangvieng Cabbage Production Group Naphadeng Village PO#075 Cabbage 31 10 21 36 13 23 Cabbage 116% 76 Vientiane province Vangvieng Cabbage Production Group Somsavad Village PO#076 Cabbage 9 8 1 8 5 3 Cabbage 89% 77 Champasak Paksong GAP Cabbage Thongkalong Village PO#078 Cabbage 13 12 1 38 18 20 Cabbage 292% 78 Champasak Paksong GAP Cabbage Kongtoun Village PO#079 Cabbage 31 30 1 31 22 9 Cabbage 100% 79 Vientiane province Vangvieng GAP Cabbage Keokuang 1 Village PO#080 Cabbage 46 32 14 0% 80 Vientiane province Vangvieng GAP Cabbage Keokuang 2 Village PO#081 Cabbage 31 28 3 0% 81 Champasak, Sekong, Saravan Paksong, Thateng, Laongam Sacha Inchi farmers under Mai Savanh Lao Sole Co.,Ltd NO#000 Sacha Inchi 214 173 41 143 114 29 Sacha Inchi 67% 131 Direct Beneficiaries Disaggregated by District # Province District # Total # Female % female % by district Commodity type 1 Vientiane Capital Parkngum 224 70 31% 11% Cassava 2 Vientiane Capital Xaythany 89 10 11% 5% Cassava 3 Vientiane Capital Sangthong 39 19 49% 2% Cassava 4 Vientiane province Vangvieng 117 56 48% 6% Cabbage 5 Vientiane province Kasy 144 79 55% 7% Cabbage 6 Champasak Paksong 278 58 21% 14% Cabbage, sacha inchi 7 Saravan Lao Ngarm 481 103 21% 24% Cabbage, cassava, coffee, sacha inchi 8 Sekong Thateng 589 115 20% 30% Cabbage, cassava, coffee, sacha inchi Grand Total 1,965 511 26% 100% 132 Annex 8. The Calculation of Sampling Weights 133 134 135